Implement verification of reproducible builds and run differential E2E tests (#468)

## Description This adds verification for reproducible builds (via [revive-differential-tests](https://github.com/paritytech/revive-differential-tests) testing framework) by: 1. Compiling [contracts](https://github.com/paritytech/resolc-compiler-tests/tree/main/fixtures/solidity) and generating hashes from each bytecode * With the resolc builds: * linux musl * macos universal * windows * (future todo: wasm32 emscripten) * With optimization levels `0`, `3`, `z` for each contract 2. Exporting the hashes 3. Comparing the hashes * For each optimization level: * Checks if each hash for a specific contract is identical across the platforms This PR also runs the differential E2E tests (via `retester`) with the resolc linux musl build. All of the above only runs if our `test` workflow succeeds. ## Related This PR initially implemented the above in this repo but has now been implemented into the separate testing framework instead. The most relevant PRs implemented alongside this PR as prerequisites are: * `revive-differential-tests`: * [Generate bytecode hashes included in the report ](https://github.com/paritytech/revive-differential-tests/pull/238) * [Fix handling of resolc's optimizer `mode` and allow setting solc optimizer enabled](https://github.com/paritytech/revive-differential-tests/pull/243) * [Fix solc and resolc optimization settings](https://github.com/paritytech/revive-differential-tests/pull/253) * [Implement a `compile` context/subcommand for standalone compilation mode](https://github.com/paritytech/revive-differential-tests/pull/239) * [Implement support for `--mode` CLI argument for `compile`](https://github.com/paritytech/revive-differential-tests/pull/246) * [Implement `export-hashes` and `compare-hashes` subcommands for report processor ](https://github.com/paritytech/revive-differential-tests/pull/250) * [Implement GH actions for compiling, exporting and comparing hashes](https://github.com/paritytech/revive-differential-tests/pull/251) * [Gate zombienet-sdk behind `cfg(unix)` to fix Windows builds](https://github.com/paritytech/revive-differential-tests/pull/254) * [Link library-using contracts with dummy addresses for `--compile` command](https://github.com/paritytech/revive-differential-tests/pull/267) * `revive`: * [Build LLVM for Windows using MSVC-compatible Clang](https://github.com/paritytech/revive/pull/497) * [Apply LLVM patches for deterministic operand order and correct peephole optimization for `minsize` ](https://github.com/paritytech/revive/pull/500) * Investigation of comparison mismatches in third-party dependencies: * `llvm-project`: * [`SelectionDAG` type legalization produces different operand ordering for `i256` multiply across platforms](https://github.com/llvm/llvm-project/issues/190148) * `solidity`: * [Windows absolute paths in standard JSON break import resolution and library linking](https://github.com/argotorg/solidity/issues/16579) ## Notes ### Failed compilations Since [resolc-compiler-tests](https://github.com/paritytech/resolc-compiler-tests) currently includes contracts that are expected to fail resolc compilation (e.g. due to unsupported instructions, contracts that are expected to work with solc for evm/revm), the compilation workflow job is not expected to fail if compilation fails for a contract. Instead, if a contract compiles with one build but not with another, a hash will be missing and will thereby generate a mismatch when comparing all hashes, failing at the verification step. ## Resolved Issues Closes [#175](https://github.com/paritytech/revive/issues/175)
2026-10-21 03:06:17 +00:00 · 2026-04-14 16:58:58 +02:00
1 parent f887e75545
commit 733e01c78e
@@ -25,6 +25,10 @@ on:
        value: ${{ jobs.build.outputs.resolc-x86_64-apple-darwin_url }}
      resolc-x86_64-apple-darwin_sha:
        value: ${{ jobs.build.outputs.resolc-x86_64-apple-darwin_sha }}
+      resolc-universal-apple-darwin_url:
+        value: ${{ jobs.create-macos-universal.outputs.resolc-universal-apple-darwin_url }}
+      resolc-universal-apple-darwin_sha:
+        value: ${{ jobs.create-macos-universal.outputs.resolc-universal-apple-darwin_sha }}
      resolc-x86_64-pc-windows-msvc_url:
        value: ${{ jobs.build.outputs.resolc-x86_64-pc-windows-msvc_url }}
      resolc-x86_64-pc-windows-msvc_sha:
@@ -37,6 +41,9 @@ on:
 env:
  CARGO_TERM_COLOR: always
  RUST_MUSL_CROSS_IMAGE: messense/rust-musl-cross@sha256:2a8837c43bf12e246f1ebd05191de9ee27fcd22f9ca81511ccd4cf75dc16d71c
+  # An empty string is interpreted as the latest published release (default behavior).
+  # To test a specific draft release, use the full name (e.g. "llvm-21.1.8-revive.5789e11").
+  LLVM_VERSION: ""

 jobs:
  build:
@@ -51,6 +58,23 @@ jobs:
      resolc-x86_64-apple-darwin_sha: ${{ steps.set-output.outputs.resolc-x86_64-apple-darwin_sha }}
      resolc-x86_64-pc-windows-msvc_url: ${{ steps.set-output.outputs.resolc-x86_64-pc-windows-msvc_url }}
      resolc-x86_64-pc-windows-msvc_sha: ${{ steps.set-output.outputs.resolc-x86_64-pc-windows-msvc_sha }}
+    # Queue jobs for the same target and commit instead of running in parallel or cancelling
+    # the first job. This allows the second workflow to use cached builds from the first if
+    # it is triggered by the same commit.
+    #
+    # Example: Workflow A and Workflow B both call this workflow for the same commit:
+    #
+    #   Time | Workflow A                | Workflow B
+    #   -----|---------------------------|---------------------------
+    #   T1   | Job starts, cache miss    |
+    #   T2   | Building...               | Job queued (same group)
+    #   T3   | Build done, cache saved   | Job queued
+    #   T4   | Job finishes              | Job starts, cache hit
+    #   T5   |                           | Skips build, job finishes
+    concurrency:
+      # Use `reusable-build` and not `${{ github.workflow }}` here so that callers share the same concurrency group.
+      group: reusable-build-${{ matrix.target }}-${{ github.sha }}
+      cancel-in-progress: false
    strategy:
      matrix:
        target:
@@ -81,18 +105,31 @@ jobs:
        if: runner.os == 'Windows'
        run: git config --system core.longpaths true
      - uses: actions/checkout@v4
+
+      # Cache build artifacts by target and commit (restores if available, saves on cache miss).
+      - name: Check Build Cache
+        id: cache
+        uses: actions/cache@v5
+        with:
+          # Use glob to match binaries with and without extensions.
+          path: resolc-${{ matrix.target }}*
+          key: build-${{ matrix.target }}-${{ github.sha }}
+
      - uses: actions-rust-lang/setup-rust-toolchain@v1
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
        with:
          rustflags: ""
          cache-key: ${{ matrix.target }}

      - name: Download LLVM
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
        uses: ./.github/actions/get-llvm
        with:
          target: ${{ matrix.target }}
+          version: ${{ env.LLVM_VERSION }}

      - name: Build (Native)
-        if: ${{ matrix.type == 'native' }}
+        if: ${{ steps.cache.outputs.cache-hit != 'true' && matrix.type == 'native' }}
        shell: bash
        run: |
          export LLVM_SYS_211_PREFIX=$PWD/llvm-${{ matrix.target }}
@@ -100,7 +137,7 @@ jobs:
          mv target/release/resolc resolc-${{ matrix.target }} || mv target/release/resolc.exe resolc-${{ matrix.target }}.exe

      - name: Build (MUSL)
-        if: ${{ matrix.type == 'musl' }}
+        if: ${{ steps.cache.outputs.cache-hit != 'true' && matrix.type == 'musl' }}
        run: |
          docker run -v $PWD:/opt/revive $RUST_MUSL_CROSS_IMAGE /bin/bash -c "
           cd /opt/revive
@@ -137,37 +174,106 @@ jobs:
          echo "resolc-${{ matrix.target }}_url=${{ steps.artifact-upload-step.outputs.artifact-url }}" >> "$GITHUB_OUTPUT"
          echo "resolc-${{ matrix.target }}_sha=${{ steps.artifact-upload-step.outputs.artifact-digest }}" >> "$GITHUB_OUTPUT"

+  create-macos-universal:
+    needs: build
+    runs-on: macos-15
+    concurrency:
+      # Use `reusable-build` and not `${{ github.workflow }}` here so that callers share the same concurrency group.
+      group: reusable-build-universal-apple-darwin-${{ github.sha }}
+      cancel-in-progress: false
+    outputs:
+      resolc-universal-apple-darwin_url: ${{ steps.set-output.outputs.resolc-universal-apple-darwin_url }}
+      resolc-universal-apple-darwin_sha: ${{ steps.set-output.outputs.resolc-universal-apple-darwin_sha }}
+    steps:
+      - name: Check Build Cache
+        id: cache
+        uses: actions/cache@v5
+        with:
+          path: resolc-universal-apple-darwin
+          key: build-universal-apple-darwin-${{ github.sha }}
+
+      - name: Download macOS Artifacts
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
+        uses: actions/download-artifact@v4
+        with:
+          pattern: resolc-*-apple-darwin
+          merge-multiple: true
+
+      - name: Create macOS Fat Binary
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
+        run: |
+          lipo resolc-aarch64-apple-darwin resolc-x86_64-apple-darwin -create -output resolc-universal-apple-darwin
+          chmod +x resolc-universal-apple-darwin
+
+      - name: Upload Universal Binary
+        uses: actions/upload-artifact@v4
+        id: artifact-upload-step
+        with:
+          name: resolc-universal-apple-darwin
+          path: resolc-universal-apple-darwin
+          retention-days: ${{ inputs.retention_days }}
+
+      - name: Set output variables
+        if: ${{ !inputs.is_release }}
+        id: set-output
+        run: |
+          echo "resolc-universal-apple-darwin_url=${{ steps.artifact-upload-step.outputs.artifact-url }}" >> "$GITHUB_OUTPUT"
+          echo "resolc-universal-apple-darwin_sha=${{ steps.artifact-upload-step.outputs.artifact-digest }}" >> "$GITHUB_OUTPUT"
+
  build-wasm:
    runs-on: ubuntu-24.04
    outputs:
      resolc_web_js_url: ${{ steps.set-output.outputs.resolc_web_js_url }}
      resolc_web_js_sha: ${{ steps.set-output.outputs.resolc_web_js_sha }}
+    # Queue jobs for the same target and commit instead of running in parallel or cancelling
+    # the first job. This allows the second workflow to use cached builds from the first if
+    # it is triggered by the same commit.
+    concurrency:
+      # Use `reusable-build` and not `${{ github.workflow }}` here so that callers share the same concurrency group.
+      group: reusable-build-wasm32-unknown-emscripten-${{ github.sha }}
+      cancel-in-progress: false
    env:
      RELEASE_RESOLC_WASM_URI: https://github.com/paritytech/revive/releases/download/${{ github.ref_name }}/resolc.wasm
    steps:
      - uses: actions/checkout@v4
+
+      # Cache build artifacts by commit (restores if available, saves on cache miss).
+      - name: Check Build Cache
+        id: cache
+        uses: actions/cache@v5
+        with:
+          path: resolc-wasm32-unknown-emscripten
+          key: build-wasm32-unknown-emscripten-${{ github.sha }}
+
      # Pin to 1.92.0 until LLVM WASM libraries are rebuilt with -fwasm-exceptions
      # See: https://github.com/paritytech/revive/issues/XXX
      - uses: actions-rust-lang/setup-rust-toolchain@v1
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
        with:
          toolchain: "1.92.0"
          target: wasm32-unknown-emscripten
          rustflags: ""

      - name: Download Host LLVM
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
        uses: ./.github/actions/get-llvm
        with:
          target: x86_64-unknown-linux-gnu
+          version: ${{ env.LLVM_VERSION }}

      - name: Download Wasm LLVM
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
        uses: ./.github/actions/get-llvm
        with:
          target: wasm32-unknown-emscripten
+          version: ${{ env.LLVM_VERSION }}

      - name: Download EMSDK
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
        uses: ./.github/actions/get-emsdk

      - name: Build
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
        run: |
          export LLVM_SYS_211_PREFIX=$PWD/llvm-x86_64-unknown-linux-gnu
          export REVIVE_LLVM_TARGET_PREFIX=$PWD/llvm-wasm32-unknown-emscripten
@@ -175,6 +281,14 @@ jobs:
          make install-wasm
          chmod -x ./target/wasm32-unknown-emscripten/release/resolc.wasm

+      - name: Compress Artifact
+        if: ${{ steps.cache.outputs.cache-hit != 'true' }}
+        run: |
+          mkdir -p resolc-wasm32-unknown-emscripten
+          mv ./target/wasm32-unknown-emscripten/release/resolc.js ./resolc-wasm32-unknown-emscripten/
+          mv ./target/wasm32-unknown-emscripten/release/resolc.wasm ./resolc-wasm32-unknown-emscripten/
+          mv ./target/wasm32-unknown-emscripten/release/resolc_web.js ./resolc-wasm32-unknown-emscripten/
+
      - name: Set Up Node.js
        uses: actions/setup-node@v3
        with:
@@ -185,8 +299,11 @@ jobs:
          mkdir -p solc
          curl -sSLo solc/soljson.js https://github.com/argotorg/solidity/releases/download/v0.8.34/soljson.js
          node -e "
-          const soljson = require('solc/soljson');
-          const createRevive = require('./target/wasm32-unknown-emscripten/release/resolc.js');
+          // Make sure to require './solc/soljson' and not 'solc/soljson' in order to use
+          // the explicitly downloaded solc, rather than resolving an npm package potentially
+          // installed in prior workflow steps.
+          const soljson = require('./solc/soljson');
+          const createRevive = require('./resolc-wasm32-unknown-emscripten/resolc.js');

          const compiler = createRevive();
          compiler.soljson = soljson;
@@ -224,13 +341,6 @@ jobs:
          if(!bytecode.startsWith('50564d')) { process.exit(1); }
          "

-      - name: Compress Artifact
-        run: |
-          mkdir -p resolc-wasm32-unknown-emscripten
-          mv ./target/wasm32-unknown-emscripten/release/resolc.js ./resolc-wasm32-unknown-emscripten/
-          mv ./target/wasm32-unknown-emscripten/release/resolc.wasm ./resolc-wasm32-unknown-emscripten/
-          mv ./target/wasm32-unknown-emscripten/release/resolc_web.js ./resolc-wasm32-unknown-emscripten/
-
      - name: Upload artifacts (Release)
        if: ${{ inputs.is_release }}
        uses: actions/upload-artifact@v4