mirror of
https://github.com/pezkuwichain/revive.git
synced 2026-10-20 18:56:17 +00:00
Fix rate-limited cloning of LLVM submodule (#569)
## Description We started seeing rate-limit `429` errors when cloning LLVM (as seen in e.g. [this failed run](https://github.com/paritytech/revive/actions/runs/28924583524/job/85809154518) and [this failed run](https://github.com/paritytech/revive/actions/runs/28926259175/job/85821336465?pr=564)). `llvm/llvm-project` now seems to be heavily rate-limited by GitHub. (Authenticating the clone did not seem to have much effect.) Even though our resolc build steps only do a treeless clone, all build steps still did the same clone. ### Changes * I've instead introduced a `.llvm-version` file that the `get-llvm` action reads from, eliminating the network request all together. * `.llvm-version` is meant to be kept in sync with the LLVM submodule version we use. This is enforced by: * Introduced a `check-llvm-version.yml` workflow * Runs once to check that the recorded version in `.llvm-version` coincides with the version detected from the submodule. * The submodule version detection *does* perform a network request as before, but this is only run once now (when relevant files change), rather than for all our resolc build jobs. * Since the differential test jobs require checking out our revive-differential-tests (rdt) submodule, those checkout steps cloned not only rdt, but also LLVM. I've updated this to not clone LLVM at all.
This commit is contained in:
4 files changed
+87
-29
No files matched your search
@@ -14,48 +14,34 @@ inputs:
|
|||||||
runs:
|
runs:
|
||||||
using: "composite"
|
using: "composite"
|
||||||
steps:
|
steps:
|
||||||
- name: detect llvm version from submodule
|
- name: check override version
|
||||||
id: detect-version
|
id: override
|
||||||
shell: bash
|
shell: bash
|
||||||
env:
|
env:
|
||||||
# To test a draft LLVM release across every CI workflow, set this to the full
|
# To test a draft LLVM release across every CI workflow, set this to the full
|
||||||
# draft name (e.g. "llvm-21.1.8-revive.0f80912") and revert to "" before merging.
|
# draft name (e.g. "llvm-21.1.8-revive.0f80912") and revert to "" before merging.
|
||||||
OVERRIDE_VERSION: ""
|
OVERRIDE_VERSION: ""
|
||||||
|
run: echo "version=${OVERRIDE_VERSION}" | tee -a "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
- name: resolve llvm version
|
||||||
|
id: resolve-version
|
||||||
|
if: ${{ steps.override.outputs.version == '' }}
|
||||||
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
if [ -n "${OVERRIDE_VERSION}" ]; then
|
if [ ! -s .llvm-version ]; then
|
||||||
echo "Using override version: ${OVERRIDE_VERSION}"
|
echo "::error file=.llvm-version::Missing or empty .llvm-version. It must contain the version matching the llvm submodule pin."
|
||||||
echo "version_prefix=${OVERRIDE_VERSION}" >> "$GITHUB_OUTPUT"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
LLVM_SHA=$(git ls-tree HEAD llvm | awk '{print $3}')
|
|
||||||
echo "LLVM_SHA: $LLVM_SHA"
|
|
||||||
LLVM_URL=$(git config -f .gitmodules submodule.llvm.url)
|
|
||||||
echo "LLVM_URL: $LLVM_URL"
|
|
||||||
if [ -z "$LLVM_SHA" ] || [ -z "$LLVM_URL" ]; then
|
|
||||||
echo "::error::Could not resolve LLVM submodule SHA or URL from this repo." >&2
|
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
echo "version_prefix=$(cat .llvm-version)" | tee -a "$GITHUB_OUTPUT"
|
||||||
# Do a treeless clone to avoid unnecessarily fetching source files.
|
|
||||||
git clone --filter=tree:0 --no-checkout "$LLVM_URL" llvm-tmp
|
|
||||||
# Try to get the most recent tag (e.g., "llvmorg-18.1.8").
|
|
||||||
LLVM_TAG=$(git -C llvm-tmp describe --tags --abbrev=0 "$LLVM_SHA")
|
|
||||||
rm -rf llvm-tmp
|
|
||||||
|
|
||||||
# Convert "llvmorg-x.y.z" to "llvm-x.y.z".
|
|
||||||
LLVM_VERSION=$(echo "$LLVM_TAG" | sed 's/^llvmorg-/llvm-/')
|
|
||||||
echo "Detected LLVM version from submodule pin: $LLVM_VERSION"
|
|
||||||
echo "version_prefix=$LLVM_VERSION" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: find asset
|
- name: find asset
|
||||||
id: find
|
id: find
|
||||||
uses: actions/github-script@v7
|
uses: actions/github-script@v7
|
||||||
env:
|
env:
|
||||||
target: ${{ inputs.target }}
|
target: ${{ inputs.target }}
|
||||||
version_prefix: ${{ steps.detect-version.outputs.version_prefix }}
|
version_prefix: ${{ steps.override.outputs.version || steps.resolve-version.outputs.version_prefix }}
|
||||||
with:
|
with:
|
||||||
result-encoding: string
|
result-encoding: string
|
||||||
script: |
|
script: |
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# Guards the checked-in .llvm-version file, which CI reads (via the get-llvm
|
||||||
|
# action) to select the prebuilt LLVM release to build against. Whenever the
|
||||||
|
# llvm submodule pin or the file changes, this derives the version from the
|
||||||
|
# pinned commit and fails if the file was not updated to match.
|
||||||
|
|
||||||
|
name: Check LLVM Version
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: ["main"]
|
||||||
|
paths:
|
||||||
|
- "llvm"
|
||||||
|
- ".llvm-version"
|
||||||
|
- ".gitmodules"
|
||||||
|
- ".github/workflows/check-llvm-version.yml"
|
||||||
|
- ".github/actions/get-llvm/action.yml"
|
||||||
|
pull_request:
|
||||||
|
types: [opened, synchronize]
|
||||||
|
paths:
|
||||||
|
- "llvm"
|
||||||
|
- ".llvm-version"
|
||||||
|
- ".gitmodules"
|
||||||
|
- ".github/workflows/check-llvm-version.yml"
|
||||||
|
- ".github/actions/get-llvm/action.yml"
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
check-llvm-version:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Checkout revive
|
||||||
|
uses: actions/checkout@v6
|
||||||
|
|
||||||
|
- name: Verify .llvm-version Matches the Submodule Pin
|
||||||
|
env:
|
||||||
|
GH_TOKEN: ${{ github.token }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
LLVM_SHA=$(git ls-tree HEAD llvm | awk '{print $3}')
|
||||||
|
LLVM_URL=$(git config -f .gitmodules submodule.llvm.url)
|
||||||
|
|
||||||
|
# Do a treeless clone to avoid unnecessarily fetching source files.
|
||||||
|
git clone --filter=tree:0 --no-checkout \
|
||||||
|
"https://x-access-token:${GH_TOKEN}@${LLVM_URL#https://}" llvm-tmp
|
||||||
|
|
||||||
|
# Try to get the most recent tag (e.g., "llvmorg-18.1.8").
|
||||||
|
DETECTED_LLVM_TAG=$(git -C llvm-tmp describe --tags --abbrev=0 "$LLVM_SHA")
|
||||||
|
|
||||||
|
# Convert "llvmorg-x.y.z" to "llvm-x.y.z".
|
||||||
|
DETECTED_LLVM_VERSION=$(echo "$DETECTED_LLVM_TAG" | sed 's/^llvmorg-/llvm-/')
|
||||||
|
RECORDED_LLVM_VERSION=$(cat .llvm-version)
|
||||||
|
|
||||||
|
if [ "$DETECTED_LLVM_VERSION" != "$RECORDED_LLVM_VERSION" ]; then
|
||||||
|
echo "::error file=.llvm-version::.llvm-version contains '$RECORDED_LLVM_VERSION' but the submodule pin describes to '$DETECTED_LLVM_VERSION'."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "OK: .llvm-version ($RECORDED_LLVM_VERSION) matches the submodule pin."
|
||||||
@@ -85,7 +85,12 @@ jobs:
|
|||||||
uses: actions/checkout@v6
|
uses: actions/checkout@v6
|
||||||
with:
|
with:
|
||||||
path: revive
|
path: revive
|
||||||
submodules: recursive
|
# Submodules are initialized selectively below to avoid cloning
|
||||||
|
# LLVM which is both large and prone to GitHub rate limiting.
|
||||||
|
submodules: false
|
||||||
|
|
||||||
|
- name: Checkout revive-differential-tests
|
||||||
|
run: git -C revive submodule update --init --recursive --depth=1 revive-differential-tests
|
||||||
|
|
||||||
- name: Download Binary
|
- name: Download Binary
|
||||||
uses: actions/download-artifact@v7
|
uses: actions/download-artifact@v7
|
||||||
@@ -124,7 +129,12 @@ jobs:
|
|||||||
uses: actions/checkout@v6
|
uses: actions/checkout@v6
|
||||||
with:
|
with:
|
||||||
path: revive
|
path: revive
|
||||||
submodules: true
|
# Submodules are initialized selectively below to avoid cloning
|
||||||
|
# LLVM which is both large and prone to GitHub rate limiting.
|
||||||
|
submodules: false
|
||||||
|
|
||||||
|
- name: Checkout revive-differential-tests
|
||||||
|
run: git -C revive submodule update --init --depth=1 revive-differential-tests
|
||||||
|
|
||||||
- name: Compare Bytecode Hashes
|
- name: Compare Bytecode Hashes
|
||||||
uses: ./revive/revive-differential-tests/.github/actions/compare-bytecode-hashes
|
uses: ./revive/revive-differential-tests/.github/actions/compare-bytecode-hashes
|
||||||
@@ -154,7 +164,11 @@ jobs:
|
|||||||
uses: actions/checkout@v6
|
uses: actions/checkout@v6
|
||||||
with:
|
with:
|
||||||
path: revive
|
path: revive
|
||||||
submodules: recursive
|
# Submodules are initialized selectively below to avoid cloning LLVM.
|
||||||
|
submodules: false
|
||||||
|
|
||||||
|
- name: Checkout revive-differential-tests
|
||||||
|
run: git -C revive submodule update --init --recursive --depth=1 revive-differential-tests
|
||||||
|
|
||||||
- name: Download Binary
|
- name: Download Binary
|
||||||
uses: actions/download-artifact@v7
|
uses: actions/download-artifact@v7
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
llvm-22.1.4
|
||||||
Reference in new issue
Block a user