ops: make the drift check see rulesets, not just branch protection (#36)

The check reported branch protection fully green while a repository ruleset on
the same branch required two status checks that no workflow produces —
'Build, Lint & Test' and 'All checks passed', names left behind when the jobs
were renamed. Nothing could merge without --admin, and the report said
everything was fine.

That is the worst shape a check can take: confident and blind. A branch can be
governed by classic protection and a ruleset at once, independently, and the
stricter wins. Reading one and calling it the answer is how a broken gate stays
invisible for months while the team learns to route around it.

--check now reads active rulesets too, and compares each required check against
what recent pull requests actually report. A rule waiting on a check that never
arrives does not slow a branch — it closes it.

Sampling uses recent PR heads rather than main's head, because the aggregate
gate only runs on pull_request; sampling main would report it as never seen and
manufacture the opposite false alarm.
This commit is contained in:
SatoshiQaziMuhammed
2026-08-01 21:04:26 -07:00
committed by GitHub
parent 39b71a428e
commit b289bbc0ae
2 changed files with 70 additions and 0 deletions
+14
View File
@@ -131,6 +131,20 @@ Get `<ENV_ID>` from `gh api repos/:owner/:repo/actions/runs/<RUN_ID>/pending_dep
This came up on 2026-07-31: a stale run sat in `waiting` and blocked the deploy
of the run behind it until it was rejected.
### Two layers, one branch
A branch can be governed by classic protection **and** a repository ruleset at
the same time, independently, and the stricter of the two wins. That is how
`main` here spent months unmergeable without anyone knowing why: classic
protection asked for `CI Gate ✅`, while a ruleset on the same branch asked for
`Build, Lint & Test` and `All checks passed` — two names no workflow has
produced since the jobs were renamed. Every merge went through `--admin`, and
the habit read as "review is slow" rather than "the gate is broken".
`--check` now reports both, and flags a required check that nothing reports.
A rule demanding a check that never arrives does not slow a branch down; it
closes it, silently, and the drift report will say everything is fine.
### Reviewer
`REVIEWER_ID` is a numeric user id rather than a login, because the API takes ids
+56
View File
@@ -102,6 +102,62 @@ for label, (got, exp) in want.items():
PY
fi
# Rulesets are a second, independent layer over the same branch — and this
# script was blind to them. On 2026-08-01 it reported branch protection fully
# green while a repository ruleset on the same branch demanded two status
# checks that no workflow produces ("Build, Lint & Test", "All checks
# passed"). Nothing could merge without --admin, and the drift report said
# everything was fine. Two layers that disagree are worse than one, so the
# check has to see both.
echo "── rulesets ($BRANCH)"
rs="$(gh api "repos/$REPO/rulesets" 2>/dev/null || echo '[]')"
python3 - "$REPO" "$rs" <<'PY2'
import json, subprocess, sys
repo, raw = sys.argv[1], sys.argv[2]
try:
sets = json.loads(raw)
except Exception:
sets = []
active = [r for r in sets if r.get('enforcement') == 'active']
if not active:
print(' \033[32m✔\033[0m no active ruleset (branch protection is the only layer)')
raise SystemExit
# Every check a ruleset demands must be a check something actually reports,
# otherwise the branch is permanently unmergeable and the reason is invisible.
# Sample recent pull request heads, not main. Required checks are the ones that
# gate a PR, and several of them (the aggregate gate especially) only run on
# pull_request — looking at main's head would report them as never seen.
reported = set()
try:
prs = json.loads(subprocess.run(
['gh', 'api', f'repos/{repo}/pulls?state=all&per_page=5'],
capture_output=True, text=True).stdout)
for pr in prs[:5]:
runs = json.loads(subprocess.run(
['gh', 'api', f"repos/{repo}/commits/{pr['head']['sha']}/check-runs"],
capture_output=True, text=True).stdout)
reported |= {c['name'] for c in runs.get('check_runs', [])}
except Exception:
pass
for r in active:
detail = json.loads(subprocess.run(
['gh', 'api', f"repos/{repo}/rulesets/{r['id']}"], capture_output=True, text=True).stdout)
for rule in detail.get('rules', []):
if rule['type'] == 'required_status_checks':
want = [c['context'] for c in rule['parameters'].get('required_status_checks', [])]
missing = [c for c in want if reported and c not in reported]
mark = '\033[31m✗\033[0m' if missing else '\033[32m✔\033[0m'
note = f" (never reported: {missing})" if missing else ''
print(f" {mark} '{r['name']}' requires {want}{note}")
if rule['type'] == 'pull_request':
n = rule['parameters'].get('required_approving_review_count')
mark = '\033[32m✔\033[0m' if n == 0 else '\033[31m✗\033[0m'
extra = '' if n == 0 else ' (expected 0 — see README)'
print(f" {mark} '{r['name']}' approvals: {n}{extra}")
PY2
echo "── environment ($ENVIRONMENT)"
env_cur="$(gh api "repos/$REPO/environments/$ENVIRONMENT" 2>/dev/null || echo '{}')"
python3 - "$env_cur" <<'PY'