Files
pwap/.github/workflows/quality-gate.yml
pezkuwichain e4abb38757 ci: gate deploys on a GitHub Environment instead of a hand-rolled wait loop
Approval was implemented as a job that polled /tmp/pexsec-gates for 30 minutes
while holding a pwap-runner slot. On 2026-07-30 the request went out at 22:14,
was not seen in time, and at 22:44 the gate cancelled the deploy — #25 stayed
merged but never reached the database, and the only trace was a failed job.

Widening that timeout was the wrong fix, so this replaces the mechanism.

Deploys now gate on the `production` environment, which has SatoshiQaziMuhammed
as a required reviewer and is restricted to protected branches. GitHub holds the
run until it is approved:

  - no runner is occupied while waiting (the loop burned one for the full window)
  - the window is 30 days, not minutes, so a missed notification costs nothing
  - the approval is recorded — who approved which SHA, in deployment history
  - approval state lives outside the workflow, so a runner restart cannot lose it

telegram-gate becomes notify-deploy-pending: it sends one message and exits in
seconds, and cannot block anything. The message now links to the run rather than
carrying Approve/Cancel buttons, since approval happens in GitHub.

Only the four jobs that touch production carry the environment. Image builds and
the version bump stay ungated and run in parallel with the wait; nothing they
produce is user-visible until a deploy job runs.
2026-07-30 18:36:12 -07:00

1104 lines
44 KiB
YAML

name: Quality Gate & Deploy
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]
workflow_dispatch:
inputs:
rollback_to:
description: 'Rollback to git SHA (skips build, redeploys old image). Empty = normal deploy.'
required: false
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: write # version bump commit
packages: write # GHCR push
env:
VITE_SUPABASE_URL: ${{ secrets.VITE_SUPABASE_URL }}
VITE_SUPABASE_ANON_KEY: ${{ secrets.VITE_SUPABASE_ANON_KEY }}
REGISTRY: ghcr.io
IMAGE_NAME: pezkuwichain/pwap-web
BACKEND_IMAGE_NAME: pezkuwichain/pwap-indexer
jobs:
# ========================================
# WEB APP - LINT, TEST & BUILD
# ========================================
web:
name: Web App
runs-on: pwap-runner
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Checkout Pezkuwi-SDK (for docs generation)
run: |
git clone https://git.pezkuwichain.io/pezkuwichain/pezkuwi-sdk.git Pezkuwi-SDK || \
git clone https://github.com/pezkuwichain/pezkuwi-sdk.git Pezkuwi-SDK
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Cache npm dependencies
uses: actions/cache@v4
with:
path: web/node_modules
key: ${{ runner.os }}-web-${{ hashFiles('web/package-lock.json') }}
restore-keys: |
${{ runner.os }}-web-
- name: Install dependencies
working-directory: ./web
run: npm ci
- name: Run Linter
working-directory: ./web
run: npm run lint
# Typecheck gate. Currently non-blocking (continue-on-error) because the
# existing codebase still has pre-existing type errors. Once the count
# reaches zero, remove `continue-on-error` to make this a hard merge block.
- name: Typecheck
working-directory: ./web
continue-on-error: true
run: npm run typecheck
- name: Run Tests
working-directory: ./web
run: npm run test
- name: Build Project
working-directory: ./web
run: npm run build
env:
VITE_NETWORK: MAINNET
VITE_WS_ENDPOINT: wss://rpc.pezkuwichain.io
VITE_WS_ENDPOINT_FALLBACK_1: wss://mainnet.pezkuwichain.io
VITE_ASSET_HUB_ENDPOINT: wss://asset-hub-rpc.pezkuwichain.io
VITE_PEOPLE_CHAIN_ENDPOINT: wss://people-rpc.pezkuwichain.io
VITE_WALLETCONNECT_PROJECT_ID: 8292a793b7640e8364c378e331e76d04
- name: Upload build artifact
uses: actions/upload-artifact@v4
with:
name: web-dist
path: web/dist/
# ========================================
# BACKEND - INDEXER SERVICE (lint/smoke/audit)
# No build step exists (plain ESM node service); we lockfile-enforce install,
# syntax-check the entry point, and audit. The audit is currently non-blocking
# because the express@5 + sqlite3(node-gyp) trees carry transitive advisories
# that need a broader dependency upgrade; flip continue-on-error off once the
# tree is clean.
# ========================================
backend:
name: Backend Indexer
runs-on: pwap-runner
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Cache npm dependencies
uses: actions/cache@v4
with:
path: backend/node_modules
key: ${{ runner.os }}-backend-${{ hashFiles('backend/package-lock.json') }}
restore-keys: |
${{ runner.os }}-backend-
- name: Install dependencies
working-directory: ./backend
run: npm ci
- name: Syntax check (entry point)
working-directory: ./backend
run: node --check src/index.js
# Real, offline test suite (node:test). Uses in-memory sqlite and a
# mocked chain/Supabase — NO live node, NO network. The integration-tests/
# *.live.test.js suites are intentionally EXCLUDED (they require a live
# chain + Supabase and are not run in CI); the `test` script scopes to
# test/*.test.js only.
- name: Run tests (offline)
working-directory: ./backend
run: npm test
- name: npm audit (high + critical, production deps)
working-directory: ./backend
continue-on-error: true
run: npm audit --audit-level=high --omit=dev
# ========================================
# BUILD & PUSH DOCKER IMAGE TO GHCR
# Immutable artifact for audit + rollback (vs ephemeral GHA artifact).
# Tagged with git SHA so any commit can be redeployed by SHA.
# ========================================
build-image:
name: Build & Push Image
runs-on: pwap-runner
needs: [web, notify-deploy-pending]
if: |
github.ref == 'refs/heads/main' &&
(github.event_name == 'push' ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.rollback_to == ''))
permissions:
contents: read
packages: write
id-token: write # cosign keyless signing via Sigstore OIDC
outputs:
image_sha: ${{ steps.meta.outputs.image_sha }}
image_digest: ${{ steps.build.outputs.digest }}
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Install cosign
uses: sigstore/cosign-installer@v3
with:
cosign-release: 'v2.4.1'
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract image metadata
id: meta
run: |
SHORT_SHA="${GITHUB_SHA:0:7}"
echo "short_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
echo "image_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
echo "image=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}" >> $GITHUB_OUTPUT
- name: Build and push
id: build
uses: docker/build-push-action@v6
with:
context: ./
file: ./web/Dockerfile
push: true
tags: |
${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.short_sha }}
${{ steps.meta.outputs.image }}:latest
build-args: |
VITE_NETWORK=MAINNET
VITE_WS_ENDPOINT=wss://rpc.pezkuwichain.io
VITE_WS_ENDPOINT_FALLBACK_1=wss://mainnet.pezkuwichain.io
VITE_ASSET_HUB_ENDPOINT=wss://asset-hub-rpc.pezkuwichain.io
VITE_PEOPLE_CHAIN_ENDPOINT=wss://people-rpc.pezkuwichain.io
VITE_WALLETCONNECT_PROJECT_ID=8292a793b7640e8364c378e331e76d04
VITE_SUPABASE_URL=${{ secrets.VITE_SUPABASE_URL }}
VITE_SUPABASE_ANON_KEY=${{ secrets.VITE_SUPABASE_ANON_KEY }}
cache-from: type=registry,ref=${{ steps.meta.outputs.image }}:cache
cache-to: type=registry,ref=${{ steps.meta.outputs.image }}:cache,mode=max
provenance: false
- name: Sign image with cosign (keyless, Sigstore Fulcio)
env:
COSIGN_EXPERIMENTAL: '1'
run: |
IMAGE_DIGEST="${{ steps.meta.outputs.image }}@${{ steps.build.outputs.digest }}"
# cosign needs its own registry auth — docker/login-action only writes
# ~/.docker/config.json which cosign on self-hosted runner can't read
echo "${{ secrets.GITHUB_TOKEN }}" | cosign login ghcr.io -u "${{ github.actor }}" --password-stdin
echo "Signing $IMAGE_DIGEST"
cosign sign --yes "$IMAGE_DIGEST"
echo "✅ Image signed (transparency log: rekor.sigstore.dev)"
# ========================================
# BUILD & PUSH BACKEND (INDEXER) IMAGE TO GHCR
# Runnable Node service image (vs the web static-dist image). SHA-tagged +
# cosign-signed for the same audit/rollback discipline as the web image.
# Gated to main/tags, never fork PRs (needs packages:write + the telegram gate).
# ========================================
build-image-backend:
name: Build & Push Backend Image
runs-on: pwap-runner
needs: [backend, notify-deploy-pending]
if: |
(github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')) &&
(github.event_name == 'push' ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.rollback_to == ''))
permissions:
contents: read
packages: write
id-token: write # cosign keyless signing via Sigstore OIDC
outputs:
image_sha: ${{ steps.meta.outputs.image_sha }}
image_digest: ${{ steps.build.outputs.digest }}
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- name: Install cosign
uses: sigstore/cosign-installer@v3
with:
cosign-release: 'v2.4.1'
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract image metadata
id: meta
run: |
SHORT_SHA="${GITHUB_SHA:0:7}"
echo "short_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
echo "image_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
echo "image=${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}" >> $GITHUB_OUTPUT
- name: Build and push
id: build
uses: docker/build-push-action@v6
with:
context: ./backend
file: ./backend/Dockerfile
push: true
tags: |
${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.short_sha }}
${{ steps.meta.outputs.image }}:latest
cache-from: type=registry,ref=${{ steps.meta.outputs.image }}:cache
cache-to: type=registry,ref=${{ steps.meta.outputs.image }}:cache,mode=max
provenance: false
- name: Sign image with cosign (keyless, Sigstore Fulcio)
env:
COSIGN_EXPERIMENTAL: '1'
run: |
IMAGE_DIGEST="${{ steps.meta.outputs.image }}@${{ steps.build.outputs.digest }}"
echo "${{ secrets.GITHUB_TOKEN }}" | cosign login ghcr.io -u "${{ github.actor }}" --password-stdin
echo "Signing $IMAGE_DIGEST"
cosign sign --yes "$IMAGE_DIGEST"
echo "✅ Backend image signed (transparency log: rekor.sigstore.dev)"
# ========================================
# DEPLOY APPROVAL
# The approval is a GitHub Environment protection rule on `production`, not
# something implemented here. No runner is held while it waits, the window is
# 30 days instead of minutes, and who approved which SHA is recorded in the
# deployment history.
#
# This job only notifies; it sends one message and exits in seconds and cannot
# block anything. It replaces a loop that polled /tmp/pexsec-gates for 30
# minutes while occupying a pwap-runner slot — on 2026-07-30 that window
# expired unseen and cancelled a deploy with nothing shipped.
# ========================================
notify-deploy-pending:
name: Notify approver
runs-on: pwap-runner
needs: [web, security-audit]
if: github.ref == 'refs/heads/main' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
timeout-minutes: 5
steps:
- name: Send Telegram notification
env:
BOT_TOKEN: ${{ secrets.PEXSEC_BOT_TOKEN }}
CEO_CHAT_ID: ${{ secrets.TELEGRAM_CEO_CHAT_ID }}
SHA: ${{ github.sha }}
ACTOR: ${{ github.actor }}
MESSAGE: ${{ github.event.head_commit.message }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
SHORT="${SHA:0:7}"
# Strip Markdown special chars to prevent Telegram parse errors
SAFE_MSG=$(echo "${MESSAGE}" | head -1 | tr -d '_*`[]()#|{}!' | cut -c1-120)
curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
-H "Content-Type: application/json" \
-d "{
\"chat_id\": \"${CEO_CHAT_ID}\",
\"parse_mode\": \"Markdown\",
\"text\": \"🚀 *pwap/web Deploy Approval*\\n\\n\`${SHORT}\` — ${ACTOR}\\n\\n_${SAFE_MSG}_\\n\\nTargets: app.pezkuwichain.io + pex.mom\\n\\nApprove in GitHub — the deploy waits until you do.\",
\"reply_markup\": {
\"inline_keyboard\": [[
{\"text\": \"🔎 Review \& Approve\", \"url\": \"${RUN_URL}\"}
]]
}
}" > /dev/null
echo "Approver notified; deployment waits on the 'production' environment."
# ========================================
# VERSION BUMP (RUNS BEFORE BOTH DEPLOYS)
# ========================================
bump-version:
name: Bump Version
runs-on: pwap-runner
needs: [web, security-audit, notify-deploy-pending, build-image]
# Skip on rollback (workflow_dispatch with rollback_to set)
if: |
github.ref == 'refs/heads/main' &&
(github.event_name == 'push' ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.rollback_to == ''))
outputs:
new_version: ${{ steps.bump.outputs.version }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Bump version
id: bump
working-directory: ./web
run: |
npm version patch --no-git-tag-version
VERSION=$(node -p "require('./package.json').version")
echo "version=$VERSION" >> $GITHUB_OUTPUT
cd ..
git add web/package.json
git commit -m "chore(web): bump version to $VERSION [skip ci]" || echo "No version change"
git push || echo "Nothing to push"
# ========================================
# DEPLOY TO app.pezkuwichain.io (DEV VPS)
# Pulls SHA-tagged image from GHCR, extracts /dist, scp to VPS.
# Health check + auto-rollback to .deploy-tag-prev on failure.
# ========================================
deploy-app:
name: Deploy app.pezkuwichain.io
runs-on: pwap-runner
environment: production
needs: [notify-deploy-pending, bump-version, build-image]
if: |
always() &&
needs.notify-deploy-pending.result == 'success' &&
((github.event_name == 'push' && needs.build-image.result == 'success' && needs.bump-version.result == 'success') ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.rollback_to != ''))
permissions:
contents: read
packages: read
env:
DOMAIN: app.pezkuwichain.io
TARGET_PATH: /var/www/subdomains/app
steps:
- name: Determine image SHA
id: sha
run: |
if [ -n "${{ github.event.inputs.rollback_to }}" ]; then
echo "sha=${{ github.event.inputs.rollback_to }}" >> $GITHUB_OUTPUT
echo "Rolling back to: ${{ github.event.inputs.rollback_to }}"
else
echo "sha=${{ needs.build-image.outputs.image_sha }}" >> $GITHUB_OUTPUT
fi
- name: Capture currently-live SHA (for auto-rollback)
id: prev
run: |
# /.deploy-sha is written into every deploy; read what's live now
PREV=$(curl -sf --max-time 5 "https://${{ env.DOMAIN }}/.deploy-sha" | head -c 40 | tr -dc 'a-f0-9' || echo "")
echo "Previous live SHA: ${PREV:-unknown}"
echo "prev=$PREV" >> $GITHUB_OUTPUT
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Install cosign (for verify)
uses: sigstore/cosign-installer@v3
with:
cosign-release: 'v2.4.1'
- name: Verify image signature (cosign keyless)
env:
COSIGN_EXPERIMENTAL: '1'
run: |
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.sha.outputs.sha }}"
echo "${{ secrets.GITHUB_TOKEN }}" | cosign login ghcr.io -u "${{ github.actor }}" --password-stdin
echo "Verifying signature for $IMAGE"
cosign verify "$IMAGE" \
--certificate-identity-regexp "^https://github.com/pezkuwichain/pwap/.github/workflows/quality-gate.yml@" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
> /dev/null
echo "✅ Signature valid — image was built by trusted CI"
- name: Extract /dist from image
run: |
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.sha.outputs.sha }}"
docker pull "$IMAGE"
CID=$(docker create "$IMAGE")
mkdir -p dist
docker cp "$CID:/dist/." dist/
docker rm "$CID" >/dev/null
# Stamp this build's SHA into dist so future deploys can read PREV
echo "${{ steps.sha.outputs.sha }}" > dist/.deploy-sha
ls -la dist/ | head -10
- name: Deploy to DEV VPS
uses: appleboy/scp-action@v1.0.0
with:
host: ${{ secrets.VPS_HOST }}
username: ${{ secrets.VPS_USER }}
key: ${{ secrets.VPS_SSH_KEY }}
port: ${{ secrets.VPS_SSH_PORT || 2222 }}
source: 'dist/*'
target: '/var/www/subdomains/app'
strip_components: 1
- name: Health check (60s window)
id: healthcheck
run: |
for i in 1 2 3 4 5 6; do
if curl -sf --max-time 10 "https://${{ env.DOMAIN }}/" >/dev/null; then
echo "✅ ${{ env.DOMAIN }} healthy"
exit 0
fi
echo "Attempt $i/6 failed, retrying in 10s..."
sleep 10
done
echo "❌ Health check failed for ${{ env.DOMAIN }}"
exit 1
# ── Automatic rollback: pull PREV SHA image, redeploy, recheck ──
- name: Auto-rollback to previous SHA
id: rollback
if: failure() && steps.healthcheck.conclusion == 'failure' && steps.prev.outputs.prev != ''
run: |
PREV="${{ steps.prev.outputs.prev }}"
echo "🔄 Rolling back ${{ env.DOMAIN }} to $PREV"
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:$PREV"
docker pull "$IMAGE"
CID=$(docker create "$IMAGE")
rm -rf dist && mkdir dist
docker cp "$CID:/dist/." dist/
docker rm "$CID" >/dev/null
echo "$PREV" > dist/.deploy-sha
echo "rollback_sha=$PREV" >> $GITHUB_OUTPUT
- name: SCP rollback artifact
if: steps.rollback.outcome == 'success'
uses: appleboy/scp-action@v1.0.0
with:
host: ${{ secrets.VPS_HOST }}
username: ${{ secrets.VPS_USER }}
key: ${{ secrets.VPS_SSH_KEY }}
port: ${{ secrets.VPS_SSH_PORT || 2222 }}
source: 'dist/*'
target: '/var/www/subdomains/app'
strip_components: 1
- name: Re-health-check after rollback
if: steps.rollback.outcome == 'success'
id: healthcheck_rb
run: |
for i in 1 2 3 4 5 6; do
if curl -sf --max-time 10 "https://${{ env.DOMAIN }}/" >/dev/null; then
echo "✅ Rolled back successfully — ${{ env.DOMAIN }} healthy on ${{ steps.rollback.outputs.rollback_sha }}"
exit 0
fi
sleep 10
done
echo "❌ Rollback also failed!"
exit 1
- name: Post-deploy notification
if: success()
run: |
echo "✅ Deployed image ${{ steps.sha.outputs.sha }} to ${{ env.DOMAIN }}"
- name: Notify failure (Telegram)
if: failure()
env:
BOT_TOKEN: ${{ secrets.PEXSEC_BOT_TOKEN }}
CEO_CHAT_ID: ${{ secrets.TELEGRAM_CEO_CHAT_ID }}
NEW_SHA: ${{ steps.sha.outputs.sha }}
PREV_SHA: ${{ steps.prev.outputs.prev }}
ROLLBACK_OUTCOME: ${{ steps.rollback.outcome }}
RECHECK_OUTCOME: ${{ steps.healthcheck_rb.outcome }}
run: |
if [ "$RECHECK_OUTCOME" = "success" ]; then
MSG="⚠️ pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed health check, AUTO-ROLLED-BACK to $PREV_SHA. Site healthy."
elif [ "$ROLLBACK_OUTCOME" = "success" ]; then
MSG="🚨 pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed AND rollback to $PREV_SHA also failed. Manual intervention needed."
elif [ -z "$PREV_SHA" ]; then
MSG="❌ pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed. No previous SHA available (first deploy?). Manual rollback: gh workflow run quality-gate.yml -f rollback_to=<sha>"
else
MSG="❌ pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed. Auto-rollback was not attempted. Manual: gh workflow run quality-gate.yml -f rollback_to=$PREV_SHA"
fi
curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
-d "chat_id=${CEO_CHAT_ID}" --data-urlencode "text=$MSG"
# ========================================
# DEPLOY TO pex.mom (VPS3 — geo-redundant mirror)
# ========================================
deploy-pex:
name: Deploy pex.mom
runs-on: pwap-runner
environment: production
needs: [notify-deploy-pending, bump-version, build-image]
if: |
always() &&
needs.notify-deploy-pending.result == 'success' &&
((github.event_name == 'push' && needs.build-image.result == 'success' && needs.bump-version.result == 'success') ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.rollback_to != ''))
permissions:
contents: read
packages: read
env:
DOMAIN: pex.mom
TARGET_PATH: /var/www/pex.mom
steps:
- name: Determine image SHA
id: sha
run: |
if [ -n "${{ github.event.inputs.rollback_to }}" ]; then
echo "sha=${{ github.event.inputs.rollback_to }}" >> $GITHUB_OUTPUT
else
echo "sha=${{ needs.build-image.outputs.image_sha }}" >> $GITHUB_OUTPUT
fi
- name: Capture currently-live SHA (for auto-rollback)
id: prev
run: |
PREV=$(curl -sf --max-time 5 "https://${{ env.DOMAIN }}/.deploy-sha" | head -c 40 | tr -dc 'a-f0-9' || echo "")
echo "Previous live SHA: ${PREV:-unknown}"
echo "prev=$PREV" >> $GITHUB_OUTPUT
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Install cosign (for verify)
uses: sigstore/cosign-installer@v3
with:
cosign-release: 'v2.4.1'
- name: Verify image signature (cosign keyless)
env:
COSIGN_EXPERIMENTAL: '1'
run: |
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.sha.outputs.sha }}"
echo "Verifying signature for $IMAGE"
cosign verify "$IMAGE" \
--certificate-identity-regexp "^https://github.com/pezkuwichain/pwap/.github/workflows/quality-gate.yml@" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
> /dev/null
echo "✅ Signature valid — image was built by trusted CI"
- name: Extract /dist from image
run: |
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.sha.outputs.sha }}"
docker pull "$IMAGE"
CID=$(docker create "$IMAGE")
mkdir -p dist
docker cp "$CID:/dist/." dist/
docker rm "$CID" >/dev/null
echo "${{ steps.sha.outputs.sha }}" > dist/.deploy-sha
- name: Deploy to VPS3
uses: appleboy/scp-action@v1.0.0
with:
host: ${{ secrets.VPS_PEX_HOST }}
username: ${{ secrets.VPS_PEX_USER }}
key: ${{ secrets.VPS_PEX_SSH_KEY }}
port: ${{ secrets.VPS_PEX_SSH_PORT || 22 }}
source: 'dist/*'
target: '/var/www/pex.mom'
strip_components: 1
- name: Health check (60s window)
id: healthcheck
run: |
for i in 1 2 3 4 5 6; do
if curl -sf --max-time 10 "https://${{ env.DOMAIN }}/" >/dev/null; then
echo "✅ ${{ env.DOMAIN }} healthy"
exit 0
fi
echo "Attempt $i/6 failed, retrying in 10s..."
sleep 10
done
echo "❌ Health check failed for ${{ env.DOMAIN }}"
exit 1
- name: Auto-rollback to previous SHA
id: rollback
if: failure() && steps.healthcheck.conclusion == 'failure' && steps.prev.outputs.prev != ''
run: |
PREV="${{ steps.prev.outputs.prev }}"
echo "🔄 Rolling back ${{ env.DOMAIN }} to $PREV"
IMAGE="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:$PREV"
docker pull "$IMAGE"
CID=$(docker create "$IMAGE")
rm -rf dist && mkdir dist
docker cp "$CID:/dist/." dist/
docker rm "$CID" >/dev/null
echo "$PREV" > dist/.deploy-sha
echo "rollback_sha=$PREV" >> $GITHUB_OUTPUT
- name: SCP rollback artifact
if: steps.rollback.outcome == 'success'
uses: appleboy/scp-action@v1.0.0
with:
host: ${{ secrets.VPS_PEX_HOST }}
username: ${{ secrets.VPS_PEX_USER }}
key: ${{ secrets.VPS_PEX_SSH_KEY }}
port: ${{ secrets.VPS_PEX_SSH_PORT || 22 }}
source: 'dist/*'
target: '/var/www/pex.mom'
strip_components: 1
- name: Re-health-check after rollback
if: steps.rollback.outcome == 'success'
id: healthcheck_rb
run: |
for i in 1 2 3 4 5 6; do
if curl -sf --max-time 10 "https://${{ env.DOMAIN }}/" >/dev/null; then
echo "✅ Rolled back successfully — ${{ env.DOMAIN }} healthy on ${{ steps.rollback.outputs.rollback_sha }}"
exit 0
fi
sleep 10
done
echo "❌ Rollback also failed!"
exit 1
- name: Post-deploy notification
if: success()
run: |
echo "✅ Deployed image ${{ steps.sha.outputs.sha }} to ${{ env.DOMAIN }}"
- name: Notify failure (Telegram)
if: failure()
env:
BOT_TOKEN: ${{ secrets.PEXSEC_BOT_TOKEN }}
CEO_CHAT_ID: ${{ secrets.TELEGRAM_CEO_CHAT_ID }}
NEW_SHA: ${{ steps.sha.outputs.sha }}
PREV_SHA: ${{ steps.prev.outputs.prev }}
ROLLBACK_OUTCOME: ${{ steps.rollback.outcome }}
RECHECK_OUTCOME: ${{ steps.healthcheck_rb.outcome }}
run: |
if [ "$RECHECK_OUTCOME" = "success" ]; then
MSG="⚠️ pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed health check, AUTO-ROLLED-BACK to $PREV_SHA. Site healthy."
elif [ "$ROLLBACK_OUTCOME" = "success" ]; then
MSG="🚨 pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed AND rollback to $PREV_SHA also failed. Manual intervention needed."
elif [ -z "$PREV_SHA" ]; then
MSG="❌ pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed. No previous SHA available (first deploy?). Manual rollback: gh workflow run quality-gate.yml -f rollback_to=<sha>"
else
MSG="❌ pwap/web ${{ env.DOMAIN }}: deploy ($NEW_SHA) failed. Auto-rollback was not attempted. Manual: gh workflow run quality-gate.yml -f rollback_to=$PREV_SHA"
fi
curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
-d "chat_id=${CEO_CHAT_ID}" --data-urlencode "text=$MSG"
# ========================================
# DEPLOY BACKEND (INDEXER) TO ITS HOST
# Runs a container from the SHA-tagged GHCR image on the backend VPS.
# Same discipline as the web deploy: cosign verify → deploy → health check →
# auto-rollback to previous SHA → Telegram notify. Gated behind the SAME
# same environment approval as the web deploy. Only main/tags, never fork PRs.
#
# STATEFUL DB: the sqlite file lives in the named docker volume
# `pwap-indexer-db` (mounted at /data). Deploy replaces the container/image
# but NEVER the volume — indexer state survives every deploy and rollback.
#
# Host param via secret BACKEND_VPS_HOST (no hardcoded IP). If the backend
# shares the web DEV VPS, set BACKEND_VPS_HOST = VPS_HOST (see backend/DEPLOY.md).
# ========================================
# Config gate: the backend indexer only deploys once its host is provisioned.
# Until BACKEND_VPS_HOST is set, deploy-backend skips cleanly (neutral, no false
# failure/alert) instead of erroring on "missing server host" every push.
backend-cfg:
name: Backend deploy config check
runs-on: pwap-runner
if: (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')) && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
outputs:
configured: ${{ steps.c.outputs.configured }}
steps:
- id: c
env:
BH: ${{ secrets.BACKEND_VPS_HOST }}
run: |
if [ -n "$BH" ]; then
echo "configured=true" >> "$GITHUB_OUTPUT"
else
echo "::notice::BACKEND_VPS_HOST unset — backend indexer deploy skipped (provision the host to enable)"
echo "configured=false" >> "$GITHUB_OUTPUT"
fi
deploy-backend:
name: Deploy Backend (indexer)
runs-on: pwap-runner
environment: production
needs: [notify-deploy-pending, build-image-backend, backend-cfg]
if: |
always() &&
needs.backend-cfg.outputs.configured == 'true' &&
(github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')) &&
needs.notify-deploy-pending.result == 'success' &&
((github.event_name == 'push' && needs.build-image-backend.result == 'success') ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.rollback_to != ''))
permissions:
contents: read
packages: read
steps:
- name: Determine image SHA
id: sha
run: |
if [ -n "${{ github.event.inputs.rollback_to }}" ]; then
echo "sha=${{ github.event.inputs.rollback_to }}" >> $GITHUB_OUTPUT
echo "Rolling back backend to: ${{ github.event.inputs.rollback_to }}"
else
echo "sha=${{ needs.build-image-backend.outputs.image_sha }}" >> $GITHUB_OUTPUT
fi
- name: Install cosign (for verify)
uses: sigstore/cosign-installer@v3
with:
cosign-release: 'v2.4.1'
- name: Log in to GHCR (runner-side, for verify)
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Verify image signature (cosign keyless)
env:
COSIGN_EXPERIMENTAL: '1'
run: |
IMAGE="${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}:${{ steps.sha.outputs.sha }}"
echo "${{ secrets.GITHUB_TOKEN }}" | cosign login ghcr.io -u "${{ github.actor }}" --password-stdin
echo "Verifying signature for $IMAGE"
cosign verify "$IMAGE" \
--certificate-identity-regexp "^https://github.com/pezkuwichain/pwap/.github/workflows/quality-gate.yml@" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
> /dev/null
echo "✅ Signature valid — backend image was built by trusted CI"
- name: Deploy container to backend host (health-checked, auto-rollback)
id: deploy
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.BACKEND_VPS_HOST }}
username: ${{ secrets.BACKEND_VPS_USER }}
key: ${{ secrets.BACKEND_VPS_SSH_KEY }}
port: ${{ secrets.BACKEND_VPS_SSH_PORT || 22 }}
envs: IMAGE_REF,NEW_SHA,GHCR_USER,GHCR_TOKEN,WS_ENDPOINT,HOST_PORT
command_timeout: 10m
script_stop: true
script: |
set -e
REGISTRY_IMAGE="ghcr.io/pezkuwichain/pwap-indexer"
IMAGE="$REGISTRY_IMAGE:$NEW_SHA"
STATE_DIR="/opt/pwap-indexer"
NAME="pwap-indexer"
HOST_PORT="${HOST_PORT:-3001}"
mkdir -p "$STATE_DIR"
PREV="$(cat "$STATE_DIR/.deploy-sha" 2>/dev/null || echo "")"
echo "Deploying $IMAGE (previous: ${PREV:-none})"
echo "$GHCR_TOKEN" | docker login ghcr.io -u "$GHCR_USER" --password-stdin
docker pull "$IMAGE"
run_container () {
local img="$1"
docker rm -f "$NAME" >/dev/null 2>&1 || true
# Named volume pwap-indexer-db = stateful sqlite DB. NEVER removed
# by this script — it is created on first run and reused forever.
# Config comes from the workflow (GitHub secrets/vars), NOT a
# hand-placed file on the host. The indexer (src/index.js) only
# needs WS_ENDPOINT beyond the fixed PORT/DB_PATH; it runs no
# KYC/council service, so no seed/Supabase secret is required.
# Host port is configurable (HOST_PORT, default 3001) so the indexer
# can co-exist with whatever else runs on the box; the container
# always listens on 3001 internally.
docker run -d --name "$NAME" --restart unless-stopped \
-p "${HOST_PORT}:3001" \
-v pwap-indexer-db:/data \
-e DB_PATH=/data/transactions.db \
-e PORT=3001 \
-e WS_ENDPOINT="$WS_ENDPOINT" \
"$img"
}
healthy () {
for i in $(seq 1 12); do
if curl -fsS --max-time 5 "http://localhost:${HOST_PORT}/health" >/dev/null 2>&1; then
return 0
fi
echo "health attempt $i/12 failed, retry in 5s..."
sleep 5
done
return 1
}
run_container "$IMAGE"
if healthy; then
echo "$NEW_SHA" > "$STATE_DIR/.deploy-sha"
echo "✅ Backend healthy on $NEW_SHA"
docker image prune -f >/dev/null 2>&1 || true
exit 0
fi
echo "❌ Health check failed for $NEW_SHA"
if [ -n "$PREV" ]; then
echo "🔄 Rolling back to $PREV"
docker pull "$REGISTRY_IMAGE:$PREV"
run_container "$REGISTRY_IMAGE:$PREV"
if healthy; then
echo "$PREV" > "$STATE_DIR/.deploy-sha"
echo "✅ Rolled back to $PREV — backend healthy"
exit 1 # still fail the job: the new SHA did not deploy
fi
echo "🚨 Rollback to $PREV ALSO failed"
exit 1
fi
echo "No previous SHA to roll back to."
exit 1
env:
IMAGE_REF: ${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}:${{ steps.sha.outputs.sha }}
NEW_SHA: ${{ steps.sha.outputs.sha }}
GHCR_USER: ${{ github.actor }}
GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Non-secret chain endpoint; override via the INDEXER_WS_ENDPOINT secret if needed.
WS_ENDPOINT: ${{ secrets.INDEXER_WS_ENDPOINT || 'wss://rpc.pezkuwichain.io' }}
# Host port for the indexer (container is always 3001 internally).
HOST_PORT: ${{ secrets.INDEXER_HOST_PORT || '3001' }}
- name: Post-deploy notification
if: success()
run: echo "✅ Deployed backend image ${{ steps.sha.outputs.sha }} to backend host"
- name: Notify failure (Telegram)
if: failure()
env:
BOT_TOKEN: ${{ secrets.PEXSEC_BOT_TOKEN }}
CEO_CHAT_ID: ${{ secrets.TELEGRAM_CEO_CHAT_ID }}
NEW_SHA: ${{ steps.sha.outputs.sha }}
run: |
MSG="❌ pwap/backend indexer: deploy ($NEW_SHA) failed. If a previous SHA existed the host auto-rolled-back to it (DB volume untouched). Manual: gh workflow run quality-gate.yml -f rollback_to=<sha>"
curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
-d "chat_id=${CEO_CHAT_ID}" --data-urlencode "text=$MSG"
# ========================================
# DEPLOY - SUPABASE (edge functions + migrations)
# Same Telegram-gated cutover as the frontend/backend: syncs the edge functions
# into the self-hosted edge-runtime volume (hot-reloaded, no restart) and applies
# any un-recorded migrations transactionally against the fund-custody Postgres.
# Runs in parallel with deploy-app so functions, migrations and the new frontend
# go live in the same post-approval window (minimal fail-closed gap).
# ========================================
deploy-supabase:
name: Deploy Supabase (functions + migrations)
runs-on: pwap-runner
environment: production
needs: [notify-deploy-pending]
if: |
always() &&
needs.notify-deploy-pending.result == 'success' &&
github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Pack supabase tree
run: |
# __tests__ are Deno tests — kept out of the runtime bundle entirely.
tar czf supabase-deploy.tgz -C web/supabase \
--exclude='__tests__' functions migrations deploy
- name: Upload bundle to staging
uses: appleboy/scp-action@v1.0.0
with:
host: ${{ secrets.SUPABASE_VPS_HOST }}
username: ${{ secrets.SUPABASE_VPS_USER }}
key: ${{ secrets.SUPABASE_VPS_SSH_KEY }}
port: ${{ secrets.SUPABASE_VPS_SSH_PORT || 22 }}
source: "supabase-deploy.tgz"
target: /opt/supabase-self-hosted/deploy-staging
- name: Sync edge functions + apply migrations (health-checked)
id: apply
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.SUPABASE_VPS_HOST }}
username: ${{ secrets.SUPABASE_VPS_USER }}
key: ${{ secrets.SUPABASE_VPS_SSH_KEY }}
port: ${{ secrets.SUPABASE_VPS_SSH_PORT || 22 }}
command_timeout: 15m
script: |
set -euo pipefail
BASE=/opt/supabase-self-hosted/deploy-staging
STAGING="$BASE/tree"
DB=supabase-db
echo "── 0. Unpack bundle ──"
rm -rf "$STAGING" && mkdir -p "$STAGING"
tar xzf "$BASE/supabase-deploy.tgz" -C "$STAGING"
echo "── 1. Sync edge functions (through the ownership gate) ──"
# The runtime volume is shared with pezkuwi-telegram-miniapp. A plain
# rsync here silently replaced that project's telegram-auth on
# 2026-06-28 and broke its sign-in for a month, so writes now go
# through a gate that refuses any name this project does not own.
# Ownership: /opt/supabase-self-hosted/functions-registry.json
# __tests__ are Deno test dirs — the gate skips __-prefixed dirs.
supabase-deploy-functions --project pwap-web --src "$STAGING/functions"
echo "── 2. Pre-flight: required edge-runtime secrets ──"
EF_ENV="$(docker exec supabase-edge-functions env 2>/dev/null || true)"
miss=""
for k in SUPABASE_URL SUPABASE_SERVICE_ROLE_KEY PLATFORM_PRIVATE_KEY; do
grep -q "^$k=" <<<"$EF_ENV" || miss="$miss $k"
done
if [ -n "$miss" ]; then
echo "::warning::edge-runtime missing secrets:$miss (fund functions may fail until set)"
fi
echo "── 3. Apply pending migrations (transactional, tracked) ──"
bash "$STAGING/deploy/apply-migrations.sh" "$STAGING/migrations" "$DB"
echo "── 4. Verify the fund-custody guards actually took effect ──"
q() { docker exec -i "$DB" psql -U postgres -tAq -c "$1"; }
fail=0
for fn in "release_escrow_internal" "lock_escrow_internal" "refund_escrow_internal" "request_withdraw"; do
# any signature match; expect NO anon EXECUTE
oid="$(q "SELECT oid FROM pg_proc WHERE proname='$fn' LIMIT 1;")"
if [ -n "$oid" ]; then
priv="$(q "SELECT has_function_privilege('anon', $oid, 'EXECUTE');")"
echo " anon EXECUTE $fn = $priv"
[ "$priv" = "f" ] || { echo "::error::anon still has EXECUTE on $fn"; fail=1; }
fi
done
trg="$(q "SELECT tgname FROM pg_trigger WHERE tgname IN ('trg_freeze_trade_financials','trg_freeze_offer_financials');")"
n_trg="$(printf '%s' "$trg" | grep -c . || true)"
echo " freeze triggers present: ${trg:-<none>} (count=$n_trg)"
[ "$n_trg" = "2" ] || { echo "::error::trade/offer freeze triggers missing (expected 2, got $n_trg)"; fail=1; }
rm -rf "$BASE"
[ "$fail" = "0" ] && echo "✅ Supabase deploy verified" || { echo "❌ post-deploy verification failed"; exit 1; }
- name: Telegram notify
if: always()
env:
BOT_TOKEN: ${{ secrets.PEXSEC_BOT_TOKEN }}
CEO_CHAT_ID: ${{ secrets.TELEGRAM_CEO_CHAT_ID }}
OUTCOME: ${{ steps.apply.outcome }}
run: |
if [ "$OUTCOME" = "success" ]; then
MSG="✅ pwap Supabase: edge functions synced + migrations applied & verified (${{ github.sha }})."
else
MSG="🚨 pwap Supabase deploy FAILED (${{ github.sha }}) — functions/migrations may be partially applied. Check the run."
fi
curl -s -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
-d chat_id="${CEO_CHAT_ID}" -d text="$MSG" >/dev/null || true
# ========================================
# SECURITY CHECKS (BLOCKING)
# npm audit (high + critical) + TruffleHog secret scan
# ========================================
security-audit:
name: Security Audit
runs-on: pwap-runner
needs: [web]
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: ${{ github.event_name == 'pull_request' && 0 || 1 }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Web — npm audit (high + critical, production deps only)
working-directory: ./web
run: |
npm ci
# Audit only production dependencies. Build tooling (vite, esbuild,
# vite-plugin-node-polyfills → elliptic, etc.) ships to no user, and
# advisories on those dev deps kept blocking production deploys.
npm audit --audit-level=high --omit=dev
- name: TruffleHog — PR diff (verified secrets only)
if: github.event_name == 'pull_request'
uses: trufflesecurity/trufflehog@main
with:
base: ${{ github.event.pull_request.base.sha }}
head: ${{ github.event.pull_request.head.sha }}
extra_args: --only-verified
- name: TruffleHog — full repo scan (verified secrets only)
if: github.event_name != 'pull_request'
uses: trufflesecurity/trufflehog@main
with:
path: ./
extra_args: --only-verified
# ========================================
# CI GATE — explicit merge-block
# All required checks must succeed (or be skipped, e.g. for rollback path).
# Branch protection on main should require this job's success.
# ========================================
ci-gate:
name: CI Gate ✅
runs-on: pwap-runner
needs: [web, backend, security-audit]
if: always()
steps:
- name: Verify all required jobs succeeded or were intentionally skipped
run: |
results='${{ toJSON(needs) }}'
echo "$results" | python3 -c "
import json, sys
needs = json.load(sys.stdin)
failed = [name for name, job in needs.items() if job['result'] not in ('success', 'skipped')]
if failed:
print('❌ Required jobs failed: ' + ', '.join(failed))
sys.exit(1)
print('✅ All required CI jobs passed or skipped')
"