mirror of
https://github.com/pezkuwichain/pwap.git
synced 2026-08-12 16:11:35 +00:00
db: replace the migration history with a baseline of the live schema
The migrations did not describe this database, and could not be made to.
Three findings, in the order they surfaced:
* 25 functions declared across six migrations — all recorded as applied — did
not exist. Their tables did. Two were reached by the app, so merchant tier
upgrades and post-trade reputation updates had been quietly dead. This only
came to light because a user hit "Could not find the function
public.upsert_user_profile(...)" while toggling a notification setting.
* admin_roles has three conflicting definitions across the set and production
matches none of them. 001 says (id, user_id, role, granted_by, granted_at),
COMBINED says (user_id, role, created_at), production has
(id, user_id, role, permissions, created_at, updated_at).
* Applied to an empty database, five migrations fail. The legacy 0NN filenames
sort before the 14-digit timestamps they depend on — "013" < "20241117054600"
— so 013 runs before the migration creating the table it alters. The set
could never have been replayed from scratch.
So it could not be tested, could not rebuild the database, and did not match what
was running. Widening or patching it would have been dressing up a history that
was already fiction.
The baseline is a pg_dump of the live public schema, which matches production by
construction. Privileges are included deliberately: the REVOKEs on
lock_escrow_internal, release_escrow_internal, refund_escrow_internal and
request_withdraw are the 20260725030000 hardening, and dropping them would hand
fund movement back to anon.
Verified step by step against production before committing:
- applies to an empty database cleanly, after three real obstacles were fixed
(extensions live in the `extensions` schema, supabase_admin membership,
platform-level ALTER DEFAULT PRIVILEGES that cannot apply outside Supabase)
- produces 83 tables / 39 functions / 222 indexes / 360 policies — identical
counts to production
- recorded as applied in supabase_migrations without touching the 37 existing
rows, then dry-run confirmed "up to date — no pending migrations", so the
next deploy will not try to replay it over live tables
- drift check now reports "every declared function is present"; the known-gaps
list drops from 22 entries to zero
The old files move to migrations/archive/ rather than being deleted — they are
the only record of why parts of this look the way they do. Their README says
plainly not to run them, and why.
CI now applies migrations to an empty Postgres on every PR. That check was
impossible while the old set was the starting point; it is the thing that stops
this class of drift from being discovered by a user again.
This commit is contained in:
@@ -1082,10 +1082,96 @@ jobs:
|
|||||||
# All required checks must succeed (or be skipped, e.g. for rollback path).
|
# All required checks must succeed (or be skipped, e.g. for rollback path).
|
||||||
# Branch protection on main should require this job's success.
|
# Branch protection on main should require this job's success.
|
||||||
# ========================================
|
# ========================================
|
||||||
|
# Applies every migration to an empty Postgres, so a migration that cannot run
|
||||||
|
# from scratch fails here rather than on the production host.
|
||||||
|
#
|
||||||
|
# This was not possible before 2026-08-01: the old set could not be replayed at
|
||||||
|
# all. Five migrations failed on an empty database, partly because the legacy
|
||||||
|
# 0NN filenames sort before the 14-digit timestamps they depend on
|
||||||
|
# ("013" < "20241117054600"), and the schema they produced did not match
|
||||||
|
# production anyway. The baseline replaced them with a dump of the live schema,
|
||||||
|
# which is what makes this check meaningful.
|
||||||
|
migration-test:
|
||||||
|
name: Migration test
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: postgres:17
|
||||||
|
env:
|
||||||
|
POSTGRES_PASSWORD: postgres
|
||||||
|
POSTGRES_DB: migration_test
|
||||||
|
ports:
|
||||||
|
- 5432:5432
|
||||||
|
options: >-
|
||||||
|
--health-cmd pg_isready
|
||||||
|
--health-interval 10s
|
||||||
|
--health-timeout 5s
|
||||||
|
--health-retries 5
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Install psql
|
||||||
|
run: sudo apt-get update -qq && sudo apt-get install -y -qq postgresql-client
|
||||||
|
|
||||||
|
# Stand-ins for auth.uid(), auth.users, storage.objects and the Supabase
|
||||||
|
# roles. Without them the run would fail on the environment rather than on
|
||||||
|
# anything a migration got wrong.
|
||||||
|
- name: Apply Supabase stubs
|
||||||
|
env:
|
||||||
|
PGPASSWORD: postgres
|
||||||
|
run: |
|
||||||
|
psql -h localhost -U postgres -d migration_test -v ON_ERROR_STOP=1 -q \
|
||||||
|
-f web/supabase/deploy/test-bootstrap.sql
|
||||||
|
echo "stubs applied"
|
||||||
|
|
||||||
|
- name: Apply migrations in order
|
||||||
|
env:
|
||||||
|
PGPASSWORD: postgres
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
failed=0
|
||||||
|
for f in $(ls web/supabase/migrations/*.sql | sort); do
|
||||||
|
base="$(basename "$f")"
|
||||||
|
case "$base" in COMBINED*) echo " skip $base"; continue;; esac
|
||||||
|
# --single-transaction so a failure leaves nothing half-applied,
|
||||||
|
# matching how apply-migrations.sh runs them in production
|
||||||
|
if psql -h localhost -U postgres -d migration_test \
|
||||||
|
-v ON_ERROR_STOP=1 --single-transaction -q -f "$f" 2>/tmp/err.log; then
|
||||||
|
echo " ✔ $base"
|
||||||
|
else
|
||||||
|
failed=$((failed+1))
|
||||||
|
echo " ✗ $base"
|
||||||
|
grep -E "ERROR" /tmp/err.log | head -3 | sed 's/^/ /'
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ $failed -gt 0 ]; then
|
||||||
|
echo "::error::$failed migration(s) cannot be applied to an empty database"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "all migrations applied from scratch"
|
||||||
|
|
||||||
|
# A migration set that applies cleanly but produces nothing is still broken.
|
||||||
|
- name: Verify the schema was actually built
|
||||||
|
env:
|
||||||
|
PGPASSWORD: postgres
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
read -r tables functions <<<"$(psql -h localhost -U postgres -d migration_test -tAq -c \
|
||||||
|
"SELECT (SELECT count(*) FROM pg_tables WHERE schemaname='public'),
|
||||||
|
(SELECT count(*) FROM pg_proc p JOIN pg_namespace n ON n.oid=p.pronamespace
|
||||||
|
WHERE n.nspname='public')" | tr '|' ' ')"
|
||||||
|
echo "built: $tables tables, $functions functions"
|
||||||
|
if [ "$tables" -lt 50 ] || [ "$functions" -lt 20 ]; then
|
||||||
|
echo "::error::schema looks incomplete — expected the full public schema"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
ci-gate:
|
ci-gate:
|
||||||
name: CI Gate ✅
|
name: CI Gate ✅
|
||||||
runs-on: pwap-runner
|
runs-on: pwap-runner
|
||||||
needs: [web, backend, security-audit]
|
needs: [web, backend, security-audit, migration-test]
|
||||||
if: always()
|
if: always()
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
|
|||||||
@@ -1,41 +1,14 @@
|
|||||||
# Functions declared in migrations that are knowingly absent from the database.
|
# Functions declared in migrations that are knowingly absent from the database.
|
||||||
#
|
#
|
||||||
# Six migrations were recorded as applied but only ran partway: their tables
|
# Empty, and it should stay that way.
|
||||||
# exist, their function bodies never ran. Those files carry "Run this in Supabase
|
|
||||||
# SQL Editor" headers — they were pasted in by hand before apply-migrations.sh
|
|
||||||
# existed, and a run that stopped midway still got recorded.
|
|
||||||
#
|
#
|
||||||
# Nothing in the app calls any function listed here, and several are trigger
|
# This file used to list 22 functions. They came from six migrations recorded as
|
||||||
# bodies whose triggers were never created either. Creating them now would switch
|
# applied that had only run partway — their tables existed, their function bodies
|
||||||
# on behaviour that has never been live (P2P notification triggers, fraud
|
# never ran. That gap is gone: the baseline (00000000000000_baseline.sql) is a
|
||||||
# counters, stats updaters) rather than restore something that broke — a product
|
# dump of the live schema, so everything it declares exists by construction.
|
||||||
# decision, not a repair. They stay listed until someone deliberately enables them.
|
|
||||||
#
|
#
|
||||||
# The three the app does reach were repaired in 20260730150000 and are not here:
|
# The file is kept because the drift check points here when something is missing.
|
||||||
# apply_for_tier_upgrade, check_tier_eligibility, update_p2p_reputation
|
# If that happens, the first question is whether the function should exist — a
|
||||||
|
# name listed here is a decision not to have it, not a way to quiet the check.
|
||||||
#
|
#
|
||||||
# Remove a line when you actually create the function. Anything missing that is
|
# One name per line, no trailing comments.
|
||||||
# NOT listed here is new drift and the deploy will say so.
|
|
||||||
|
|
||||||
approve_tier_application
|
|
||||||
calculate_merchant_stats
|
|
||||||
calculate_user_risk_score
|
|
||||||
cancel_expired_offers
|
|
||||||
check_trade_allowed
|
|
||||||
decrement_escrow_balance
|
|
||||||
generate_referral_code
|
|
||||||
get_payment_method_details
|
|
||||||
increment_escrow_balance
|
|
||||||
log_suspicious_activity
|
|
||||||
notify_on_dispute_opened
|
|
||||||
notify_on_new_message
|
|
||||||
notify_on_new_trade
|
|
||||||
notify_on_payment_sent
|
|
||||||
notify_on_trade_completed
|
|
||||||
reset_daily_fraud_counters
|
|
||||||
reset_weekly_fraud_counters
|
|
||||||
update_discussion_activity
|
|
||||||
update_fraud_indicators_on_trade
|
|
||||||
update_merchant_stats_on_trade
|
|
||||||
update_rating_stats_trigger
|
|
||||||
update_user_rating_stats
|
|
||||||
|
|||||||
@@ -0,0 +1,88 @@
|
|||||||
|
-- Minimal stand-ins for what Supabase provides, so migrations can be applied to
|
||||||
|
-- a plain Postgres in CI.
|
||||||
|
--
|
||||||
|
-- The migrations reference auth.uid() 127 times, auth.users 95 times,
|
||||||
|
-- auth.role() 11 times and storage.objects 3 times. None of that exists in a
|
||||||
|
-- stock Postgres image, so without these stubs every run would fail on the
|
||||||
|
-- environment rather than on anything the migrations got wrong.
|
||||||
|
--
|
||||||
|
-- These are deliberately the thinnest thing that lets DDL resolve. They are not
|
||||||
|
-- a Supabase emulation and must never be applied to a real database: RLS
|
||||||
|
-- policies compiled against these stubs would behave differently, because
|
||||||
|
-- auth.uid() here returns NULL rather than the caller's id.
|
||||||
|
|
||||||
|
CREATE SCHEMA IF NOT EXISTS auth;
|
||||||
|
CREATE SCHEMA IF NOT EXISTS storage;
|
||||||
|
CREATE SCHEMA IF NOT EXISTS extensions;
|
||||||
|
|
||||||
|
-- Supabase installs these into the `extensions` schema, and the dumped schema
|
||||||
|
-- calls them fully qualified as extensions.uuid_generate_v4(). Installing them
|
||||||
|
-- into public instead makes the baseline fail on a name that does resolve in
|
||||||
|
-- production.
|
||||||
|
CREATE EXTENSION IF NOT EXISTS "uuid-ossp" SCHEMA extensions;
|
||||||
|
CREATE EXTENSION IF NOT EXISTS "pgcrypto" SCHEMA extensions;
|
||||||
|
|
||||||
|
-- Enough of auth.users for foreign keys to resolve. Real Supabase has far more
|
||||||
|
-- columns; migrations only reference id and email.
|
||||||
|
CREATE TABLE IF NOT EXISTS auth.users (
|
||||||
|
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
email text
|
||||||
|
);
|
||||||
|
|
||||||
|
-- In production these read the request's JWT claims. Here they return NULL,
|
||||||
|
-- which is fine for CREATE POLICY (only the expression has to type-check) and
|
||||||
|
-- is why this must not touch a real database.
|
||||||
|
CREATE OR REPLACE FUNCTION auth.uid() RETURNS uuid
|
||||||
|
LANGUAGE sql STABLE AS $$ SELECT NULL::uuid $$;
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION auth.role() RETURNS text
|
||||||
|
LANGUAGE sql STABLE AS $$ SELECT NULL::text $$;
|
||||||
|
|
||||||
|
CREATE OR REPLACE FUNCTION auth.email() RETURNS text
|
||||||
|
LANGUAGE sql STABLE AS $$ SELECT NULL::text $$;
|
||||||
|
|
||||||
|
-- storage.objects: three migrations attach policies to it.
|
||||||
|
CREATE TABLE IF NOT EXISTS storage.objects (
|
||||||
|
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
bucket_id text,
|
||||||
|
name text,
|
||||||
|
owner uuid
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS storage.buckets (
|
||||||
|
id text PRIMARY KEY,
|
||||||
|
name text
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Roles the migrations GRANT to. CREATE ROLE is not idempotent, hence the guard.
|
||||||
|
DO $$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'anon') THEN
|
||||||
|
CREATE ROLE anon NOLOGIN;
|
||||||
|
END IF;
|
||||||
|
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'authenticated') THEN
|
||||||
|
CREATE ROLE authenticated NOLOGIN;
|
||||||
|
END IF;
|
||||||
|
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'service_role') THEN
|
||||||
|
CREATE ROLE service_role NOLOGIN BYPASSRLS;
|
||||||
|
END IF;
|
||||||
|
-- The dump carries ALTER DEFAULT PRIVILEGES FOR ROLE supabase_admin, which
|
||||||
|
-- requires membership in that role rather than just its existence.
|
||||||
|
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'supabase_admin') THEN
|
||||||
|
CREATE ROLE supabase_admin NOLOGIN;
|
||||||
|
END IF;
|
||||||
|
END $$;
|
||||||
|
|
||||||
|
GRANT supabase_admin TO CURRENT_USER;
|
||||||
|
|
||||||
|
GRANT USAGE ON SCHEMA public, auth, storage TO anon, authenticated, service_role;
|
||||||
|
|
||||||
|
-- Supabase ships a realtime publication that migrations add tables to.
|
||||||
|
-- Without it, ALTER PUBLICATION fails on the environment rather than on
|
||||||
|
-- anything the migration got wrong.
|
||||||
|
DO $$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (SELECT 1 FROM pg_publication WHERE pubname = 'supabase_realtime') THEN
|
||||||
|
CREATE PUBLICATION supabase_realtime;
|
||||||
|
END IF;
|
||||||
|
END $$;
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,53 @@
|
|||||||
|
# Archived migrations
|
||||||
|
|
||||||
|
Superseded by `../00000000000000_baseline.sql` on 2026-08-01. Kept, not deleted:
|
||||||
|
they are the only record of why parts of the schema look the way they do, even
|
||||||
|
where they no longer describe it.
|
||||||
|
|
||||||
|
**Do not run these.** They are not a working migration set and several are unsafe
|
||||||
|
to replay.
|
||||||
|
|
||||||
|
## Why they were replaced
|
||||||
|
|
||||||
|
These files were applied by hand through the Supabase SQL editor over months —
|
||||||
|
most still carry `-- Run this in Supabase SQL Editor` headers. Some runs stopped
|
||||||
|
partway and were recorded as applied regardless, so the recorded history stopped
|
||||||
|
matching the database. Three things made that concrete:
|
||||||
|
|
||||||
|
**Functions declared but absent.** 25 functions across six migrations, all marked
|
||||||
|
applied, did not exist in the database. Their tables did. Two were reached by the
|
||||||
|
app, so merchant tier upgrades and post-trade reputation updates had been quietly
|
||||||
|
dead. Found on 2026-07-30 only because a user hit
|
||||||
|
`Could not find the function public.upsert_user_profile(...)` while toggling a
|
||||||
|
notification setting.
|
||||||
|
|
||||||
|
**Conflicting definitions.** `admin_roles` is defined three different ways and
|
||||||
|
production matches none of them:
|
||||||
|
|
||||||
|
| source | columns |
|
||||||
|
|---|---|
|
||||||
|
| `001_initial_schema.sql` | id, user_id, role, granted_by, granted_at |
|
||||||
|
| `COMBINED_p2p_full_system.sql` | user_id, role, created_at |
|
||||||
|
| production | id, user_id, role, permissions, created_at, updated_at |
|
||||||
|
|
||||||
|
**Not replayable.** Applied to an empty database, five of them fail. The legacy
|
||||||
|
`0NN` filenames sort before the 14-digit timestamps they depend on — `"013"` <
|
||||||
|
`"20241117054600"` — so `013` runs before the migration that creates the table it
|
||||||
|
alters. That ordering could never have worked from scratch.
|
||||||
|
|
||||||
|
## What replaced them
|
||||||
|
|
||||||
|
A `pg_dump` of the live `public` schema, which matches production by
|
||||||
|
construction. Verified: applying it to an empty database produces 83 tables, 39
|
||||||
|
functions, 222 indexes and 360 policies — identical counts to production.
|
||||||
|
|
||||||
|
CI now applies migrations to an empty Postgres on every PR, so a migration that
|
||||||
|
cannot run from scratch fails there instead of on the production host. That check
|
||||||
|
was impossible while this set was the starting point.
|
||||||
|
|
||||||
|
## If you need something from here
|
||||||
|
|
||||||
|
Read it, take the statement you need, and write a new forward migration against
|
||||||
|
the baseline. Do not re-run the file — `20241117054601` inserts payment methods
|
||||||
|
with no `ON CONFLICT`, `015` has 17 unguarded inserts, and `018` overwrites the
|
||||||
|
live hot wallet address.
|
||||||
Reference in New Issue
Block a user