Sync the lists with polkadot-js/phishing every day, and publish them with checks (#4)

* feat: sync the lists with polkadot-js/phishing daily, and check them before publishing

The lists had not moved since January 2026, the site that serves them was
copied by hand, and the rebrand had turned five real phishing domains into
pezkuwichain.* names, unblocking them.

- scripts/syncUpstream.mjs and sync-upstream.yml take upstream's lists every
  day, lay pezkuwi/overlay.json over them, and commit to master. A day out of
  bounds (too many sites unblocked or added, scam addresses dropped, one of our
  domains blocked, one of our entries lost) writes nothing and opens an issue.
- pezkuwi/overlay.json holds our own entries with their reasons: the five
  pezkuwichain.* names stay blocked on purpose, and our domains are protected.
- scripts/lists.mjs holds the rules; scripts/lists.test.mjs tests them and the
  sync end to end; scripts/checkLists.mjs checks the committed files, including
  that they are what sortAll writes. lists.yml runs all three.
- pezkuwi/host/ is what phishing.pezkuwichain.io runs: it pulls master hourly
  and refuses a rewritten history, a blocked own domain, a shrink over 2%, or
  shards that disagree, with a sandbox test of each refusal.
- sortAll strips a trailing dot. Upstream's 817 "scam.xyz." entries were filed
  under an empty top-level domain in all/all.json, where checkIfDenied never
  looks; 816 repeated a listed name, and prenads.xyz was blocked nowhere else.
- sortAll uses Node's own directory functions instead of @pezkuwi/dev, so the
  sync and the checks run without installing the build toolchain.
- The auto-approve, auto-merge and crosscheck workflows, and the IPFS job,
  depended on tokens and secrets this repository does not have; removed. The
  remaining workflows are pinned to commits, with no default permissions and
  a timeout on every job.

* chore(lists): first sync with polkadot-js/phishing

Written by scripts/syncUpstream.mjs from upstream bf5e7c6:
658 sites added; nodelinkapp.xyz, org.in and terra-claim.com unblocked by
upstream; the five polkadot-js domains the rebrand had renamed blocked again;
816 duplicate "scam.xyz." entries folded into their names and the stray
all/all.json gone. 54,051 blocked sites, 279 scam addresses.

* fix(lists): commit the .build shard, and refuse a shard git ignores

.gitignore held "build/", which also matched all/build/all.json, the shard of
the .build top-level domain. Every sort wrote it and no commit held it, so the
eight .build scam sites were never published; upstream has the same gap. The
directory is excluded from the ignore rule, the shard is committed, and
checkLists.mjs now fails on any shard git would ignore (with the old rule
restored it does).

Also clears the repository's lint on lists.test.mjs: imports sorted, and
node:test's describe and it calls marked void.

* test(lists): await describe and it, as the repository's lint requires

no-floating-promises requires the promises node:test's describe and it return
to be handled, and no-void forbids handling them with void. Awaited, the 25
cases run as before.
This commit is contained in:
SatoshiQaziMuhammed authored and GitHub committed 2026-09-13 17:34:51 +04:00
1 parent 70bbc740cf
commit 767a92904d
112 files changed
+5311 -4811

No files matched your search

-16
View File
@@ -1,16 +0,0 @@
name: bot
on:
pull_request:
types: [labeled]
jobs:
approve:
if: "! startsWith(github.event.head_commit.message, '[CI Skip]') && (!github.event.pull_request || github.event.pull_request.head.repo.full_name == github.repository)"
runs-on: ubuntu-latest
steps:
- uses: jacogr/action-approve@795afd1dd096a2071d7ec98740661af4e853b7da
with:
authors: jacogr, TarikGul
labels: -auto
token: ${{ secrets.GH_PAT_BOT }}
-16
View File
@@ -1,16 +0,0 @@
name: bot
on:
pull_request:
types: [labeled]
jobs:
merge:
runs-on: ubuntu-latest
steps:
- uses: jacogr/action-merge@d2d64b4545acd93b0a9575177d3d215ae3f92029
with:
checks: pr (build),pr (lint),pr (test)
labels: -auto
strategy: squash
token: ${{ secrets.GH_PAT_BOT }}
-49
View File
@@ -1,49 +0,0 @@
name: crosscheck
on:
schedule:
- cron: '45 5/1 * * *'
jobs:
crosscheck:
runs-on: ubuntu-latest
env:
YARN_ENABLE_SCRIPTS: false
CI_LOG: 123
GITHUB_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 'lts/*'
- name: check
run: |
yarn install --immutable
yarn phishing:crosscheck
- name: issue
if: ${{ failure() }}
uses: JasonEtco/create-an-issue@e27dddc79c92bc6e4562f268fffa5ed752639abd
with:
filename: .github/crosscheck.md
update_existing: true
addrcheck:
runs-on: ubuntu-latest
env:
YARN_ENABLE_SCRIPTS: false
CI_LOG: 123
GITHUB_TOKEN: ${{ github.token }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 'lts/*'
- name: check
run: |
yarn install --immutable
yarn phishing:addrcheck
- name: issue
if: ${{ failure() }}
uses: JasonEtco/create-an-issue@e27dddc79c92bc6e4562f268fffa5ed752639abd
with:
filename: .github/addrcheck.md
update_existing: true
+34
View File
@@ -0,0 +1,34 @@
name: Lists
# The rules every change to the lists must pass: scripts/lists.test.mjs for the rules and the
# sync, scripts/checkLists.mjs for the committed files. Nothing to install, so this check stands
# on its own when the build toolchain does not.
on:
pull_request:
push:
branches: [master]
permissions: {}
jobs:
lists:
name: lists
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Rules for the lists and the sync
run: node --test scripts/lists.test.mjs
- name: The committed lists
run: node scripts/checkLists.mjs
# What phishing.pezkuwichain.io runs before it serves a new master: against a scratch
# repository and web root, with these lists.
- name: The host's publish checks
run: bash pezkuwi/host/test-publish.sh
+7 -1
View File
@@ -4,13 +4,19 @@ on:
schedule:
- cron: '10 2/3 * * *'
permissions: {}
jobs:
lock:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
issues: write
pull-requests: write
env:
YARN_ENABLE_SCRIPTS: false
steps:
- uses: dessant/lock-threads@c1b35aecc5cdb1a34539d14196df55838bb2f836
- uses: dessant/lock-threads@c1b35aecc5cdb1a34539d14196df55838bb2f836 # v4.0.0
with:
github-token: ${{ github.token }}
issue-inactive-days: '7'
+10 -4
View File
@@ -1,6 +1,8 @@
name: PR
on: [pull_request]
permissions: {}
jobs:
pr:
continue-on-error: true
@@ -8,15 +10,19 @@ jobs:
matrix:
step: ['lint', 'test', 'build', 'deno']
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
env:
YARN_ENABLE_SCRIPTS: false
CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 'lts/*'
- uses: denoland/setup-deno@v1
- uses: denoland/setup-deno@22d081ff2d3a40755e97629de92e3bcbfa7cf2ed # v2.0.5
with:
deno-version: vx.x.x
- name: ${{ matrix.step }}
+8 -26
View File
@@ -4,45 +4,27 @@ on:
branches:
- master
permissions: {}
jobs:
master:
if: "! startsWith(github.event.head_commit.message, '[CI Skip]')"
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
env:
CI_LOG: 123
YARN_ENABLE_SCRIPTS: false
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- uses: actions/setup-node@v4
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 'lts/*'
- name: Install and Build
run: |
yarn install
yarn build
# publish to ipfs when a release is detected
ipfs:
if: "startsWith(github.event.head_commit.message, '[CI Skip] release/stable')"
strategy:
matrix:
step: ['build:release:ipfs']
name: ${{ matrix.step }}
runs-on: ubuntu-latest
env:
YARN_ENABLE_SCRIPTS: false
CF_API_TOKEN: ${{ secrets.CF_API_TOKEN }}
PINATA_API_KEY: ${{ secrets.PINATA_API_KEY }}
PINATA_SECRET_KEY: ${{ secrets.PINATA_SECRET_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 'lts/*'
- name: ${{ matrix.step }}
run: |
yarn install --immutable
./scripts/ipfsPrep.sh
yarn ${{ matrix.step }}
+82
View File
@@ -0,0 +1,82 @@
name: Sync upstream lists
# Once a day, bring polkadot-js/phishing's lists here with our own entries laid over them
# (pezkuwi/overlay.json). scripts/syncUpstream.mjs writes nothing when the day is out of bounds:
# too many sites unblocked or added, one of our domains blocked, one of our entries lost. Then
# this job opens an issue instead, and a person decides. A day within bounds is committed to
# master directly (decision of 2026-09-13); phishing.pezkuwichain.io picks master up on its own
# (pezkuwi/host/).
on:
schedule:
- cron: "23 4 * * *"
workflow_dispatch:
permissions: {}
concurrency:
group: sync-upstream
cancel-in-progress: false
jobs:
sync:
name: sync
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write
issues: write
steps:
# No credentials in the checkout: the sync and the checks process upstream's data, and only
# the commit step, below, is handed the token it pushes with.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Fetch upstream
run: git fetch --no-tags --depth=1 https://github.com/polkadot-js/phishing.git master
- name: Sync
id: sync
run: |
set -uo pipefail
node scripts/syncUpstream.mjs FETCH_HEAD 2>&1 | tee sync.out
echo "rc=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT"
- name: Check
if: steps.sync.outputs.rc == '0'
run: |
node --test scripts/lists.test.mjs
node scripts/checkLists.mjs
- name: Commit to master
if: steps.sync.outputs.rc == '0'
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
if [ -z "$(git status --porcelain -- all.json address.json known.json all meta pezkuwi/upstream.json)" ]; then
echo "upstream lists have not moved"
exit 0
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -A -- all.json address.json known.json all meta pezkuwi/upstream.json
git commit -q -m "chore(lists): sync with polkadot-js/phishing" -m "$(grep -E '^(SYNC |unblocked: )' sync.out)"
gh auth setup-git
git push origin HEAD:master
- name: Ask a person
if: steps.sync.outputs.rc != '0'
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
title="Upstream list sync stopped and needs a person"
body="$(printf 'The daily sync refused to write the lists.\n\n```\n%s\n```\n\nRun: %s/%s/actions/runs/%s\n' \
"$(tail -n 40 sync.out)" "$GITHUB_SERVER_URL" "$GITHUB_REPOSITORY" "$GITHUB_RUN_ID")"
open="$(gh issue list --state open --search "in:title \"$title\"" --json number --jq '.[0].number // empty')"
if [ -n "$open" ]; then gh issue comment "$open" --body "$body"; else gh issue create --title "$title" --body "$body"; fi
exit 1