fix(chains): drop node URLs that embed Nova's provider keys (#72)

* fix(chains): drop node URLs that embed Nova's Infura keys

Nova's configs carry Nova's own Infura project keys inside node URLs, and
every sync copied them into ours: Ethereum had four keyed Infura nodes and
Base one, across 43 published files. Our Android build already fills
`{INFURA_API_KEY}` from our own secret; the keyed URLs used another
organisation's quota and would break for every user when Nova rotates them.

- sync_from_nova.py strips nodes whose URL embeds an Infura key, and drops a
  chain left with no node (only Ethereum - Goerli, a retired testnet, in the
  dev files); merge-chains.py applies the same filter
- the published chain files are rewritten with that filter; nothing else in
  them changes (each file round-trips byte for byte through json.dump)
- check_pezkuwi_integrity.py refuses a keyed Infura URL under chains/

* fix(chains): strip every keyed provider URL, and make the gate fail closed

The first pass matched only `infura.io/.../v3/<lowercase hex>`, so it missed
`https://...infura.io/v3/<key>`, upper-case keys and hosts, and Nova's
OnFinality `?apikey=` URLs (Moonriver, and the Hades testnet in v20/v21
preConfigured). preConfigured was copied from Nova without the filter, and the
integrity check passed if grep itself failed.

- one predicate, is_keyed(), in sync_from_nova.py: Infura /v3/ ids and
  apikey/api_key/key/token query values, case-insensitive; {PLACEHOLDER}
  values are allowed
- preConfigured details are filtered after the copy; a chain left with no
  node loses its details file and its index entry (Hades, v20/v21 only)
- rewritten files keep their own indent, so the diff is the data change only
- check 5 imports is_keyed() and checks every string in every JSON file
  under chains/; an unreadable file or an empty tree is an error
- check 3 reports a grep error instead of passing on it
This commit is contained in:
SatoshiQaziMuhammed authored and GitHub committed 2026-10-17 20:55:00 +04:00
1 parent 8bd31ac686
commit 5ca2661eb2
59 files changed
+113 -1365

No files matched your search

+42 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Refuse the four regressions this repo has already had.
"""Refuse the five regressions this repo has already had.
Each check below exists because the thing it forbids happened, was fixed, and in two
cases came back. They are cheap; the failures they catch are not.
@@ -88,6 +88,8 @@ grep = subprocess.run(
capture_output=True,
text=True,
)
if grep.returncode not in (0, 1):
errors.append(f"grep for working-branch asset URLs failed: {grep.stderr.strip()}")
if grep.stdout.strip():
files = grep.stdout.strip().splitlines()
errors.append(
@@ -107,6 +109,43 @@ for fixture in ("pezkuwi_assets_for_testBalance.json",):
f"tests/{fixture} has drifted from pezkuwi-overlay/tests/{fixture}"
)
# ── 5. No third-party API key in a URL ───────────────────────────────────────
#
# Nova's configs shipped Nova's own provider keys (Infura project ids, OnFinality
# apikeys) inside node URLs, and every sync copied them into ours. Our builds fill
# {INFURA_API_KEY}-style placeholders from our own secrets; a keyed URL is someone
# else's quota and dies when they rotate it. The predicate is the one the sync
# uses, and every string in every JSON file is checked, so a file that does not
# parse is a failure, not a pass.
sys.path.insert(0, str(ROOT / "scripts"))
from sync_from_nova import is_keyed # noqa: E402
keyed_files = []
json_files = sorted(p for p in (ROOT / "chains").rglob("*.json"))
if not json_files:
errors.append("chains/ has no JSON files; the key check would have checked nothing")
for path in json_files:
try:
doc = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
errors.append(f"{path.relative_to(ROOT)} could not be read for the key check: {exc}")
continue
stack = [doc]
while stack:
node = stack.pop()
if isinstance(node, dict):
stack.extend(node.values())
elif isinstance(node, list):
stack.extend(node)
elif isinstance(node, str) and is_keyed(node):
keyed_files.append(path)
break
if keyed_files:
errors.append(
f"{len(keyed_files)} file(s) under chains/ embed a provider key in a URL; "
f"use a {{PLACEHOLDER}}, e.g. {keyed_files[0].relative_to(ROOT)}"
)
if errors:
for e in errors:
print(f"::error::{e}")
@@ -114,4 +153,5 @@ if errors:
sys.exit(1)
print("Pezkuwi integrity: on-chain assets only, sufficiency declared, "
"assets served from master, fixture in step with its source.")
"assets served from master, fixture in step with its source, "
"no third-party key in a node URL.")