mirror of
https://github.com/pezkuwichain/pezkuwi-subquery.git
synced 2026-10-20 18:56:24 +00:00
Stop one lost extrinsic from silencing the noter bot (#30)
Every report waits behind the previous one, and a submission only settled when it reached a block. One the pool dropped, rejected or replaced never settled, so every report queued after it waited forever. The watchdog meant to catch a silent bot was armed only after the first scan returned, so when that first scan was the one that hung there was no watchdog at all: the production container logged its last line one minute after starting on 2026-10-13 and stayed up, silent, for more than a day. - submit.js: a submission settles on a terminal status (Dropped, Invalid, Usurped, FinalityTimeout) or after SUBMIT_TIMEOUT_MS, and releases its status subscription. - The watchdog is armed before connecting and counts from the process start until a scan completes, so a heartbeat left by an earlier process in the same container does not count for this one. - The noter's own account is skipped: the chain refuses a noter's report about themselves, so it failed in every scan. - Unit tests for both rules, run inside the noter image in CI.
This commit is contained in:
1 parent
d25afe25bb
commit
1e8d5e3675
5 files changed
+209
-40
No files matched your search
@@ -0,0 +1,67 @@
|
||||
// Watching one extrinsic, and the liveness rule for the scan loop.
|
||||
//
|
||||
// Kept apart from bot.js, which connects to three chains the moment it is imported, so that
|
||||
// both can be tested without a chain.
|
||||
|
||||
/**
|
||||
* Statuses after which an extrinsic will never reach a block. Each of them used to be ignored:
|
||||
* the promise below only settled on `isInBlock`, so a submission the pool dropped, rejected
|
||||
* or replaced left it pending forever, and since every submission waits its turn behind the
|
||||
* previous one, one lost extrinsic stopped every report after it.
|
||||
*/
|
||||
const TERMINAL = ['isDropped', 'isInvalid', 'isUsurped', 'isFinalityTimeout'];
|
||||
|
||||
/**
|
||||
* Sign and submit `tx`, settling once it is in a block or can never be.
|
||||
*
|
||||
* Resolves on `isInBlock` without a dispatch error, as before: an extrinsic on a fork that is
|
||||
* later retracted is not lost, because the next scan finds neither a pending nor a cached
|
||||
* entry and submits again. Rejects on a dispatch error, on a terminal status, and when nothing
|
||||
* has settled after `timeoutMs`, which also covers a status subscription the node stopped
|
||||
* answering.
|
||||
*/
|
||||
export function watchSubmission(tx, signer, { decodeError, timeoutMs }) {
|
||||
return new Promise((resolve, reject) => {
|
||||
let unsub = null;
|
||||
let settled = false;
|
||||
const settle = (fn, value) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
clearTimeout(timer);
|
||||
if (typeof unsub === 'function') unsub();
|
||||
fn(value);
|
||||
};
|
||||
const timer = setTimeout(
|
||||
() => settle(reject, new Error(`no block after ${timeoutMs} ms`)),
|
||||
timeoutMs,
|
||||
);
|
||||
|
||||
tx.signAndSend(signer, ({ status, dispatchError }) => {
|
||||
if (status.isInBlock) {
|
||||
if (dispatchError) settle(reject, new Error(decodeError(dispatchError)));
|
||||
else settle(resolve, status.asInBlock.toHex());
|
||||
return;
|
||||
}
|
||||
const terminal = TERMINAL.find((flag) => status[flag]);
|
||||
if (terminal) settle(reject, new Error(`extrinsic ${status.type ?? terminal.slice(2)}`));
|
||||
})
|
||||
.then((u) => {
|
||||
unsub = u;
|
||||
if (settled) u();
|
||||
})
|
||||
.catch((err) => settle(reject, err));
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the bot has gone too long without completing a scan.
|
||||
*
|
||||
* Counted from the later of the last completed scan and the process start. Counting only from
|
||||
* the last scan left a process whose *first* scan hung with no watchdog at all, because the
|
||||
* watchdog was armed after that scan returned: on 2026-10-13 the production container stopped
|
||||
* logging one minute after it started and stayed up, silent, for more than a day.
|
||||
*/
|
||||
export function isStale({ lastScanAt, startedAt, now, staleAfter }) {
|
||||
const since = Math.max(Number.isFinite(lastScanAt) ? lastScanAt : 0, startedAt);
|
||||
return now - since > staleAfter;
|
||||
}
|
||||
Reference in new issue
Block a user