noter: report a scan group in one batch, count progress as liveness (#34)

The first scan after the Zagros People upgrade re-confirmed all 98
accounts, one transaction per account, a block each. It ran past the
15-minute staleness window and the watchdog killed it halfway; the next
interval had meanwhile started a second scan beside it, which read the
same accounts before the first one's reports landed and sent them again
(124 submissions for 98 accounts). On mainnet every tracked citizen's
re-confirmation falls due in the same scan, every three days.

- A scan group's updates go out as one utility.forceBatch. The accounts
  are unrelated, so one refused report no longer takes the others with
  it; each refused item is logged with its decoded error.
- The heartbeat records progress -- a group read and, if it had anything
  to report, reported -- instead of a finished scan. A group that cannot
  be read, or whose report is refused outright, is not progress, so a
  mute noter still goes stale.
- A periodic scan does not start while the previous one is running.

noter/submit.js

@@ -18,9 +18,9 @@ const TERMINAL = ['isDropped', 'isInvalid', 'isUsurped', 'isFinalityTimeout'];
  * later retracted is not lost, because the next scan finds neither a pending nor a cached
  * entry and submits again. Rejects on a dispatch error, on a terminal status, and when nothing
  * has settled after `timeoutMs`, which also covers a status subscription the node stopped
- * answering.
+ * answering. `onInBlock`, if given, sees the block's events for this extrinsic first.
  */
-export function watchSubmission(tx, signer, { decodeError, timeoutMs }) {
+export function watchSubmission(tx, signer, { decodeError, timeoutMs, onInBlock }) {
   return new Promise((resolve, reject) => {
     let unsub = null;
     let settled = false;
@@ -36,10 +36,14 @@ export function watchSubmission(tx, signer, { decodeError, timeoutMs }) {
       timeoutMs,
     );
 
-    tx.signAndSend(signer, ({ status, dispatchError }) => {
+    tx.signAndSend(signer, ({ status, dispatchError, events }) => {
       if (status.isInBlock) {
-        if (dispatchError) settle(reject, new Error(decodeError(dispatchError)));
-        else settle(resolve, status.asInBlock.toHex());
+        if (dispatchError) {
+          settle(reject, new Error(decodeError(dispatchError)));
+        } else {
+          if (!settled && onInBlock) onInBlock(events ?? []);
+          settle(resolve, status.asInBlock.toHex());
+        }
         return;
       }
       const terminal = TERMINAL.find((flag) => status[flag]);
@@ -65,3 +69,25 @@ export function isStale({ lastScanAt, startedAt, now, staleAfter }) {
   const since = Math.max(Number.isFinite(lastScanAt) ? lastScanAt : 0, startedAt);
   return now - since > staleAfter;
 }
+
+/**
+ * Which calls of a `utility.forceBatch` failed, by their position in the batch.
+ *
+ * `forceBatch` runs every call whatever the others do and reports each one, in call order, as
+ * `ItemCompleted` or `ItemFailed`. A failed item does not fail the transaction, so nothing else
+ * would ever surface it.
+ */
+export function failedBatchItems(events) {
+  const failed = [];
+  let index = 0;
+  for (const { event } of events) {
+    if (event.section !== 'utility') continue;
+    if (event.method === 'ItemCompleted') {
+      index++;
+    } else if (event.method === 'ItemFailed') {
+      failed.push({ index, error: event.data[0] });
+      index++;
+    }
+  }
+  return failed;
+}