Reversion Safety tools for overseer and subsystems (#3104)

* guide: reversion safety * guide: manage reversion safety in subsystems * add leaf status to ActivatedLeaf * add an LRU-cache to overseer for staleness detection * update ActivatedLeaf usages in tests to contain status field * add variant where missed accidentally * add some helpers to LeafStatus * address grumbles
2026-05-31 05:11:02 +00:00 · 2021-05-31 13:54:05 -05:00
parent 6b166a7a1f
commit 963993d288
18 changed files with 270 additions and 20 deletions
@@ -170,7 +170,7 @@ On receiving an `OverseerSignal::BlockFinalized(h)`, we fetch the block number `
 #### `OverseerSignal::ActiveLeavesUpdate`

 On receiving an `OverseerSignal::ActiveLeavesUpdate(update)`:
-  * We determine the set of new blocks that were not in our previous view. This is done by querying the ancestry of all new items in the view and contrasting against the stored `BlockNumber`s. Typically, there will be only one new block. We fetch the headers and information on these blocks from the ChainApi subsystem.
+  * We determine the set of new blocks that were not in our previous view. This is done by querying the ancestry of all new items in the view and contrasting against the stored `BlockNumber`s. Typically, there will be only one new block. We fetch the headers and information on these blocks from the ChainApi subsystem. Stale leaves in the update can be ignored.
  * We update the `StoredBlockRange` and the `BlockNumber` maps.
  * We use the RuntimeApiSubsystem to determine information about these blocks. It is generally safe to assume that runtime state is available for recent, unfinalized blocks. In the case that it isn't, it means that we are catching up to the head of the chain and needn't worry about assignments to those blocks anyway, as the security assumption of the protocol tolerates nodes being temporarily offline or out-of-date.
    * We fetch the set of candidates included by each block by dispatching a `RuntimeApiRequest::CandidateEvents` and checking the `CandidateIncluded` events.
@@ -15,7 +15,7 @@ Output:

 ## Functionality

-Upon receipt of an `ActiveLeavesUpdate`, launch bitfield signing job for each `activated` head. Stop the job for each `deactivated` head.
+Upon receipt of an `ActiveLeavesUpdate`, launch bitfield signing job for each `activated` head referring to a fresh leaf. Stop the job for each `deactivated` head.

 ## Bitfield Signing Job

@@ -32,7 +32,7 @@ The subsystem should maintain a set of handles to Candidate Backing Jobs that ar
 ### On Overseer Signal

 * If the signal is an [`OverseerSignal`][OverseerSignal]`::ActiveLeavesUpdate`:
-  * spawn a Candidate Backing Job for each `activated` head, storing a bidirectional channel with the Candidate Backing Job in the set of handles.
+  * spawn a Candidate Backing Job for each `activated` head referring to a fresh leaf, storing a bidirectional channel with the Candidate Backing Job in the set of handles.
  * cease the Candidate Backing Job for each `deactivated` head, if any.
 * If the signal is an [`OverseerSignal`][OverseerSignal]`::Conclude`: Forward conclude messages to all jobs, wait a small amount of time for them to join, and then exit.

@@ -26,6 +26,8 @@ The hierarchy of subsystems:

 The overseer determines work to do based on block import events and block finalization events. It does this by keeping track of the set of relay-parents for which work is currently being done. This is known as the "active leaves" set. It determines an initial set of active leaves on startup based on the data on-disk, and uses events about blockchain import to update the active leaves. Updates lead to [`OverseerSignal`](../types/overseer-protocol.md#overseer-signal)`::ActiveLeavesUpdate` being sent according to new relay-parents, as well as relay-parents to stop considering. Block import events inform the overseer of leaves that no longer need to be built on, now that they have children, and inform us to begin building on those children. Block finalization events inform us when we can stop focusing on blocks that appear to have been orphaned.

+The overseer is also responsible for tracking the freshness of active leaves. Leaves are fresh when they're encountered for the first time, and stale when they're encountered for subsequent times. This can occur after chain reversions or when the fork-choice rule abandons some chain. This distinction is used to manage **Reversion Safety**. Consensus messages are often localized to a specific relay-parent, and it is often a misbehavior to equivocate or sign two conflicting messages. When reverting the chain, we may begin work on a leaf that subsystems have already signed messages for. Subsystems which need to account for reversion safety should avoid performing work on stale leaves.
+
 The overseer's logic can be described with these functions:

 ## On Startup
@@ -38,6 +40,7 @@ The overseer's logic can be described with these functions:
 ## On Block Import Event

 * Apply the block import event to the active leaves. A new block should lead to its addition to the active leaves set and its parent being deactivated.
+* Mark any stale leaves as stale. The overseer should track all leaves it activates to determine whether leaves are fresh or stale.
 * Send an `OverseerSignal::ActiveLeavesUpdate` message to all subsystems containing all activated and deactivated leaves.
 * Ensure all `ActiveLeavesUpdate` messages are flushed before resuming activity as a message router.

@@ -29,8 +29,18 @@ Either way, there will be some top-level type encapsulating messages from the ov
 Indicates a change in active leaves. Activated leaves should have jobs, whereas deactivated leaves should lead to winding-down of work based on those leaves.

 ```rust
+enum LeafStatus {
+    // A leaf is fresh when it's the first time the leaf has been encountered.
+    // Most leaves should be fresh.
+    Fresh,
+    // A leaf is stale when it's encountered for a subsequent time. This will
+    // happen when the chain is reverted or the fork-choice rule abandons some
+    // chain.
+    Stale,
+}
+
 struct ActiveLeavesUpdate {
-    activated: [(Hash, Number)], // in practice, these should probably be a SmallVec
+    activated: [(Hash, Number, LeafStatus)], // in practice, these should probably be a SmallVec
    deactivated: [Hash],
 }
 ```