I am dumb and can't spell (#1366)

* rename implementor's guide to implementer's guide * fix typos in more places
2026-06-13 17:31:05 +00:00 · 2020-07-07 10:10:36 -04:00
parent 37da08a764
commit 42bd096413
54 changed files with 5 additions and 5 deletions
@@ -0,0 +1,3 @@
+# Utility Subsystems
+
+The utility subsystems are an assortment which don't have a natural home in another subsystem collection.
@@ -0,0 +1,52 @@
+# Availability Store
+
+This is a utility subsystem responsible for keeping available certain data and pruning that data.
+
+The two data types:
+
+- Full PoV blocks of candidates we have validated
+- Availability chunks of candidates that were backed and noted available on-chain.
+
+For each of these data we have pruning rules that determine how long we need to keep that data available.
+
+PoV hypothetically only need to be kept around until the block where the data was made fully available is finalized. However, disputes can revert finality, so we need to be a bit more conservative. We should keep the PoV until a block that finalized availability of it has been finalized for 1 day.
+
+> TODO: arbitrary, but extracting `acceptance_period` is kind of hard here...
+
+Availability chunks need to be kept available until the dispute period for the corresponding candidate has ended. We can accomplish this by using the same criterion as the above, plus a delay. This gives us a pruning condition of the block finalizing availability of the chunk being final for 1 day + 1 hour.
+
+> TODO: again, concrete acceptance-period would be nicer here, but complicates things
+
+There is also the case where a validator commits to make a PoV available, but the corresponding candidate is never backed. In this case, we keep the PoV available for 1 hour.
+
+> TODO: ideally would be an upper bound on how far back contextual execution is OK.
+
+There may be multiple competing blocks all ending the availability phase for a particular candidate. Until (and slightly beyond) finality, it will be unclear which of those is actually the canonical chain, so the pruning records for PoVs and Availability chunks should keep track of all such blocks.
+
+## Protocol
+
+Input: [`AvailabilityStoreMessage`](../../types/overseer-protocol.md#availability-store-message)
+
+## Functionality
+
+On `StartWork`:
+
+- Note any new candidates backed in the block. Update pruning records for any stored `PoVBlock`s.
+- Note any newly-included candidates backed in the block. Update pruning records for any stored availability chunks.
+
+On block finality events:
+
+- > TODO: figure out how we get block finality events from overseer
+- Handle all pruning based on the newly-finalized block.
+
+On `QueryPoV` message:
+
+- Return the PoV block, if any, for that candidate hash.
+
+On `QueryChunk` message:
+
+- Determine if we have the chunk indicated by the parameters and return it and its inclusion proof via the response channel if so.
+
+On `StoreChunk` message:
+
+- Store the chunk along with its inclusion proof under the candidate hash and validator index.
@@ -0,0 +1,23 @@
+# Candidate Validation
+
+This subsystem is responsible for handling candidate validation requests. It is a simple request/response server.
+
+A variety of subsystems want to know if a parachain block candidate is valid. None of them care about the detailed mechanics of how a candidate gets validated, just the results. This subsystem handles those details.
+
+## Protocol
+
+Input: [`CandidateValidationMessage`](../../types/overseer-protocol.md#validation-request-type)
+
+Output: Validation result via the provided response side-channel.
+
+## Functionality
+
+Given the hashes of a relay parent and a parachain candidate block, and either its PoV or the information with which to retrieve the PoV from the network, spawn a short-lived async job to determine whether the candidate is valid.
+
+Each job follows this process:
+
+- Get the full candidate from the current relay chain state
+- Check the candidate's proof
+   > TODO: that's extremely hand-wavey. What does that actually entail?
+- Generate either `Statement::Valid` or `Statement::Invalid`. Note that this never generates `Statement::Seconded`; Candidate Backing is the only subsystem which upgrades valid to seconded.
+- Return the statement on the provided channel.
@@ -0,0 +1,7 @@
+# Misbehavior Arbitration
+
+The Misbehavior Arbitration subsystem collects reports of validator misbehavior, and slashes the stake of both misbehaving validator nodes and false accusers.
+
+> TODO: It is not yet fully specified; that problem is postponed to a future PR.
+
+One policy question we've decided even so: in the event that MA has to call all validators to check some block about which some validators disagree, the minority voters all get slashed, and the majority voters all get rewarded. Validators which abstain have a minor slash penalty, but probably not in the same order of magnitude as those who vote wrong.
@@ -0,0 +1,46 @@
+# Network Bridge
+
+One of the main features of the overseer/subsystem duality is to avoid shared ownership of resources and to communicate via message-passing. However, implementing each networking subsystem as its own network protocol brings a fair share of challenges.
+
+The most notable challenge is coordinating and eliminating race conditions of peer connection and disconnection events. If we have many network protocols that peers are supposed to be connected on, it is difficult to enforce that a peer is indeed connected on all of them or the order in which those protocols receive notifications that peers have connected. This becomes especially difficult when attempting to share peer state across protocols. All of the Parachain-Host's gossip protocols eliminate DoS with a data-dependency on current chain heads. However, it is inefficient and confusing to implement the logic for tracking our current chain heads as well as our peers' on each of those subsystems. Having one subsystem for tracking this shared state and distributing it to the others is an improvement in architecture and efficiency.
+
+One other piece of shared state to track is peer reputation. When peers are found to have provided value or cost, we adjust their reputation accordingly.
+
+So in short, this Subsystem acts as a bridge between an actual network component and a subsystem's protocol.
+
+## Protocol
+
+Input: [`NetworkBridgeMessage`](../../types/overseer-protocol.md#network-bridge-message)
+Output: Varying, based on registered event producers.
+
+## Functionality
+
+Track a set of all Event Producers, each associated with a 4-byte protocol ID.
+There are two types of network messages this sends and receives:
+
+- ProtocolMessage(ProtocolId, Bytes)
+- ViewUpdate(View)
+
+`StartWork` and `StopWork` determine the computation of our local view. A `ViewUpdate` is issued to each connected peer, and a `NetworkBridgeUpdate::OurViewChange` is issued for each registered event producer.
+
+On `RegisterEventProducer`:
+
+- Add the event producer to the set of event producers. If there is a competing entry, ignore the request.
+
+On `ProtocolMessage` arrival:
+
+- If the protocol ID matches an event producer, produce the message from the `NetworkBridgeEvent::PeerMessage(sender, bytes)`, otherwise ignore and reduce peer reputation slightly
+- dispatch message via overseer.
+
+On `ViewUpdate` arrival:
+
+- Do validity checks and note the most recent view update of the peer.
+- For each event producer, dispatch the result of a `NetworkBridgeEvent::PeerViewChange(view)` via overseer.
+
+On `ReportPeer` message:
+
+- Adjust peer reputation according to cost or benefit provided
+
+On `SendMessage` message:
+
+- Issue a corresponding `ProtocolMessage` to each listed peer with given protocol ID and bytes.
@@ -0,0 +1,9 @@
+# Peer Set Manager
+
+> TODO
+
+## Protocol
+
+## Functionality
+
+## Jobs, if any
@@ -0,0 +1,62 @@
+# Provisioner
+
+Relay chain block authorship authority is governed by BABE and is beyond the scope of the Overseer and the rest of the subsystems. That said, ultimately the block author needs to select a set of backable parachain candidates and other consensus data, and assemble a block from them. This subsystem is responsible for providing the necessary data to all potential block authors.
+
+A major feature of the provisioner: this subsystem is responsible for ensuring that parachain block candidates are sufficiently available before sending them to potential block authors.
+
+## Provisionable Data
+
+There are several distinct types of provisionable data, but they share this property in common: all should eventually be included in a relay chain block.
+
+### Backed Candidates
+
+The block author can choose 0 or 1 backed parachain candidates per parachain; the only constraint is that each backed candidate has the appropriate relay parent. However, the choice of a backed candidate must be the block author's; the provisioner must ensure that block authors are aware of all available [`BackedCandidate`s](../../types/backing.md#backed-candidate).
+
+### Signed Bitfields
+
+[Signed bitfields](../../types/availability.md#signed-availability-bitfield) are attestations from a particular validator about which candidates it believes are available.
+
+### Misbehavior Reports
+
+Misbehavior reports are self-contained proofs of misbehavior by a validator or group of validators. For example, it is very easy to verify a double-voting misbehavior report: the report contains two votes signed by the same key, advocating different outcomes. Concretely, misbehavior reports become inherents which cause dots to be slashed.
+
+Note that there is no mechanism in place which forces a block author to include a misbehavior report which it doesn't like, for example if it would be slashed by such a report. The chain's defense against this is to have a relatively long slash period, such that it's likely to encounter an honest author before the slash period expires.
+
+### Dispute Inherent
+
+The dispute inherent is similar to a misbehavior report in that it is an attestation of misbehavior on the part of a validator or group of validators. Unlike a misbehavior report, it is not self-contained: resolution requires coordinated action by several validators. The canonical example of a dispute inherent involves an approval checker discovering that a set of validators has improperly approved an invalid parachain block: resolving this requires the entire validator set to re-validate the block, so that the minority can be slashed.
+
+Dispute resolution is complex and is explained in substantially more detail [here](../../runtime/validity.md).
+
+> TODO: The provisioner is responsible for selecting remote disputes to replay. Let's figure out the details.
+
+## Protocol
+
+Input: [`ProvisionerMessage`](../../types/overseer-protocol.md#provisioner-message). Backed candidates come from the [Candidate Backing subsystem](../backing/candidate-backing.md), signed bitfields come from the [Bitfield Distribution subsystem](../availability/bitfield-distribution.md), and misbehavior reports and disputes come from the [Misbehavior Arbitration subsystem](misbehavior-arbitration.md).
+
+At initialization, this subsystem has no outputs. Block authors can send a `ProvisionerMessage::RequestBlockAuthorshipData`, which includes a channel over which provisionable data can be sent. All appropriate provisionable data will then be sent over this channel, as it is received.
+
+Note that block authors must re-send a `ProvisionerMessage::RequestBlockAuthorshipData` for each relay parent they are interested in receiving provisionable data for.
+
+## Block Production
+
+When a validator is selected by BABE to author a block, it becomes a block producer. The provisioner is the subsystem best suited to choosing which specific backed candidates and availability bitfields should be assembled into the block. To engage this functionality, a `ProvisionerMessage::RequestInherentData` is sent; the response is a set of non-conflicting candidates and the appropriate bitfields. Non-conflicting generally means that there are never two distinct parachain candidates included for the same parachain.
+
+One might ask: given `ProvisionerMessage::RequestInherentData`, what's the point of `ProvisionerMessage::RequestBlockAuthorshipData`? The answer is that the block authorship data includes more information than is present in the inherent data; disputes, for example.
+
+## Functionality
+
+The subsystem should maintain a set of handles to Block Authorship Provisioning Jobs that are currently live.
+
+### On Overseer Signal
+
+- `StartWork`: spawn a Block Authorship Provisioning Job with the given relay parent, storing a bidirectional channel with that job.
+- `StopWork`: terminate the Block Authorship Provisioning Job for the given relay parent, if any.
+
+### On `ProvisionerMessage`
+
+Forward the message to the appropriate Block Authorship Provisioning Job, or discard if no appropriate job is currently active.
+
+## Block Authorship Provisioning Job
+
+Maintain the set of channels to block authors. On receiving provisionable data, send a copy over each channel.
@@ -0,0 +1,19 @@
+# Runtime API
+
+The Runtime API subsystem is responsible for providing a single point of access to runtime state data via a set of pre-determined queries. This prevents shared ownership of a blockchain client resource by providing
+
+## Protocol
+
+Input: [`RuntimeApiMessage`](../../types/overseer-protocol.md#runtime-api-message)
+
+Output: None
+
+## Functionality
+
+On receipt of `RuntimeApiMessage::Request(relay_parent, request)`, answer the request using the post-state of the relay_parent provided and provide the response to the side-channel embedded within the request.
+
+> TODO Do some caching. The underlying rocksdb already has a cache of trie nodes so duplicate requests are unlikely to hit disk. Not required for functionality.
+
+## Jobs
+
+> TODO Don't limit requests based on parent hash, but limit caching. No caching should be done for any requests on relay_parents that are not live based on `StartWork` or `StopWork` messages. Maybe with some leeway for things that have just been stopped.