Dispute spam protection (#4134)

* Mostly notes.

* Better error messages.

* Introduce Fatal/NonFatal + drop back channel participation

- Fatal/NonFatal - in order to make it easier to use utility functions.
- We drop the back channel in dispute participation as it won't be
needed any more.

* Better error messages.

* Utility function for receiving `CandidateEvent`s.

* Ordering module typechecks.

* cargo fmt

* Prepare spam slots module.

* Implement SpamSlots mechanism.

* Implement queues.

* cargo fmt

* Participation.

* Participation taking shape.

* Finish participation.

* cargo fmt

* Cleanup.

* WIP: Cleanup + Integration.

* Make `RollingSessionWindow` initialized by default.

* Make approval voting typecheck.

* Get rid of lazy_static & fix approval voting tests

* Move `SessionWindowSize` to node primitives.

* Implement dispute coordinator initialization.

* cargo fmt

* Make queues return error instead of boolean.

* Initialized: WIP

* Introduce chain api for getting finalized block.

* Fix ordering to only prune candidates on finalized events.

* Pruning of old sessions in spam slots.

* New import logic.

* Make everything typecheck.

* Fix warnings.

* Get rid of obsolete dispute-participation.

* Fixes.

* Add back accidentelly deleted Cargo.lock

* Deliver disputes in an ordered fashion.

* Add module docs for errors

* Use type synonym.

* hidden docs.

* Fix overseer tests.

* Ordering provider taking `CandidateReceipt`.

... To be kicked on one next commit.

* Fix ordering to use relay_parent

as included block is not unique per candidate.

* Add comment in ordering.rs.

* Take care of duplicate entries in queues.

* Better spam slots.

* Review remarks + docs.

* Fix db tests.

* Participation tests.

* Also scrape votes on first leaf for good measure.

* Make tests typecheck.

* Spelling.

* Only participate in actual disputes, not on every import.

* Don't account backing votes to spam slots.

* Fix more tests.

* Don't participate if we don't have keys.

* Fix tests, typos and warnings.

* Fix merge error.

* Spelling fixes.

* Add missing docs.

* Queue tests.

* More tests.

* Add metrics + don't short circuit import.

* Basic test for ordering provider.

* Import fix.

* Remove dead link.

* One more dead link.

Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
This commit is contained in:
Robert Klotzner
2021-11-19 18:08:21 +01:00
committed by GitHub
parent ef3addb6a2
commit 25974f2076
45 changed files with 4099 additions and 2621 deletions
@@ -4,4 +4,4 @@ The approval subsystems implement the node-side of the [Approval Protocol](../..
We make a divide between the [assignment/voting logic](approval-voting.md) and the [distribution logic](approval-distribution.md) that distributes assignment certifications and approval votes. The logic in the assignment and voting also informs the GRANDPA voting rule on how to vote.
These subsystems are intended to flag issues and begin [participating in live disputes](../disputes/dispute-participation.md). Dispute subsystems also track all observed votes (backing, approval, and dispute-specific) by all validators on all candidates.
These subsystems are intended to flag issues and begin participating in live disputes. Dispute subsystems also track all observed votes (backing, approval, and dispute-specific) by all validators on all candidates.
@@ -2,7 +2,7 @@
This is the central subsystem of the node-side components which participate in disputes. This subsystem wraps a database which tracks all statements observed by all validators over some window of sessions. Votes older than this session window are pruned.
This subsystem will be the point which produce dispute votes, either positive or negative, based on locally-observed validation results as well as a sink for votes received by other subsystems. When importing a dispute vote from another node, this will trigger the [dispute participation](dispute-participation.md) subsystem to recover and validate the block and call back to this subsystem.
This subsystem will be the point which produce dispute votes, either positive or negative, based on locally-observed validation results as well as a sink for votes received by other subsystems. When importing a dispute vote from another node, this will trigger participation in the dispute.
## Database Schema
@@ -56,11 +56,10 @@ Input: [`DisputeCoordinatorMessage`][DisputeCoordinatorMessage]
Output:
- [`RuntimeApiMessage`][RuntimeApiMessage]
- [`DisputeParticipationMessage`][DisputeParticipationMessage]
## Functionality
This assumes a constant `DISPUTE_WINDOW: SessionIndex`. This should correspond to at least 1 day.
This assumes a constant `DISPUTE_WINDOW: SessionWindowSize`. This should correspond to at least 1 day.
Ephemeral in-memory state:
@@ -75,8 +74,7 @@ struct State {
Check DB for recorded votes for non concluded disputes we have not yet
recorded a local statement for.
For all of those send `DisputeParticipationMessage::Participate` message to
dispute participation subsystem.
For all of those initiate dispute participation.
### On `OverseerSignal::ActiveLeavesUpdate`
@@ -171,4 +169,3 @@ Do nothing.
[DisputeStatement]: ../../types/disputes.md#disputestatement
[DisputeCoordinatorMessage]: ../../types/overseer-protocol.md#dispute-coordinator-message
[RuntimeApiMessage]: ../../types/overseer-protocol.md#runtime-api-message
[DisputeParticipationMessage]: ../../types/overseer-protocol.md#dispute-participation-message
@@ -21,9 +21,9 @@ This design should result in a protocol that is:
### Output
- [`DisputeCoordinatorMessage::ActiveDisputes`][DisputeParticipationMessage]
- [`DisputeCoordinatorMessage::ImportStatements`][DisputeParticipationMessage]
- [`DisputeCoordinatorMessage::QueryCandidateVotes`][DisputeParticipationMessage]
- [`DisputeCoordinatorMessage::ActiveDisputes`][DisputeCoordinatorMessage]
- [`DisputeCoordinatorMessage::ImportStatements`][DisputeCoordinatorMessage]
- [`DisputeCoordinatorMessage::QueryCandidateVotes`][DisputeCoordinatorMessage]
- [`RuntimeApiMessage`][RuntimeApiMessage]
### Wire format
@@ -357,4 +357,3 @@ no real harm done: There was no serious attack to begin with.
[DisputeDistributionMessage]: ../../types/overseer-protocol.md#dispute-distribution-message
[RuntimeApiMessage]: ../../types/overseer-protocol.md#runtime-api-message
[DisputeParticipationMessage]: ../../types/overseer-protocol.md#dispute-participation-message
@@ -1,68 +0,0 @@
# Dispute Participation
This subsystem is responsible for actually participating in disputes: when notified of a dispute, we need to recover the candidate data, validate the candidate, and cast our vote in the dispute.
Fortunately, most of that work is handled by other subsystems; this subsystem is just a small glue component for tying other subsystems together and issuing statements based on their validity.
## Protocol
Input: [`DisputeParticipationMessage`][DisputeParticipationMessage]
Output:
- [`RuntimeApiMessage`][RuntimeApiMessage]
- [`CandidateValidationMessage`][CandidateValidationMessage]
- [`AvailabilityRecoveryMessage`][AvailabilityRecoveryMessage]
- [`AvailabilityStoreMessage`][AvailabilityStoreMessage]
- [`ChainApiMessage`][ChainApiMessage]
## Functionality
In-memory state:
```rust
struct State {
recent_block_hash: Option<(BlockNumber, Hash)>
}
```
### On `OverseerSignal::ActiveLeavesUpdate`
Update `recent_block` in in-memory state according to the highest observed active leaf.
### On `OverseerSignal::BlockFinalized`
Do nothing.
### On `OverseerSignal::Conclude`
Conclude.
### On `DisputeParticipationMessage::Participate`
* Decompose into parts: `{ candidate_hash, candidate_receipt, session, voted_indices }`
* Issue an [`AvailabilityRecoveryMessage::RecoverAvailableData`][AvailabilityRecoveryMessage]
* Report back availability result to the `AvailabilityRecoveryMessage` sender
via the `report_availability` oneshot.
* If the result is `Unavailable`, return.
* If the result is `Invalid`, [cast invalid votes](#cast-votes) and return.
* If the data is recovered, dispatch a [`RuntimeApiMessage::ValidationCodeByHash`][RuntimeApiMessage] with the parameters `(candidate_receipt.descriptor.validation_code_hash)` at `state.recent_block.hash`.
* Dispatch a [`AvailabilityStoreMessage::StoreAvailableData`][AvailabilityStoreMessage] with the data.
* If the code is not fetched from the chain, return. This should be impossible with correct relay chain configuration, at least if chain synchronization is working correctly.
* Dispatch a [`CandidateValidationMessage::ValidateFromExhaustive`][CandidateValidationMessage] with the available data and the validation code and `APPROVAL_EXECUTION_TIMEOUT` as the timeout parameter.
* If the validation result is `Invalid`, [cast invalid votes](#cast-votes) and return.
* If the validation fails, [cast invalid votes](#cast-votes) and return.
* If the validation succeeds, compute the `CandidateCommitments` based on the validation result and compare against the candidate receipt's `commitments_hash`. If they match, [cast valid votes](#cast-votes) and if not, [cast invalid votes](#cast-votes).
### Cast Votes
This requires the parameters `{ candidate_receipt, candidate_hash, session, voted_indices }` as well as a choice of either `Valid` or `Invalid`.
Invoke [`DisputeCoordinatorMessage::IssueLocalStatement`][DisputeCoordinatorMessage] with `is_valid` according to the parameterization,.
[RuntimeApiMessage]: ../../types/overseer-protocol.md#runtime-api-message
[DisputeParticipationMessage]: ../../types/overseer-protocol.md#dispute-participation-message
[DisputeCoordinatorMessage]: ../../types/overseer-protocol.md#dispute-coordinator-message
[CandidateValidationMessage]: ../../types/overseer-protocol.md#candidate-validation-message
[AvailabilityRecoveryMessage]: ../../types/overseer-protocol.md#availability-recovery-message
[ChainApiMessage]: ../../types/overseer-protocol.md#chain-api-message
[AvailabilityStoreMessage]: ../../types/overseer-protocol.md#availability-store-message