Commit Graph

13 Commits

Author SHA1 Message Date
Robert Klotzner 25974f2076 Dispute spam protection (#4134)
* Mostly notes.

* Better error messages.

* Introduce Fatal/NonFatal + drop back channel participation

- Fatal/NonFatal - in order to make it easier to use utility functions.
- We drop the back channel in dispute participation as it won't be
needed any more.

* Better error messages.

* Utility function for receiving `CandidateEvent`s.

* Ordering module typechecks.

* cargo fmt

* Prepare spam slots module.

* Implement SpamSlots mechanism.

* Implement queues.

* cargo fmt

* Participation.

* Participation taking shape.

* Finish participation.

* cargo fmt

* Cleanup.

* WIP: Cleanup + Integration.

* Make `RollingSessionWindow` initialized by default.

* Make approval voting typecheck.

* Get rid of lazy_static & fix approval voting tests

* Move `SessionWindowSize` to node primitives.

* Implement dispute coordinator initialization.

* cargo fmt

* Make queues return error instead of boolean.

* Initialized: WIP

* Introduce chain api for getting finalized block.

* Fix ordering to only prune candidates on finalized events.

* Pruning of old sessions in spam slots.

* New import logic.

* Make everything typecheck.

* Fix warnings.

* Get rid of obsolete dispute-participation.

* Fixes.

* Add back accidentelly deleted Cargo.lock

* Deliver disputes in an ordered fashion.

* Add module docs for errors

* Use type synonym.

* hidden docs.

* Fix overseer tests.

* Ordering provider taking `CandidateReceipt`.

... To be kicked on one next commit.

* Fix ordering to use relay_parent

as included block is not unique per candidate.

* Add comment in ordering.rs.

* Take care of duplicate entries in queues.

* Better spam slots.

* Review remarks + docs.

* Fix db tests.

* Participation tests.

* Also scrape votes on first leaf for good measure.

* Make tests typecheck.

* Spelling.

* Only participate in actual disputes, not on every import.

* Don't account backing votes to spam slots.

* Fix more tests.

* Don't participate if we don't have keys.

* Fix tests, typos and warnings.

* Fix merge error.

* Spelling fixes.

* Add missing docs.

* Queue tests.

* More tests.

* Add metrics + don't short circuit import.

* Basic test for ordering provider.

* Import fix.

* Remove dead link.

* One more dead link.

Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2021-11-19 17:08:21 +00:00
Bernhard Schuster 30bdc8f5d4 collect included disputes from on-chain (#3924)
* dummy: impl another runtime API

* query the on chain disputes, and inform self

* make use of the refactor

* minro

* SPLIT ME

* write dispute values

* wip

* impl for all runtimes

* chore: fmt

* [] -> get

* fixup mock runtime

* fixup

* fixup discovery for overseer init

* chore: fmt

* spellcheck

* rename imported_on_chain_disputes -> on_chain_votes

* reduction

* make it mockable

* rename and refactor

* don't query on chain info if it's not needed

* yikes

* fmt

* fix test

* minimal fix for existing tests

* attempt to fetch the session info from the rolling window before falling back

* moved

* comments

* comments

* test for backing votes

* rename

* Update runtime/polkadot/src/lib.rs

* chore: spellcheck + dict

* chore: fmt

* fixup cache size

* add warning

* logging, rationale, less defense

* introduce new unchecked, that still checks in debug builds

* fix

* draft alt approach

* fix unused imports

* include the session

* Update node/core/dispute-coordinator/src/real/mod.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* provide where possible

* expand comment

* fixin

* fixup

* ValidityVote <-> ValidityAttestation <-> CompactStatement has a 1:1 representation

* mark TODO

* Update primitives/src/v1/mod.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* address review comments

* update docs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2021-10-06 19:16:23 +00:00
Robert Habermeier 6002865874 Make candidate validation timeouts configurable (#4001)
* pvf: make execution timeout configurable

* guide: add timeouts to candidate validation params

* add timeouts to candidate validation messages

* fmt

* port backing to use the backing pvf timeout

* port approval-voting to use the execution timeout

* port dispute participation to use the correct timeout

* fmt

* address grumbles & test failure
2021-10-04 09:53:36 -05:00
Andronik Ordian 33fe763188 some fixes to please cargo-spellcheck (#3550)
* some fixes to please cargo-spellcheck

* some (not all) fixes for the impl guide

* fix
2021-08-02 11:00:04 +00:00
Denis Pisarev fc253e6e4d WIP: CI: add spellcheck (#3421)
* CI: add spellcheck

* revert me

* CI: explicit command for spellchecker

* spellcheck: edit misspells

* CI: run spellcheck on diff

* spellcheck: edits

* spellcheck: edit misspells

* spellcheck: add rules

* spellcheck: mv configs

* spellcheck: more edits

* spellcheck: chore

* spellcheck: one more thing

* spellcheck: and another one

* spellcheck: seems like it doesn't get to an end

* spellcheck: new words after rebase

* spellcheck: new words appearing out of nowhere

* chore

* review edits

* more review edits

* more edits

* wonky behavior

* wonky behavior 2

* wonky behavior 3

* change git behavior

* spellcheck: another bunch of new edits

* spellcheck: new words are koming out of nowhere

* CI: finding the master

* CI: fetching master implicitly

* CI: undebug

* new errors

* a bunch of new edits

* and some more

* Update node/core/approval-voting/src/approval_db/v1/mod.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update xcm/xcm-executor/src/assets.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <write@reusable.software>

* Suggestions from the code review

* CI: scan only changed files

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-07-14 19:22:58 +02:00
Lldenaurois 2d66b8f256 Dispute Coordinator: Batch queries (#3459)
* disputes: Allow batch queries in dispute-coordinator

This commit moves to batch queries when responding to QueryCandidateVotes
messages. This simplifies the code in the provisioner and dispute-coordinator
by no longer requiring to make use of a FuturesOrdered when awaiting multiple
quries. Instead, the provisioner need only request the batch itself.

* node/approval-voting: Address Feedback to fail on query element missing.

* Address feedback

* Fix implementer's guide
2021-07-12 20:06:14 -05:00
Robert Habermeier e512222749 Wire up candidate backing, approval-voting to disputes (#3348)
* add a from_backing_statement to SignedDisputeStatement

* inform dispute coordinator of all backing statements

* add dispute coordinator message to backing tests

* send positive dispute statement with every approval

* issue disputes when encountering invalid candidates.

* try to fix flaky test for CI (passed locally)

* guide: keep track of concluded-positive disputes until pruned

* guide: block implications

* guide: new dispute inherent flow

* mostly implement recency changes for dispute coordinator

* add a clock to dispute coordinator

* adjust DB tests

* fix and add new dispute coordinator tests

* provisioner: select disputes

* import all validators' approvals

* address nit: refactor backing statement submission

* gracefully handle disconnected dispute coordinator

* remove `review` comment

* fix up old_tests

* fix approval-voting compilation

* fix backing compilation

* use known-leaves in WaitForActivation

* follow-up test fixing

* add back allow(dead_code)
2021-07-09 21:15:51 +00:00
Robert Klotzner 20993b32b1 Guide updates for disputes. (#3401)
* Guide updates for disputes.

* Working availability recovery flood protection.

* More fixes.

* Formatting.

* Fix.

* Update roadmap/implementers-guide/src/node/disputes/dispute-participation.md

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Review remarks.

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2021-07-08 23:19:08 +02:00
Robert Klotzner e3340e44b5 Dispute distribution guide (#3158)
* Dispute distribution initial design.

* WIP.

* Dispute distribution guide update.

* Make invalid statement include `InvalidStatementKind`.

* Clarify the scope of disputes.

* A few fixes + introduced back pressure oneshot.

* Fixes and spam protection WIP.

* More spam considerations.

* More fixes.

* Fixes + add note about not dispute participating nodes.
2021-06-22 18:23:53 -05:00
André Silva d8d61bad36 implement dispute participation subsystem (#3234)
* implement dispute participation subsystem

* guide: minor fix in dispute participation

* Update node/core/dispute-participation/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* dispute: add comments to participation logic

* dispute-coordinator: fix test compilation

* implementers-guide: update dispute participation

* dispute-participation: add error for missing validation code

* dispute-participation: add tests

* Update node/core/dispute-participation/src/lib.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* guide: update overseer protocol dispute participation message

* dispute-participation: remove duplication in tests

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2021-06-14 15:41:02 -05:00
Robert Habermeier 5bc2b2779d Dispute Coordinator Subsystem (#3150)
* skeleton for dispute-coordinator

* add coordinator and participation message types

* begin dispute-coordinator DB

* functions for loading

* implement strongly-typed DB transaction

* add some tests for DB transaction

* core logic for pruning

* guide: update candidate-votes key for coordinator

* update candidate-votes key

* use big-endian encoding for session, and implement upper bound generator

* finish implementing pruning

* add a test for note_current_session

* define state of the subsystem itself

* barebones subsystem definition

* control flow

* more control flow

* implement session-updating logic

* trace

* control flow for message handling

* Update node/core/dispute-coordinator/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update node/subsystem/src/messages.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* some more control flow

* guide: remove overlay

* more control flow

* implement some DB getters

* make progress on importing statements

* add SignedDisputeStatement struct

* move ApprovalVote to shared primitives

* add a signing-payload API to explicit dispute statements

* add signing-payload to CompactStatement

* add relay-parent hash to seconded/valid dispute variatns

* correct import

* type-safe wrapper around dispute statements

* use checked dispute statement in message type

* extract rolling session window cache to subsystem-util

* extract session window tests

* approval-voting: use rolling session info cache

* reduce dispute window to match runtime in practice

* add byzantine_threshold and supermajority_threshold utilities to primitives

* integrate rolling session window

* Add PartialOrd to CandidateHash

* add Ord to CandidateHash

* implement active dispute update

* add dispute messages to AllMessages

* add dispute stubs to overseer

* inform dispute participation to participate

* implement issue_local_statement

* implement `determine_undisputed_chain`

* fix warnings

* test harness for dispute coordinator tests

* add more helpers to test harness

* add some more helpers

* some tests for dispute coordinator

* ignore wrong validator indices

* test finality voting rule constraint

* add more tests

* add variants to network bridge

* fix test compilation

* remove most dispute coordinator functionality

as of #3222 we can do most of the work within the approval voting subsystem

* Revert "remove most dispute coordinator functionality"

This reverts commit 9cd615e8eb6ca0b382cbaff525d813e753d6004e.

* Use thiserror

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Update node/core/dispute-coordinator/src/lib.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* extract tests to separate module

* address nit

* adjust run_iteration API

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
2021-06-13 06:35:18 -05:00
André Silva 8c3bc2b2bb replace HistoricalValidationCode usages with ValidationCodeByHash (#3210)
* replace HistoricalValidationCode usages with ValidationCodeByHash

* runtime-api: tabify tests file

* update implementers guide
2021-06-11 18:00:57 +00:00
Robert Habermeier ccfabaa0c6 Node-side subsystems for Disputes (#2566)
* dispute subsystem files

* rename

* fix linkcheck

* flesh out section README

* coordinator schema

* DisputeCoordinatorMessage

* stub & coordinator protocol

* dispute coordinator

* add some more message fields

* move links to bottom

* dispute participation

* Cleen It Up !

* runtime: store candidate receipts in dispute state

yeah, this is a little heavier. why are you reading this?

* Revert "runtime: store candidate receipts in dispute state"

This reverts commit 51c10bfd4d866e287e6bd88f317ed57ed987eaee.

* add dispute availability statement type and prepare for availability

* add 'spam slots' to disputes runtmie

* return Spam Slots info from runtime

* rework `ImportStatement` to `ImportStatements`

* some more methods for dispute coordinator

* candidates-included runtime API

* algo for providing disputes to runtime.

* handle signing with coordinator

* dispute coordinator chain ops

* remove dead file

* remove keystore from dispute participation

* adjust ApprovedAncestor to return the necssary data

* discuss how approved ancestor and determine undisputed chain are used together

* add TODO

* initiate disputes from approval voting

* route statements from candidate backing and approval voting

* fix guide build
2021-04-01 03:00:46 +02:00