* WIP: Get rid of request multiplexer.
* WIP
* Receiver for handling of incoming requests.
* Get rid of useless `Fault` abstraction.
The things the type system let us do are not worth getting abstracted in
its own type. Instead error handling is going to be merely a pattern.
* Make most things compile again.
* Port availability distribution away from request multiplexer.
* Formatting.
* Port dispute distribution over.
* Fixup statement distribution.
* Handle request directly in collator protocol.
+ Only allow fatal errors at top level.
* Use direct request channel for availability recovery.
* Finally get rid of request multiplexer
Fixes#2842 and paves the way for more back pressure possibilities.
* Fix overseer and statement distribution tests.
* Fix collator protocol and network bridge tests.
* Fix tests in availability recovery.
* Fix availability distribution tests.
* Fix dispute distribution tests.
* Add missing dependency
* Typos.
* Review remarks.
* More remarks.
* Return `relay_parent` as result of collation seconded signal as well
Before we only returned the seconded statement. However, to verify the
statement in a future proof way, we also need the relay parent that was
used as a context to sign the statement.
* FMT
* CI: add spellcheck
* revert me
* CI: explicit command for spellchecker
* spellcheck: edit misspells
* CI: run spellcheck on diff
* spellcheck: edits
* spellcheck: edit misspells
* spellcheck: add rules
* spellcheck: mv configs
* spellcheck: more edits
* spellcheck: chore
* spellcheck: one more thing
* spellcheck: and another one
* spellcheck: seems like it doesn't get to an end
* spellcheck: new words after rebase
* spellcheck: new words appearing out of nowhere
* chore
* review edits
* more review edits
* more edits
* wonky behavior
* wonky behavior 2
* wonky behavior 3
* change git behavior
* spellcheck: another bunch of new edits
* spellcheck: new words are koming out of nowhere
* CI: finding the master
* CI: fetching master implicitly
* CI: undebug
* new errors
* a bunch of new edits
* and some more
* Update node/core/approval-voting/src/approval_db/v1/mod.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update xcm/xcm-executor/src/assets.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* Suggestions from the code review
* CI: scan only changed files
Co-authored-by: Andronik Ordian <write@reusable.software>
* disputes: Allow batch queries in dispute-coordinator
This commit moves to batch queries when responding to QueryCandidateVotes
messages. This simplifies the code in the provisioner and dispute-coordinator
by no longer requiring to make use of a FuturesOrdered when awaiting multiple
quries. Instead, the provisioner need only request the batch itself.
* node/approval-voting: Address Feedback to fail on query element missing.
* Address feedback
* Fix implementer's guide
* Move on to next validator after timeout.
* Better naming.
* Wrong implementation of validator fetch timeouts.
* Validator side: Move on to next collator
if download takes too long.
* Drop multiple requests from same validator.
* Add test that next response is sent after timeout.
* Multiple requests by same validator should get dropped.
* Test that another collator is tried
after exclusive download time.
* Add dep.
* Cleanup.
* Merge fix.
* Review remarks.
* Fixes.
* Add log targets to trace logs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Only send one collation per relay parent at a time to validators
This changes the way we are sending collations to validators. Before we
answered every collation request immediatley. Now we only answer one
pov request at a time per relay parent. This should bring down the
bandwidth requirements and should help parachains to include bigger
blocks more easily.
* Guide updates
* Review feedback.
* Dispute distribution initial design.
* WIP.
* Dispute distribution guide update.
* Make invalid statement include `InvalidStatementKind`.
* Clarify the scope of disputes.
* A few fixes + introduced back pressure oneshot.
* Fixes and spam protection WIP.
* More spam considerations.
* More fixes.
* Fixes + add note about not dispute participating nodes.
* Only fetch one collation at a time per relay parent
Before a validator would fetch all collations that were advertised to
him. This pr changes the behavior to always just fetch one collation at
a time. If fetching fails, the validator will start fetching one of the
other collations.
* Use enum to be more explicit
* Review comments
* crate skeleton and type definitions
* add ChainSelectionMessage
* add error type
* run loop
* fix overseer
* simplify determine_new_blocks API
* write an overlay struct and fetch new blocks
* add new function to overlay
* more flow
* add leaves to overlay and add a strong type around leaves-set
* add is_parent_viable
* implement block import, ignoring reversions
* add stagnant-at to overlay
* add stagnant
* add revert consensus log
* flow for reversions
* extract and import block reversions
* recursively update viability
* remove redundant parameter from WriteBlockEntry
* do some removal of viable leaves
* address grumbles
* refactor
* address grumbles
* add comment about non-monotonicity
* extract backend to submodule
* begin the hunt for viable leaves
* viability pivots for updating the active leaves
* remove LeafSearchFrontier
* partially -> explicitly viable and untwist some booleans
* extract tree to submodule
* implement block finality update
* Implement block approval routine
* implement stagnant detection
* ensure blocks pruned on finality are removed from the active leaves set
* write down some planned test cases
* floww
* leaf loading
* implement best_leaf_containing
* write down a few more tests to do
* remove dependence of tree on header
* guide: ChainApiMessage::BlockWeight
* node: BlockWeight ChainAPI
* fix compile issue
* note a few TODOs for the future
* fetch block weight using new BlockWeight ChainAPI
* implement unimplemented
* sort leaves by block number after weight
* remove warnings and add more TODOs
* create test module
* storage for test backend
* wrap inner in mutex
* add write waker query to test backend
* Add OverseerSignal -> FromOverseer conversion
* add test harnes
* add no-op test
* add some more test helpers
* the first test
* more progress on tests
* test two subtrees
* determine-new-blocks: cleaner genesis avoidance and tighter ancestry requests
* don't make ancestry requests when asking for one block
* add a couple more tests
* add to AllMessages in guide
* remove bad spaces from bridge
* compact iterator
* test import with gaps
* more reversion tests
* test finalization pruning subtrees
* fixups
* test clobbering and fix bug in overlay
* exhaustive backend state after finalizaiton tested
* more finality tests
* leaf tests
* test approval
* test ChainSelectionMessage::Leaves thoroughly
* remove TODO
* avoid Ordering::is_ne so CI can build
* comment algorithmic complexity
* Update node/core/chain-selection/src/lib.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* skeleton for dispute-coordinator
* add coordinator and participation message types
* begin dispute-coordinator DB
* functions for loading
* implement strongly-typed DB transaction
* add some tests for DB transaction
* core logic for pruning
* guide: update candidate-votes key for coordinator
* update candidate-votes key
* use big-endian encoding for session, and implement upper bound generator
* finish implementing pruning
* add a test for note_current_session
* define state of the subsystem itself
* barebones subsystem definition
* control flow
* more control flow
* implement session-updating logic
* trace
* control flow for message handling
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update node/subsystem/src/messages.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* some more control flow
* guide: remove overlay
* more control flow
* implement some DB getters
* make progress on importing statements
* add SignedDisputeStatement struct
* move ApprovalVote to shared primitives
* add a signing-payload API to explicit dispute statements
* add signing-payload to CompactStatement
* add relay-parent hash to seconded/valid dispute variatns
* correct import
* type-safe wrapper around dispute statements
* use checked dispute statement in message type
* extract rolling session window cache to subsystem-util
* extract session window tests
* approval-voting: use rolling session info cache
* reduce dispute window to match runtime in practice
* add byzantine_threshold and supermajority_threshold utilities to primitives
* integrate rolling session window
* Add PartialOrd to CandidateHash
* add Ord to CandidateHash
* implement active dispute update
* add dispute messages to AllMessages
* add dispute stubs to overseer
* inform dispute participation to participate
* implement issue_local_statement
* implement `determine_undisputed_chain`
* fix warnings
* test harness for dispute coordinator tests
* add more helpers to test harness
* add some more helpers
* some tests for dispute coordinator
* ignore wrong validator indices
* test finality voting rule constraint
* add more tests
* add variants to network bridge
* fix test compilation
* remove most dispute coordinator functionality
as of #3222 we can do most of the work within the approval voting subsystem
* Revert "remove most dispute coordinator functionality"
This reverts commit 9cd615e8eb6ca0b382cbaff525d813e753d6004e.
* Use thiserror
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* extract tests to separate module
* address nit
* adjust run_iteration API
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Robert Klotzner