* CI: add spellcheck
* revert me
* CI: explicit command for spellchecker
* spellcheck: edit misspells
* CI: run spellcheck on diff
* spellcheck: edits
* spellcheck: edit misspells
* spellcheck: add rules
* spellcheck: mv configs
* spellcheck: more edits
* spellcheck: chore
* spellcheck: one more thing
* spellcheck: and another one
* spellcheck: seems like it doesn't get to an end
* spellcheck: new words after rebase
* spellcheck: new words appearing out of nowhere
* chore
* review edits
* more review edits
* more edits
* wonky behavior
* wonky behavior 2
* wonky behavior 3
* change git behavior
* spellcheck: another bunch of new edits
* spellcheck: new words are koming out of nowhere
* CI: finding the master
* CI: fetching master implicitly
* CI: undebug
* new errors
* a bunch of new edits
* and some more
* Update node/core/approval-voting/src/approval_db/v1/mod.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update xcm/xcm-executor/src/assets.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* Suggestions from the code review
* CI: scan only changed files
Co-authored-by: Andronik Ordian <write@reusable.software>
* node/approval-voting: Introduce Backend trait and Overlaybackend
This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.
* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight
* node/approval-voting: Add load_all_blocks to overlay
* node/approval-voting: Get all module tests to pass.
This commit modifies all tests to ensure tests are passing.
* node/approval-voting: Address oversights in the previous commit
This commit addresses some oversights in the prior commit.
1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general
* node/approval-voting: Move from TestDB to dyn KeyValueDB
This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.
* node/approval-voting: Address feedback
* node/approval-voting: Add license to ops.rs
* node/approval-voting: Address second-pass feedback
* Add TODO
* node/approval-voting: Bump spec_version
* node/approval-voting: Address final comments.
* node/approval-voting: Introduce framework for full subsystem tests
* node/approval-voting: Introduce basic tests to attempt to provide
coverage via full subsystem tests
* node/approval-voting: Introduce Chainbuilder
* node/approval-voting: Introduce Backend trait and Overlaybackend
This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.
* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight
* node/approval-voting: Add load_all_blocks to overlay
* node/approval-voting: Get all module tests to pass.
This commit modifies all tests to ensure tests are passing.
* node/approval-voting: Address oversights in the previous commit
This commit addresses some oversights in the prior commit.
1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general
* node/approval-voting: Move from TestDB to dyn KeyValueDB
This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.
* node/approval-voting: Address feedback
* node/approval-voting: Add license to ops.rs
* node/approval-voting: Address second-pass feedback
* Add TODO
* node/approval-voting: Bump spec_version
* node/approval-voting: Address final comments.
* DB skeleton
* key formats
* lexicographic test
* custom types for DB
* implement backend for db-v1
* remove VoidBackend and integrate with real DbBackend
* detect stagnant blocks on in interval
* fix tests
* add tests for stagnant
* send ChainSelectionMessage::Approved
* tests for DB backend
* unused import
* upgrade kvdb-memorydb
Co-authored-by: Andronik Ordian <write@reusable.software>
* node/approval-voting: Introduce LruCache for pending Approval work
This commit adds an LruCache that is intended to track the approval work
submitted as background tasks in order to ensure that the validator needn't
launch duplicate approval work for the same candidate across multiple blocks.
A simple state machine is also introduced in order to differentiate pending
and completed tasks. In addition, this LruCache will retain ValidationResults
from the completed approval work once the task has completed. As per LruCache
implementation, the oldest tasks will get evicted as new approval work is
submitted to this cache.
* node/approval-voting: Revert changes to master
This commit reverts changes from the previous commit in order
to simplify addressing the architecture discussion raised in the PR.
* node/approval-voting: remove background task mpsc construct
This diff removes the mpsc construct for background tasks in preparation
for a move to leveraging RemoteHandles to launch approvals, rather than
passing ApprovalRequests to a mpsc channel and handling the ApprovalRequests
in the main subsystem task.
* node/approval-voting: Introduce LRU Cache
This commit introduces an LRU Cache but does not yet make use of it.
* node/approval-voting: Remove BackgroundTasksMap and memoize currently_checking
This commit removes the BackgroundTasksMap in the main subsystem task
and introduces a method to keep track of RemoteHandles in such a way that
we can ensure that a task is spawned once for a CandidateHash and
relay parent tuple.
* node/approval-voting: Remove BackgroundTasksMap and memoize currently_checking
This commit removes the BackgroundTasksMap in the main subsystem task
and introduces a map of FuturesUnordered per BlockNumber. In addition,
a FusedFuture is generated by iterating across all FuturesUnordered for
the BlockNumbers for which at least one candidate has approvals work
running in the subsystem.
* node/approval-voting: Address Rob's comments
This diff removes the prior HashMap<BlockNumber, FuturesUnordered>
construction and instead moves to a simple FuturesUnordered where
all the work is await with Timeout.
* node/approval-voting: Update Cargo.lock
Due to a mismatch in rustc versions
* node/approval-voting: Make use of actions when issuing_approval
This commit fixes a small oversight in the logic of the prior commit.
* node/approval-voting: Address Rob's feedback
* node/approval-voting: Introduce lazy launch_approval evaluation
* node/approval-voting: Send DistibruteApproval message on every LaunchApproval
In addition to fixed the DistributeApproval bug, this commit also
increases the size of the approvals cache and ensures the StaleGuard
is removed when the advantageous approval state is reached.
* node/approval-voting: Address final comments
This commit removes the CandidateIndex from the ApprovalVoteRequest.
Instead, the launch_approval function will compute the candidate_index
from the block entry.
In addition, a comment has been added explaining the difficulty of
issuing approvals in the handle_actions function.
* node/approval-voting: Set timeout to be 120s rather than 2s
* Update Cargo.lock
* node/approval-voting: test for invalid validator index in assignments
This commit adds a unit test to show that, currently, validator indexes
greater than n_validators (or the length of the approvals bitvector) are
counted in n_assignments. In the subsequent commit we will correct this
behavior.
* node/approval-voting: ignore invalid validator indexes in n_assignments
This commit ignores any validator assignments whose index is beyond
n_validators. Without this check, an improperly crafted assignment would
be counted towards the approval.
It still remains that n_assignments and count_no_shows inspect the
number of validators and approvals, respectively. Ideally we would
add greater safety around ensuring these two values cannot differ.
* node/approval-voting: extract and test count_no_shows method
This commit extracts no_show computation into a pure function so that it can be
extensively unit tested.
* node/approval-voting: ignore invalid validator indexes in count_no_show
Previously indexes that were past the length of the approvals bitvector
would contribute to the no_show count or the next_no_show value. This
commit changes the behavior to ignore garbage values.
* node/approval-voting: add comment for next_no_show adding clock_drift
* node/approval_checking: break out filled_tranch_iterator method
In the subsequent commit, we will begin to test this method in
isolation.
* node/approval-voting: fix tranche back-filling algorithm
Previously, this algorithm would generate duplicate, empty entries for
tranches (1..pre_end). This is caused because the initial value (0) for
gap_end is treated as the end of a prior tranche that wasn't actually
processed. The first pass thus would add (1..tranche) empty entries, in
addition to the (0..pre_end) empty entries chained at the end of the
method.
This is fixed by using the current tranche as the gap_start for the
first iteration, ensuring that the approval_entries_filled only produces
entries in the range (pre_end..post_start).
* Address feedback
* skeleton for dispute-coordinator
* add coordinator and participation message types
* begin dispute-coordinator DB
* functions for loading
* implement strongly-typed DB transaction
* add some tests for DB transaction
* core logic for pruning
* guide: update candidate-votes key for coordinator
* update candidate-votes key
* use big-endian encoding for session, and implement upper bound generator
* finish implementing pruning
* add a test for note_current_session
* define state of the subsystem itself
* barebones subsystem definition
* control flow
* more control flow
* implement session-updating logic
* trace
* control flow for message handling
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update node/subsystem/src/messages.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* some more control flow
* guide: remove overlay
* more control flow
* implement some DB getters
* make progress on importing statements
* add SignedDisputeStatement struct
* move ApprovalVote to shared primitives
* add a signing-payload API to explicit dispute statements
* add signing-payload to CompactStatement
* add relay-parent hash to seconded/valid dispute variatns
* correct import
* type-safe wrapper around dispute statements
* use checked dispute statement in message type
* extract rolling session window cache to subsystem-util
* extract session window tests
* approval-voting: use rolling session info cache
* reduce dispute window to match runtime in practice
* add byzantine_threshold and supermajority_threshold utilities to primitives
* integrate rolling session window
* Add PartialOrd to CandidateHash
* add Ord to CandidateHash
* implement active dispute update
* add dispute messages to AllMessages
* add dispute stubs to overseer
* inform dispute participation to participate
* implement issue_local_statement
* implement `determine_undisputed_chain`
* fix warnings
* test harness for dispute coordinator tests
* add more helpers to test harness
* add some more helpers
* some tests for dispute coordinator
* ignore wrong validator indices
* test finality voting rule constraint
* add more tests
* add variants to network bridge
* fix test compilation
* remove most dispute coordinator functionality
as of #3222 we can do most of the work within the approval voting subsystem
* Revert "remove most dispute coordinator functionality"
This reverts commit 9cd615e8eb6ca0b382cbaff525d813e753d6004e.
* Use thiserror
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* extract tests to separate module
* address nit
* adjust run_iteration API
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Simplify some Option / Result / ? operator patterns
When they identically match a combinator on those types.
Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
* adjust review comments
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* extract database from av-store itself
* generalize approval-voting over database type
* modes (without handling) and pruning old wakeups
* rework approval importing
* add our_approval_sig to ApprovalEntry
* import assignment
* guide updates for check-full-approval changes
* some aux functions
* send messages when becoming active.
* guide: network bridge sends view updates only when done syncing
* network bridge: send view updates only when done syncing
* tests for new network-bridge behavior
* add a test for updating approval entry with sig
* fix some warnings
* test load-all-blocks
* instantiate new parachains DB
* fix network-bridge empty view updates
* tweak
* fix wasm build, i think
* Update node/core/approval-voting/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* add some versioning to parachains_db
* warnings
* fix merge changes
* remove versioning again
Co-authored-by: Andronik Ordian <write@reusable.software>
* guide updates
* keep interactions alive until receivers drop
* retry indefinitely
* cancel approval tasks on finality
* use swap_remove instead of remove
* Remove stuff out of the runtime that does not belong there.
There might be more, but it is a start.
* White space fixes.
* Fix tests.
* Leave whitespace in ui tests alone.
* Add back zstd for no reason.
* Fix browser wasm (hopefully)
* overseer: pass messages directly between subsystems
* test that message is held on to
* Update node/overseer/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* give every subsystem an unbounded sender too
* remove metered_channel::name
1. we don't provide good names
2. these names are never used anywhere
* unused mut
* remove unnecessary &mut
* subsystem unbounded_send
* remove unused MaybeTimer
We have channel size metrics that serve the same purpose better now and the implementation of message timing was pretty ugly.
* remove comment
* split up senders and receivers
* update metrics
* fix tests
* fix test subsystem context
* use SubsystemSender in jobs system now
* refactor of awful jobs code
* expose public `run` on JobSubsystem
* update candidate backing to new jobs & use unbounded
* bitfield signing
* candidate-selection
* provisioner
* approval voting: send unbounded for assignment/approvals
* async not needed
* begin bridge split
* split up network tasks into background worker
* port over network bridge
* Update node/network/bridge/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* rename ValidationWorkerNotifications
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
* add number to `ActivatedLeavesUpdate`
* update subsystem util and overseer
* use new ActivatedLeaf everywhere
* sort view
* sorted and limited view in network bridge
* use live block hash only if it's newer
* grumples
Chevdor