* node/approval-voting: Introduce Backend trait and Overlaybackend
This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.
* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight
* node/approval-voting: Add load_all_blocks to overlay
* node/approval-voting: Get all module tests to pass.
This commit modifies all tests to ensure tests are passing.
* node/approval-voting: Address oversights in the previous commit
This commit addresses some oversights in the prior commit.
1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general
* node/approval-voting: Move from TestDB to dyn KeyValueDB
This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.
* node/approval-voting: Address feedback
* node/approval-voting: Add license to ops.rs
* node/approval-voting: Address second-pass feedback
* Add TODO
* node/approval-voting: Bump spec_version
* node/approval-voting: Address final comments.
* node/approval-voting: Introduce framework for full subsystem tests
* node/approval-voting: Introduce basic tests to attempt to provide
coverage via full subsystem tests
* node/approval-voting: Introduce Chainbuilder
* node/approval-voting: Introduce Backend trait and Overlaybackend
This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.
* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight
* node/approval-voting: Add load_all_blocks to overlay
* node/approval-voting: Get all module tests to pass.
This commit modifies all tests to ensure tests are passing.
* node/approval-voting: Address oversights in the previous commit
This commit addresses some oversights in the prior commit.
1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general
* node/approval-voting: Move from TestDB to dyn KeyValueDB
This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.
* node/approval-voting: Address feedback
* node/approval-voting: Add license to ops.rs
* node/approval-voting: Address second-pass feedback
* Add TODO
* node/approval-voting: Bump spec_version
* node/approval-voting: Address final comments.
* DB skeleton
* key formats
* lexicographic test
* custom types for DB
* implement backend for db-v1
* remove VoidBackend and integrate with real DbBackend
* detect stagnant blocks on in interval
* fix tests
* add tests for stagnant
* send ChainSelectionMessage::Approved
* tests for DB backend
* unused import
* upgrade kvdb-memorydb
Co-authored-by: Andronik Ordian <write@reusable.software>
* skeleton for dispute-coordinator
* add coordinator and participation message types
* begin dispute-coordinator DB
* functions for loading
* implement strongly-typed DB transaction
* add some tests for DB transaction
* core logic for pruning
* guide: update candidate-votes key for coordinator
* update candidate-votes key
* use big-endian encoding for session, and implement upper bound generator
* finish implementing pruning
* add a test for note_current_session
* define state of the subsystem itself
* barebones subsystem definition
* control flow
* more control flow
* implement session-updating logic
* trace
* control flow for message handling
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update node/subsystem/src/messages.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* some more control flow
* guide: remove overlay
* more control flow
* implement some DB getters
* make progress on importing statements
* add SignedDisputeStatement struct
* move ApprovalVote to shared primitives
* add a signing-payload API to explicit dispute statements
* add signing-payload to CompactStatement
* add relay-parent hash to seconded/valid dispute variatns
* correct import
* type-safe wrapper around dispute statements
* use checked dispute statement in message type
* extract rolling session window cache to subsystem-util
* extract session window tests
* approval-voting: use rolling session info cache
* reduce dispute window to match runtime in practice
* add byzantine_threshold and supermajority_threshold utilities to primitives
* integrate rolling session window
* Add PartialOrd to CandidateHash
* add Ord to CandidateHash
* implement active dispute update
* add dispute messages to AllMessages
* add dispute stubs to overseer
* inform dispute participation to participate
* implement issue_local_statement
* implement `determine_undisputed_chain`
* fix warnings
* test harness for dispute coordinator tests
* add more helpers to test harness
* add some more helpers
* some tests for dispute coordinator
* ignore wrong validator indices
* test finality voting rule constraint
* add more tests
* add variants to network bridge
* fix test compilation
* remove most dispute coordinator functionality
as of #3222 we can do most of the work within the approval voting subsystem
* Revert "remove most dispute coordinator functionality"
This reverts commit 9cd615e8eb6ca0b382cbaff525d813e753d6004e.
* Use thiserror
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* extract tests to separate module
* address nit
* adjust run_iteration API
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* extract database from av-store itself
* generalize approval-voting over database type
* modes (without handling) and pruning old wakeups
* rework approval importing
* add our_approval_sig to ApprovalEntry
* import assignment
* guide updates for check-full-approval changes
* some aux functions
* send messages when becoming active.
* guide: network bridge sends view updates only when done syncing
* network bridge: send view updates only when done syncing
* tests for new network-bridge behavior
* add a test for updating approval entry with sig
* fix some warnings
* test load-all-blocks
* instantiate new parachains DB
* fix network-bridge empty view updates
* tweak
* fix wasm build, i think
* Update node/core/approval-voting/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* add some versioning to parachains_db
* warnings
* fix merge changes
* remove versioning again
Co-authored-by: Andronik Ordian <write@reusable.software>
* WIP
* availability distribution, still very wip.
Work on the requesting side of things.
* Some docs on what I intend to do.
* Checkpoint of session cache implementation
as I will likely replace it with something smarter.
* More work, mostly on cache
and getting things to type check.
* Only derive MallocSizeOf and Debug for std.
* availability-distribution: Cache feature complete.
* Sketch out logic in `FetchTask` for actual fetching.
- Compile fixes.
- Cleanup.
* Format cleanup.
* More format fixes.
* Almost feature complete `fetch_task`.
Missing:
- Check for cancel
- Actual querying of peer ids.
* Finish FetchTask so far.
* Directly use AuthorityDiscoveryId in protocol and cache.
* Resolve `AuthorityDiscoveryId` on sending requests.
* Rework fetch_task
- also make it impossible to check the wrong chunk index.
- Export needed function in validator_discovery.
* From<u32> implementation for `ValidatorIndex`.
* Fixes and more integration work.
* Make session cache proper lru cache.
* Use proper lru cache.
* Requester finished.
* ProtocolState -> Requester
Also make sure to not fetch our own chunk.
* Cleanup + fixes.
* Remove unused functions
- FetchTask::is_finished
- SessionCache::fetch_session_info
* availability-distribution responding side.
* Cleanup + Fixes.
* More fixes.
* More fixes.
adder-collator is running!
* Some docs.
* Docs.
* Fix reporting of bad guys.
* Fix tests
* Make all tests compile.
* Fix test.
* Cleanup + get rid of some warnings.
* state -> requester
* Mostly doc fixes.
* Fix test suite.
* Get rid of now redundant message types.
* WIP
* Rob's review remarks.
* Fix test suite.
* core.relay_parent -> leaf for session request.
* Style fix.
* Decrease request timeout.
* Cleanup obsolete errors.
* Metrics + don't fail on non fatal errors.
* requester.rs -> requester/mod.rs
* Panic on invalid BadValidator report.
* Fix indentation.
* Use typed default timeout constant.
* Make channel size 0, as each sender gets one slot anyways.
* Fix incorrect metrics initialization.
* Fix build after merge.
* More fixes.
* Hopefully valid metrics names.
* Better metrics names.
* Some tests that already work.
* Slightly better docs.
* Some more tests.
* Fix network bridge test.
* add tracing to approval voting
* notify if session info is not working
* add dispute period to chain specs
* propagate genesis session to parachains runtime
* use `on_genesis_session`
* protect against zero cores in computation
* tweak voting rule to be based off of best and add logs
* genesis configuration should use VRF slots only
* swallow more keystore errors
* add some docs
* make validation-worker args non-optional and update clap
* better tracing for bitfield signing and provisioner
* pass amount of bits in bitfields to inclusion instead of recomputing
* debug -> warn for some logs
* better tracing for availability recovery
* a little av-store tracing
* bridge: forward availability recovery messages
* add missing try_from impl
* some more tracing
* improve approval distribution tracing
* guide: hold onto pending approval messages until NewBlocks
* Hold onto pending approval messages until NewBlocks
* guide: adjust comment
* process all actions for one wakeup at a time
* vec
* fix network bridge test
* replace randomness-collective-flip with Babe
* remove PairNotFound
* integrate approval voting into overseer
* expose public API and make keystore arc
* integrate overseer in service
* guide: `ApprovedAncestor` returns block number
* return block number along with hash from ApprovedAncestor
* introduce a voting rule for reporting on approval checking
* integrate the delay voting rule
* Rococo configuration
* fix compilation and add slack
* fix web-wasm build
* tweak parameterization
* migrate voting rules to asycn
* remove hack comment
* skeleton
* skeleton aux-schema module
* start approval types
* start aux schema with aux store
* doc
* finish basic types
* start approval types
* doc
* finish basic types
* write out schema types
* add debug and codec impls to approval types
* add debug and codec impls to approval types
also add some key computation
* add debug and codec impls to approval types
* getters for block and candidate entries
* grumbles
* remove unused AssignmentId
* load_decode utility
* implement DB clearing
* function for adding new block entry to aux store
* start `canonicalize` implementation
* more skeleton
* finish implementing canonicalize
* tag TODO
* implement a test AuxStore
* add allow(unused)
* basic loading and deleting test
* block_entry test function
* add a test for `add_block_entry`
* ensure range is exclusive at end
* test clear()
* test that add_block sets children
* add a test for canonicalize
* extract Pre-digest from header
* utilities for extracting RelayVRFStory from the header-chain
* add approval voting message types
* approval distribution message type
* subsystem skeleton
* state struct
* add futures-timer
* prepare service for babe slot duration
* more skeleton
* better integrate AuxStore
* RelayVRF -> RelayVRFStory
* canonicalize
* implement some tick functionality
* guide: tweaks
* check_approval
* more tweaks and helpers
* guide: add core index to candidate event
* primitives: add core index to candidate event
* runtime: add core index to candidate events
* head handling (session window)
* implement `determine_new_blocks`
* add TODO
* change error type on functions
* compute RelayVRFModulo assignments
* compute RelayVRFDelay assignments
* fix delay tranche calc
* assignment checking
* pluralize
* some dummy code for fetching assignments
* guide: add babe epoch runtime API
* implement a current_epoch() runtime API
* compute assignments
* candidate events get backing group
* import blocks and assignments into DB
* push block approval meta
* add message types, no overseer integration yet
* notify approval distribution of new blocks
* refactor import into separate functions
* impl tranches_to_approve
* guide: improve function signatures
* guide: remove Tick from ApprovalEntry
* trigger and broadcast assignment
* most of approval launching
* remove byteorder crate
* load blocks back to finality, except on startup
* check unchecked assignments
* add claimed core to approval voting message
* fix checks
* assign only to backing group
* remove import_checked_assignment from guide
* newline
* import assignments
* abstract out a bit
* check and import approvals
* check full approvals from assignment import too
* comment
* create a Transaction utility
* must_use
* use transaction in `check_full_approvals`
* wire up wakeups
* add Ord to CandidateHash
* wakeup refactoring
* return candidate info from add_block_entry
* schedule wakeups
* background task: do candidate validation
* forward candidate validation requests
* issue approval votes when requested
* clean up a couple TODOs
* fix up session caching
* clean up last unimplemented!() items
* fix remaining warnings
* remove TODO
* implement handle_approved_ancestor
* update Cargo.lock
* fix runtime API tests
* guide: cleanup assignment checking
* use claimed candidate index instead of core
* extract time to a trait
* tests module
* write a mock clock for testing
* allow swapping out the clock
* make abstract over assignment criteria
* add some skeleton tests and simplify params
* fix backing group check
* do backing group check inside check_assignment_cert
* write some empty test functions to implement
* add a test for non-backing
* test that produced checks pass
* some empty test ideas
* runtime/inclusion: remove outdated TODO
* fix compilation
* av-store: fix tests
* dummy cert
* criteria tests
* move `TestStore` to main tests file
* fix unused warning
* test harness beginnings
* resolve slots renaming fallout
* more compilation fixes
* wip: extract pure data into a separate module
* wip: extract pure data into a separate module
* move types completely to v1
* add persisted_entries
* add conversion trait impls
* clean up some warnings
* extract import logic to own module
* schedule wakeups
* experiment with Actions
* uncomment approval-checking
* separate module for approval checking utilities
* port more code to use actions
* get approval pipeline using actions
* all logic is uncommented
* main loop processes actions
* all loop logic uncommented
* separate function for handling actions
* remove last unimplemented item
* clean up warnings
* State gives read-only access to underlying DB
* tests for approval checking
* tests for approval criteria
* skeleton test module for import
* list of import tests to do
* some test glue code
* test reject bad assignment
* test slot too far in future
* test reject assignment with unknown candidate
* remove loads_blocks tests
* determine_new_blocks back to finalized & harness
* more coverage for determining new blocks
* make `imported_block_info` have less reliance on State
* candidate_info tests
* tests for session caching
* remove println
* extricate DB and main TestStores
* rewrite approval checking logic to counteract early delays
* move state out of function
* update approval-checking tests
* tweak wakeups & scheduling logic
* rename check_full_approvals
* test that assignment import updates candidate
* some approval import tests
* some tests for check_and_apply_approval
* add 'full' qualifier to avoid confusion
* extract should-trigger logic to separate function
* some tests for all triggering
* tests for when we trigger assignments
* test wakeups
* add block utilities for testing
* some more tests for approval updates
* approved_ancestor tests
* new action type for launch approval
* process-wakeup tests
* clean up some warnings
* fix in_future test
* approval checking tests
* tighten up too-far-in-future
* special-case genesis when caching sessions
* fix bitfield len
Co-authored-by: Andronik Ordian <write@reusable.software>
Robert Habermeier