* Don't wait for dispute coordinator
in backing and approval-voting - we are single threaded there, so this
is blocking everything.
* Add missing import.
* Don't warn on dropped receiver.
* remove v0 primitives from polkadot-primitives
* first pass: remove v0
* fix fallout in erasure-coding
* remove v1 primitives, consolidate to v2
* the great import update
* update runtime_api_impl_v1 to v2 as well
* guide: add `Version` request for runtime API
* add version query to runtime API
* reintroduce OldV1SessionInfo in a limited way
* parity db subsystem without cache and no splitted column
* fmt
* fix path (auto from parity-db fail)
* lru cache for db column with cache
* Revert "lru cache for db column with cache"
This reverts commit ae177bc5e107a075eff6a21f651218ada6599b74.
* Write_lock mutex
* theoric code for bridges
* revert changes
* Revert bridge changes
* fix spec_version
* update parity db
* test purge-db
* Use specific ordered collection with paritydb.
* Revert "Use specific ordered collection with paritydb."
This reverts commit 8b66d0a4ae914cba1af0f44050d45dd6d9327c6b.
* fix chain selection tests.
* remove patch
* fix auto.
* Remove useless exists directory method
* purge chain without parity-db removal
* spellcheck
* renamings and filtering.
* fix assertion
* format
* update parity-db and fmt
* Auto keep using rocksdb when it exists.
* Revert "Auto keep using rocksdb when it exists."
This reverts commit cea49b32ae590bdce31fed5c45f3c028ae0c7564.
* Update kvdb version.
* alter currently-checking-set to launch work only on new candidates
* fmt
* fix compilation
* address review
* Introduce approvals cache test that ensures approval work is only triggered once for each Candidate Hash
* Fix formatting
* Address Feedback
* Move final message await into handle function
Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
* remove Default from CandidateHash
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* chore: fmt
* remove backed candidate default
* Partial migration away from CandidateReceipt::default
* Remove more CandidateReceipt defaults
* fmt
* Mostly remove CommittedCandidateReceipt default usage
* Remove CommittedCandidateReceipt
* Remove more Defaults from polakdot primitives v1 + fmt
* Remove more Default from polkadot primites v1
* WIP trying to get overseer example + tests to compile
* feat: add primitives test helpers
* reduce deps of helper
* update primitive helpers
* make candidate validation compile
* fixup cargo lock
* make av-store compile
* fixup disputes coordinator tests
* test: fixup backing
* test: fixup approval voting
* fixup bitfield signing
* test: fixup runtime-api
* test: fixup availability dist
* foxi[ pverseer test]
* remove some Defaults, remove bounds from `dummy`
All `fn dummy` in primitives need to be removed anyways.
This aids in the transition.
* it's a test helper, so always use std
* test: fixup parachains runtime tests
Excluding benches.
* fix keyring
* fix paras runtime properly, no more default
* Remove fn dummy() usage from approval voting
* Move TestCandidateBuilder out of av store to test helpers
* Make candidate validation tests pass
* Make most dispute coirdinator tests pass
* Make provisioner tests work
* Make availability recovery tests work with test helpers
* Update polkadot-collator-protocol tests
* Update statement distribution tests
* Update polkadot overseer examples and tests
* Derive default for validation code so we don't break unrelated things
* Make para runtime test pass (no bench)
* Some more work
* chore: cargo fmt
* cargo fix
* avoid some Default::default
* fixup dispute coordinator test
* remove unused crate deps
* remove Default::default wherever possible, replace by dummy_* for the most part
* chore: cargo fmt
* Remove some warnings
* Remove CommittedCandidateReceipt dummy
* Remove CandidateReceipt dummy
* Remove CandidateDescriptor dummy
* Remove commented out code
* Fix para runtime tests
* chore: nightly
* Some updates to the builder
* Dynamically adjust mock head data size
* Make dispute cooridinator tests work
* Fix test candidate_backing_reorders_votes work
* +nightly-2021-10-29 fmt
* Spelling and remove a default use in builder
* Various clean up
* More small updates
* fmt
* More small updates
* Doc comments for test helpers
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
* Update lib.rs
* review comments
* fix warnings
* fix test by using correct candidate receipt relay parent
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>
* node/approval-voting: Introduce Backend trait and Overlaybackend
This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.
* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight
* node/approval-voting: Add load_all_blocks to overlay
* node/approval-voting: Get all module tests to pass.
This commit modifies all tests to ensure tests are passing.
* node/approval-voting: Address oversights in the previous commit
This commit addresses some oversights in the prior commit.
1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general
* node/approval-voting: Move from TestDB to dyn KeyValueDB
This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.
* node/approval-voting: Address feedback
* node/approval-voting: Add license to ops.rs
* node/approval-voting: Address second-pass feedback
* Add TODO
* node/approval-voting: Bump spec_version
* node/approval-voting: Address final comments.
* node/approval-voting: Introduce framework for full subsystem tests
* node/approval-voting: Introduce basic tests to attempt to provide
coverage via full subsystem tests
* node/approval-voting: Introduce Chainbuilder
* node/approval-voting: Introduce Backend trait and Overlaybackend
This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.
* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight
* node/approval-voting: Add load_all_blocks to overlay
* node/approval-voting: Get all module tests to pass.
This commit modifies all tests to ensure tests are passing.
* node/approval-voting: Address oversights in the previous commit
This commit addresses some oversights in the prior commit.
1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general
* node/approval-voting: Move from TestDB to dyn KeyValueDB
This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.
* node/approval-voting: Address feedback
* node/approval-voting: Add license to ops.rs
* node/approval-voting: Address second-pass feedback
* Add TODO
* node/approval-voting: Bump spec_version
* node/approval-voting: Address final comments.
* DB skeleton
* key formats
* lexicographic test
* custom types for DB
* implement backend for db-v1
* remove VoidBackend and integrate with real DbBackend
* detect stagnant blocks on in interval
* fix tests
* add tests for stagnant
* send ChainSelectionMessage::Approved
* tests for DB backend
* unused import
* upgrade kvdb-memorydb
Co-authored-by: Andronik Ordian <write@reusable.software>
* skeleton for dispute-coordinator
* add coordinator and participation message types
* begin dispute-coordinator DB
* functions for loading
* implement strongly-typed DB transaction
* add some tests for DB transaction
* core logic for pruning
* guide: update candidate-votes key for coordinator
* update candidate-votes key
* use big-endian encoding for session, and implement upper bound generator
* finish implementing pruning
* add a test for note_current_session
* define state of the subsystem itself
* barebones subsystem definition
* control flow
* more control flow
* implement session-updating logic
* trace
* control flow for message handling
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update node/subsystem/src/messages.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* some more control flow
* guide: remove overlay
* more control flow
* implement some DB getters
* make progress on importing statements
* add SignedDisputeStatement struct
* move ApprovalVote to shared primitives
* add a signing-payload API to explicit dispute statements
* add signing-payload to CompactStatement
* add relay-parent hash to seconded/valid dispute variatns
* correct import
* type-safe wrapper around dispute statements
* use checked dispute statement in message type
* extract rolling session window cache to subsystem-util
* extract session window tests
* approval-voting: use rolling session info cache
* reduce dispute window to match runtime in practice
* add byzantine_threshold and supermajority_threshold utilities to primitives
* integrate rolling session window
* Add PartialOrd to CandidateHash
* add Ord to CandidateHash
* implement active dispute update
* add dispute messages to AllMessages
* add dispute stubs to overseer
* inform dispute participation to participate
* implement issue_local_statement
* implement `determine_undisputed_chain`
* fix warnings
* test harness for dispute coordinator tests
* add more helpers to test harness
* add some more helpers
* some tests for dispute coordinator
* ignore wrong validator indices
* test finality voting rule constraint
* add more tests
* add variants to network bridge
* fix test compilation
* remove most dispute coordinator functionality
as of #3222 we can do most of the work within the approval voting subsystem
* Revert "remove most dispute coordinator functionality"
This reverts commit 9cd615e8eb6ca0b382cbaff525d813e753d6004e.
* Use thiserror
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Update node/core/dispute-coordinator/src/lib.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* extract tests to separate module
* address nit
* adjust run_iteration API
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* extract database from av-store itself
* generalize approval-voting over database type
* modes (without handling) and pruning old wakeups
* rework approval importing
* add our_approval_sig to ApprovalEntry
* import assignment
* guide updates for check-full-approval changes
* some aux functions
* send messages when becoming active.
* guide: network bridge sends view updates only when done syncing
* network bridge: send view updates only when done syncing
* tests for new network-bridge behavior
* add a test for updating approval entry with sig
* fix some warnings
* test load-all-blocks
* instantiate new parachains DB
* fix network-bridge empty view updates
* tweak
* fix wasm build, i think
* Update node/core/approval-voting/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* add some versioning to parachains_db
* warnings
* fix merge changes
* remove versioning again
Co-authored-by: Andronik Ordian <write@reusable.software>
* WIP
* availability distribution, still very wip.
Work on the requesting side of things.
* Some docs on what I intend to do.
* Checkpoint of session cache implementation
as I will likely replace it with something smarter.
* More work, mostly on cache
and getting things to type check.
* Only derive MallocSizeOf and Debug for std.
* availability-distribution: Cache feature complete.
* Sketch out logic in `FetchTask` for actual fetching.
- Compile fixes.
- Cleanup.
* Format cleanup.
* More format fixes.
* Almost feature complete `fetch_task`.
Missing:
- Check for cancel
- Actual querying of peer ids.
* Finish FetchTask so far.
* Directly use AuthorityDiscoveryId in protocol and cache.
* Resolve `AuthorityDiscoveryId` on sending requests.
* Rework fetch_task
- also make it impossible to check the wrong chunk index.
- Export needed function in validator_discovery.
* From<u32> implementation for `ValidatorIndex`.
* Fixes and more integration work.
* Make session cache proper lru cache.
* Use proper lru cache.
* Requester finished.
* ProtocolState -> Requester
Also make sure to not fetch our own chunk.
* Cleanup + fixes.
* Remove unused functions
- FetchTask::is_finished
- SessionCache::fetch_session_info
* availability-distribution responding side.
* Cleanup + Fixes.
* More fixes.
* More fixes.
adder-collator is running!
* Some docs.
* Docs.
* Fix reporting of bad guys.
* Fix tests
* Make all tests compile.
* Fix test.
* Cleanup + get rid of some warnings.
* state -> requester
* Mostly doc fixes.
* Fix test suite.
* Get rid of now redundant message types.
* WIP
* Rob's review remarks.
* Fix test suite.
* core.relay_parent -> leaf for session request.
* Style fix.
* Decrease request timeout.
* Cleanup obsolete errors.
* Metrics + don't fail on non fatal errors.
* requester.rs -> requester/mod.rs
* Panic on invalid BadValidator report.
* Fix indentation.
* Use typed default timeout constant.
* Make channel size 0, as each sender gets one slot anyways.
* Fix incorrect metrics initialization.
* Fix build after merge.
* More fixes.
* Hopefully valid metrics names.
* Better metrics names.
* Some tests that already work.
* Slightly better docs.
* Some more tests.
* Fix network bridge test.
* add tracing to approval voting
* notify if session info is not working
* add dispute period to chain specs
* propagate genesis session to parachains runtime
* use `on_genesis_session`
* protect against zero cores in computation
* tweak voting rule to be based off of best and add logs
* genesis configuration should use VRF slots only
* swallow more keystore errors
* add some docs
* make validation-worker args non-optional and update clap
* better tracing for bitfield signing and provisioner
* pass amount of bits in bitfields to inclusion instead of recomputing
* debug -> warn for some logs
* better tracing for availability recovery
* a little av-store tracing
* bridge: forward availability recovery messages
* add missing try_from impl
* some more tracing
* improve approval distribution tracing
* guide: hold onto pending approval messages until NewBlocks
* Hold onto pending approval messages until NewBlocks
* guide: adjust comment
* process all actions for one wakeup at a time
* vec
* fix network bridge test
* replace randomness-collective-flip with Babe
* remove PairNotFound
* integrate approval voting into overseer
* expose public API and make keystore arc
* integrate overseer in service
* guide: `ApprovedAncestor` returns block number
* return block number along with hash from ApprovedAncestor
* introduce a voting rule for reporting on approval checking
* integrate the delay voting rule
* Rococo configuration
* fix compilation and add slack
* fix web-wasm build
* tweak parameterization
* migrate voting rules to asycn
* remove hack comment
* skeleton
* skeleton aux-schema module
* start approval types
* start aux schema with aux store
* doc
* finish basic types
* start approval types
* doc
* finish basic types
* write out schema types
* add debug and codec impls to approval types
* add debug and codec impls to approval types
also add some key computation
* add debug and codec impls to approval types
* getters for block and candidate entries
* grumbles
* remove unused AssignmentId
* load_decode utility
* implement DB clearing
* function for adding new block entry to aux store
* start `canonicalize` implementation
* more skeleton
* finish implementing canonicalize
* tag TODO
* implement a test AuxStore
* add allow(unused)
* basic loading and deleting test
* block_entry test function
* add a test for `add_block_entry`
* ensure range is exclusive at end
* test clear()
* test that add_block sets children
* add a test for canonicalize
* extract Pre-digest from header
* utilities for extracting RelayVRFStory from the header-chain
* add approval voting message types
* approval distribution message type
* subsystem skeleton
* state struct
* add futures-timer
* prepare service for babe slot duration
* more skeleton
* better integrate AuxStore
* RelayVRF -> RelayVRFStory
* canonicalize
* implement some tick functionality
* guide: tweaks
* check_approval
* more tweaks and helpers
* guide: add core index to candidate event
* primitives: add core index to candidate event
* runtime: add core index to candidate events
* head handling (session window)
* implement `determine_new_blocks`
* add TODO
* change error type on functions
* compute RelayVRFModulo assignments
* compute RelayVRFDelay assignments
* fix delay tranche calc
* assignment checking
* pluralize
* some dummy code for fetching assignments
* guide: add babe epoch runtime API
* implement a current_epoch() runtime API
* compute assignments
* candidate events get backing group
* import blocks and assignments into DB
* push block approval meta
* add message types, no overseer integration yet
* notify approval distribution of new blocks
* refactor import into separate functions
* impl tranches_to_approve
* guide: improve function signatures
* guide: remove Tick from ApprovalEntry
* trigger and broadcast assignment
* most of approval launching
* remove byteorder crate
* load blocks back to finality, except on startup
* check unchecked assignments
* add claimed core to approval voting message
* fix checks
* assign only to backing group
* remove import_checked_assignment from guide
* newline
* import assignments
* abstract out a bit
* check and import approvals
* check full approvals from assignment import too
* comment
* create a Transaction utility
* must_use
* use transaction in `check_full_approvals`
* wire up wakeups
* add Ord to CandidateHash
* wakeup refactoring
* return candidate info from add_block_entry
* schedule wakeups
* background task: do candidate validation
* forward candidate validation requests
* issue approval votes when requested
* clean up a couple TODOs
* fix up session caching
* clean up last unimplemented!() items
* fix remaining warnings
* remove TODO
* implement handle_approved_ancestor
* update Cargo.lock
* fix runtime API tests
* guide: cleanup assignment checking
* use claimed candidate index instead of core
* extract time to a trait
* tests module
* write a mock clock for testing
* allow swapping out the clock
* make abstract over assignment criteria
* add some skeleton tests and simplify params
* fix backing group check
* do backing group check inside check_assignment_cert
* write some empty test functions to implement
* add a test for non-backing
* test that produced checks pass
* some empty test ideas
* runtime/inclusion: remove outdated TODO
* fix compilation
* av-store: fix tests
* dummy cert
* criteria tests
* move `TestStore` to main tests file
* fix unused warning
* test harness beginnings
* resolve slots renaming fallout
* more compilation fixes
* wip: extract pure data into a separate module
* wip: extract pure data into a separate module
* move types completely to v1
* add persisted_entries
* add conversion trait impls
* clean up some warnings
* extract import logic to own module
* schedule wakeups
* experiment with Actions
* uncomment approval-checking
* separate module for approval checking utilities
* port more code to use actions
* get approval pipeline using actions
* all logic is uncommented
* main loop processes actions
* all loop logic uncommented
* separate function for handling actions
* remove last unimplemented item
* clean up warnings
* State gives read-only access to underlying DB
* tests for approval checking
* tests for approval criteria
* skeleton test module for import
* list of import tests to do
* some test glue code
* test reject bad assignment
* test slot too far in future
* test reject assignment with unknown candidate
* remove loads_blocks tests
* determine_new_blocks back to finalized & harness
* more coverage for determining new blocks
* make `imported_block_info` have less reliance on State
* candidate_info tests
* tests for session caching
* remove println
* extricate DB and main TestStores
* rewrite approval checking logic to counteract early delays
* move state out of function
* update approval-checking tests
* tweak wakeups & scheduling logic
* rename check_full_approvals
* test that assignment import updates candidate
* some approval import tests
* some tests for check_and_apply_approval
* add 'full' qualifier to avoid confusion
* extract should-trigger logic to separate function
* some tests for all triggering
* tests for when we trigger assignments
* test wakeups
* add block utilities for testing
* some more tests for approval updates
* approved_ancestor tests
* new action type for launch approval
* process-wakeup tests
* clean up some warnings
* fix in_future test
* approval checking tests
* tighten up too-far-in-future
* special-case genesis when caching sessions
* fix bitfield len
Co-authored-by: Andronik Ordian <write@reusable.software>
Robert Klotzner