4 Commits

Author	SHA1	Message	Date
Robert Habermeier	e48c687504	Implement Approval Voting Subsystem (#2112) ... * skeleton * skeleton aux-schema module * start approval types * start aux schema with aux store * doc * finish basic types * start approval types * doc * finish basic types * write out schema types * add debug and codec impls to approval types * add debug and codec impls to approval types also add some key computation * add debug and codec impls to approval types * getters for block and candidate entries * grumbles * remove unused AssignmentId * load_decode utility * implement DB clearing * function for adding new block entry to aux store * start `canonicalize` implementation * more skeleton * finish implementing canonicalize * tag TODO * implement a test AuxStore * add allow(unused) * basic loading and deleting test * block_entry test function * add a test for `add_block_entry` * ensure range is exclusive at end * test clear() * test that add_block sets children * add a test for canonicalize * extract Pre-digest from header * utilities for extracting RelayVRFStory from the header-chain * add approval voting message types * approval distribution message type * subsystem skeleton * state struct * add futures-timer * prepare service for babe slot duration * more skeleton * better integrate AuxStore * RelayVRF -> RelayVRFStory * canonicalize * implement some tick functionality * guide: tweaks * check_approval * more tweaks and helpers * guide: add core index to candidate event * primitives: add core index to candidate event * runtime: add core index to candidate events * head handling (session window) * implement `determine_new_blocks` * add TODO * change error type on functions * compute RelayVRFModulo assignments * compute RelayVRFDelay assignments * fix delay tranche calc * assignment checking * pluralize * some dummy code for fetching assignments * guide: add babe epoch runtime API * implement a current_epoch() runtime API * compute assignments * candidate events get backing group * import blocks and assignments into DB * push block approval meta * add message types, no overseer integration yet * notify approval distribution of new blocks * refactor import into separate functions * impl tranches_to_approve * guide: improve function signatures * guide: remove Tick from ApprovalEntry * trigger and broadcast assignment * most of approval launching * remove byteorder crate * load blocks back to finality, except on startup * check unchecked assignments * add claimed core to approval voting message * fix checks * assign only to backing group * remove import_checked_assignment from guide * newline * import assignments * abstract out a bit * check and import approvals * check full approvals from assignment import too * comment * create a Transaction utility * must_use * use transaction in `check_full_approvals` * wire up wakeups * add Ord to CandidateHash * wakeup refactoring * return candidate info from add_block_entry * schedule wakeups * background task: do candidate validation * forward candidate validation requests * issue approval votes when requested * clean up a couple TODOs * fix up session caching * clean up last unimplemented!() items * fix remaining warnings * remove TODO * implement handle_approved_ancestor * update Cargo.lock * fix runtime API tests * guide: cleanup assignment checking * use claimed candidate index instead of core * extract time to a trait * tests module * write a mock clock for testing * allow swapping out the clock * make abstract over assignment criteria * add some skeleton tests and simplify params * fix backing group check * do backing group check inside check_assignment_cert * write some empty test functions to implement * add a test for non-backing * test that produced checks pass * some empty test ideas * runtime/inclusion: remove outdated TODO * fix compilation * av-store: fix tests * dummy cert * criteria tests * move `TestStore` to main tests file * fix unused warning * test harness beginnings * resolve slots renaming fallout * more compilation fixes * wip: extract pure data into a separate module * wip: extract pure data into a separate module * move types completely to v1 * add persisted_entries * add conversion trait impls * clean up some warnings * extract import logic to own module * schedule wakeups * experiment with Actions * uncomment approval-checking * separate module for approval checking utilities * port more code to use actions * get approval pipeline using actions * all logic is uncommented * main loop processes actions * all loop logic uncommented * separate function for handling actions * remove last unimplemented item * clean up warnings * State gives read-only access to underlying DB * tests for approval checking * tests for approval criteria * skeleton test module for import * list of import tests to do * some test glue code * test reject bad assignment * test slot too far in future * test reject assignment with unknown candidate * remove loads_blocks tests * determine_new_blocks back to finalized & harness * more coverage for determining new blocks * make `imported_block_info` have less reliance on State * candidate_info tests * tests for session caching * remove println * extricate DB and main TestStores * rewrite approval checking logic to counteract early delays * move state out of function * update approval-checking tests * tweak wakeups & scheduling logic * rename check_full_approvals * test that assignment import updates candidate * some approval import tests * some tests for check_and_apply_approval * add 'full' qualifier to avoid confusion * extract should-trigger logic to separate function * some tests for all triggering * tests for when we trigger assignments * test wakeups * add block utilities for testing * some more tests for approval updates * approved_ancestor tests * new action type for launch approval * process-wakeup tests * clean up some warnings * fix in_future test * approval checking tests * tighten up too-far-in-future * special-case genesis when caching sessions * fix bitfield len Co-authored-by: Andronik Ordian <write@reusable.software>	2021-02-11 10:21:47 -06:00
Robert Klotzner	0cb1ccd122	Generic request/response infrastructure for Polkadot (#2352) ... * Move NetworkBridgeEvent to subsystem::messages. It is not protocol related at all, it is in fact only part of the subsystem communication as it gets wrapped into messages of each subsystem. * Request/response infrastructure is taking shape. WIP: Does not compile. * Multiplexer variant not supported by Rusts type system. * request_response::request type checks. * Cleanup. * Minor fixes for request_response. * Implement request sending + move multiplexer. Request multiplexer is moved to bridge, as there the implementation is more straight forward as we can specialize on `AllMessages` for the multiplexing target. Sending of requests is mostly complete, apart from a few `From` instances. Receiving is also almost done, initializtion needs to be fixed and the multiplexer needs to be invoked. * Remove obsolete multiplexer. * Initialize bridge with multiplexer. * Finish generic request sending/receiving. Subsystems are now able to receive and send requests and responses via the overseer. * Doc update. * Fixes. * Link issue for not yet implemented code. * Fixes suggested by @ordian - thanks! - start encoding at 0 - don't crash on zero protocols - don't panic on not yet implemented request handling * Update node/network/protocol/src/request_response/v1.rs Use index 0 instead of 1. Co-authored-by: Andronik Ordian <write@reusable.software> * Update node/network/protocol/src/request_response.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Fix existing tests. * Better avoidance of division by zoro errors. * Doc fixes. * send_request -> start_request. * Fix missing renamings. * Update substrate. * Pass TryConnect instead of true. * Actually import `IfDisconnected`. * Fix wrong import. * Update node/network/bridge/src/lib.rs typo Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Update node/network/bridge/src/multiplexer.rs Remove redundant import. Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Stop doing tracing from within `From` instance. Thanks for the catch @tomaka! * Get rid of redundant import. * Formatting cleanup. * Fix tests. * Add link to issue. * Clarify comments some more. * Fix tests. * Formatting fix. * tabs * Fix link Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> * Use map_err. Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> * Improvements inspired by suggestions by @drahnr. - Channel size is now determined by function. - Explicitely scope NetworkService::start_request. Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>	2021-02-03 20:21:09 +00:00
Bastian Köcher	7f019517ca	Companion for #7997 (#2345) ... * Companion for #7997 https://github.com/paritytech/substrate/pull/7997 * rename slot_number to slot * rename SlotNumber type in overseer docs * "Update Substrate" Co-authored-by: André Silva <andrerfosilva@gmail.com> Co-authored-by: parity-processbot <>	2021-01-28 20:04:43 +00:00
Andronik Ordian	3f1e1a6ff7	impl approval distribution (#2160) ... * initial impl approval distribution * initial tests and fixes * batching seems difficult: different peers have different needs * bridge: fix test after merge * some guide updates * only send assignments to peers who know about the block * fix a test, add approvals test * simplify * do not send assignment to peers for finalized blocks * guide: protocol input and output * one more test * more comments, logs, initial metrics * fix a typo * one more thing: early return when reimporting a thing locally	2021-01-25 18:14:32 -05:00