mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-14 00:31:07 +00:00
Asynchronous Backing MegaPR (#5022)
* inclusion emulator logic for asynchronous backing (#4790) * initial stab at candidate_context * fmt * docs & more TODOs * some cleanups * reframe as inclusion_emulator * documentations yes * update types * add constraint modifications * watermark * produce modifications * v2 primitives: re-export all v1 for consistency * vstaging primitives * emulator constraints: handle code upgrades * produce outbound HRMP modifications * stack. * method for applying modifications * method just for sanity-checking modifications * fragments produce modifications, not prospectives * make linear * add some TODOs * remove stacking; handle code upgrades * take `fragment` private * reintroduce stacking. * fragment constructor * add TODO * allow validating fragments against future constraints * docs * relay-parent number and min code size checks * check code upgrade restriction * check max hrmp per candidate * fmt * remove GoAhead logic because it wasn't helpful * docs on code upgrade failure * test stacking * test modifications against constraints * fmt * test fragments * descending or duplicate test * fmt * remove unused imports in vstaging * wrong primitives * spellcheck * Runtime changes for Asynchronous Backing (#4786) * inclusion: utility for allowed relay-parents * inclusion: use prev number instead of prev hash * track most recent context of paras * inclusion: accept previous relay-parents * update dmp advancement rule for async backing * fmt * add a comment about validation outputs * clean up a couple of TODOs * weights * fix weights * fmt * Resolve dmp todo * Restore inclusion tests * Restore paras_inherent tests * MostRecentContext test * Benchmark for new paras dispatchable * Prepare check_validation_outputs for upgrade * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs * Implementers guide changes * More tests for allowed relay parents * Add a github issue link * Compute group index based on relay parent * Storage migration * Move allowed parents tracker to shared * Compile error * Get group assigned to core at the next block * Test group assignment * fmt * Error instead of panic * Update guide * Extend doc-comment * Update runtime/parachains/src/shared.rs Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Prospective Parachains Subsystem (#4913) * docs and skeleton * subsystem skeleton * main loop * fragment tree basics & fmt * begin fragment trees & view * flesh out more of view update logic * further flesh out update logic * some refcount functions for fragment trees * add fatal/non-fatal errors * use non-fatal results * clear up some TODOs * ideal format for scheduling info * add a bunch of TODOs * some more fluff * extract fragment graph to submodule * begin fragment graph API * trees, not graphs * improve docs * scope and constructor for trees * add some test TODOs * limit max ancestors and store constraints * constructor * constraints: fix bug in HRMP watermarks * fragment tree population logic * set::retain * extract population logic * implement add_and_populate * fmt * add some TODOs in tests * implement child-selection * strip out old stuff based on wrong assumptions * use fatality * implement pruning * remove unused ancestor constraints * fragment tree instantiation * remove outdated comment * add message/request types and skeleton for handling * fmt * implement handle_candidate_seconded * candidate storage: handle backed * implement handle_candidate_backed * implement answer_get_backable_candidate * remove async where not needed * implement fetch_ancestry * add logic for run_iteration * add some docs * remove global allow(unused), fix warnings * make spellcheck happy (despite English) * fmt * bump Cargo.lock * replace tracing with gum * introduce PopulateFrom trait * implement GetHypotheticalDepths * revise docs slightly * first fragment tree scope test * more scope tests * test add_candidate * fmt * test retain * refactor test code * test populate is recursive * test contiguity of depth 0 is maintained * add_and_populate tests * cycle tests * remove PopulateFrom trait * fmt * test hypothetical depths (non-recursive) * have CandidateSeconded return membership * tree membership requests * Add a ProspectiveParachainsSubsystem struct * add a staging API for base constraints * add a `From` impl * add runtime API for staging_validity_constraints * implement fetch_base_constraints * implement `fetch_upcoming_paras` * remove reconstruction of candidate receipt; no obvious usecase * fmt * export message to broader module * remove last TODO * correctly export * fix compilation and add GetMinimumRelayParent request * make provisioner into a real subsystem with proper mesage bounds * fmt * fix ChannelsOut in overseer test * fix overseer tests * fix again * fmt * Integrate prospective parachains subsystem into backing: Part 1 (#5557) * BEGIN ASYNC candidate-backing CHANGES * rename & document modes * answer prospective validation data requests * GetMinimumRelayParents request is now plural * implement an implicit view utility for backing subsystems * implicit-view: get allowed relay parents * refactorings and improvements to implicit view * add some TODOs for tests * split implicit view updates into 2 functions * backing: define State to prepare for functional refactor * add some docs * backing: implement bones of new leaf activation logic * backing: create per-relay-parent-states * use new handle_active_leaves_update * begin extracting logic from CandidateBackingJob * mostly extract statement import from job logic * handle statement imports outside of job logic * do some TODO planning for prospective parachains integration * finish rewriting backing subsystem in functional style * add prospective parachains mode to relay parent entries * fmt * add a RejectedByProspectiveParachains error * notify prospective parachains of seconded and backed candidates * always validate candidates exhaustively in backing. * return persisted_validation_data from validation * handle rejections by prospective parachains * implement seconding sanity check * invoke validate_and_second * Alter statement table to allow multiple seconded messages per validator * refactor backing to have statements carry PVD * clean up all warnings * Add tests for implicit view * Improve doc comments * Prospective parachains mode based on Runtime API version * Add a TODO * Rework seconding_sanity_check * Iterate over responses * Update backing tests * collator-protocol: load PVD from runtime * Fix validator side tests * Update statement-distribution to fetch PVD * Fix statement-distribution tests * Backing tests with prospective paras #1 * fix per_relay_parent pruning in backing * Test multiple leaves * Test seconding sanity check * Import statement order Before creating an entry in `PerCandidateState` map wait for the approval from the prospective parachains * Add a test for correct state updates * Second multiple candidates per relay parent test * Add backing tests with prospective paras * Second more than one test without prospective paras * Add a test for prospective para blocks * Update malus * typos Co-authored-by: Chris Sosnin <chris125_@live.com> * Track occupied depth in backing per parachain (#5778) * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * fmt * Network bridge changes for asynchronous backing + update subsystems to handle versioned packets (#5991) * BEGIN STATEMENT DISTRIBUTION WORK create a vstaging network protocol which is the same as v1 * mostly make network bridge amenable to vstaging * network-bridge: fully adapt to vstaging * add some TODOs for tests * fix fallout in bitfield-distribution * bitfield distribution tests + TODOs * fix fallout in gossip-support * collator-protocol: fix message fallout * collator-protocol: load PVD from runtime * add TODO for vstaging tests * make things compile * set used network protocol version using a feature * fmt * get approval-distribution building * fix approval-distribution tests * spellcheck * nits * approval distribution net protocol test * bitfield distribution net protocol test * Revert "collator-protocol: fix message fallout" This reverts commit 07cc887303e16c6b3843ecb25cdc7cc2080e2ed1. * Network bridge tests Co-authored-by: Chris Sosnin <chris125_@live.com> * remove max_pov_size requirement from prospective pvd request (#6014) * remove max_pov_size requirement from prospective pvd request * fmt * Extract legacy statement distribution to its own module (#6026) * add compatibility type to v2 statement distribution message * warning cleanup * handle compatibility layer for v2 * clean up an unimplemented!() block * circulate statements based on version * extract legacy v1 code into separate module * remove unimplemented * clean up naming of from_requester/responder * remove TODOs * have backing share seconded statements with PVD * fmt * fix warning * Quick fix unused warning for not yet implemented/used staging messages. * Fix network bridge test * Fix wrong merge. We now have 23 subsystems (network bridge split + prospective parachains) Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at> * Version 3 is already live. * Fix tests (#6055) * Fix backing tests * Fix warnings. * fmt * collator-protocol: asynchronous backing changes (#5740) * Draft collator side changes * Start working on collations management * Handle peer's view change * Versioning on advertising * Versioned collation fetching request * Handle versioned messages * Improve docs for collation requests * Add spans * Add request receiver to overseer * Fix collator side tests * Extract relay parent mode to lib * Validator side draft * Add more checks for advertisement * Request pvd based on async backing mode * review * Validator side improvements * Make old tests green * More fixes * Collator side tests draft * Send collation test * fmt * Collator side network protocol versioning * cleanup * merge artifacts * Validator side net protocol versioning * Remove fragment tree membership request * Resolve todo * Collator side core state test * Improve net protocol compatibility * Validator side tests * more improvements * style fixes * downgrade log * Track implicit assignments * Limit the number of seconded candidates per para * Add a sanity check * Handle fetched candidate * fix tests * Retry fetch * Guard against dequeueing while already fetching * Reintegrate connection management * Timeout on advertisements * fmt * spellcheck * update tests after merge * validator assignment fixes for backing and collator protocol (#6158) * Rename depth->ancestry len in tests * Refactor group assignments * Remove implicit assignments * backing: consider occupied core assignments * Track a single para on validator side * Refactor prospective parachains mode request (#6179) * Extract prospective parachains mode into util * Skip activations depending on the mode * backing: don't send backed candidate to provisioner (#6185) * backing: introduce `CanSecond` request for advertisements filtering (#6225) * Drop BoundToRelayParent * draft changes * fix backing tests * Fix genesis ancestry * Fix validator side tests * more tests * cargo generate-lockfile * Implement `StagingValidityConstraints` Runtime API method (#6258) * Implement StagingValidityConstraints * spellcheck * fix ump params * Update hrmp comment * Introduce ump per candidate limit * hypothetical earliest block * refactor primitives usage * hypothetical earliest block number test * fix build * Prepare the Runtime for asynchronous backing upgrade (#6287) * Introduce async backing params to runtime config * fix cumulus config * use config * finish runtimes * Introduce new staging API * Update collator protocol * Update provisioner * Update prospective parachains * Update backing * Move async backing params lower in the config * make naming consistent * misc * Use real prospective parachains subsystem (#6407) * Backport `HypotheticalFrontier` into the feature branch (#6605) * implement more general HypotheticalFrontier * fmt * drop unneeded request Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Resolve todo about legacy leaf activation (#6447) * fix bug/warning in handling membership answers * Remove `HypotheticalDepthRequest` in favor of `HypotheticalFrontierRequest` (#6521) * Remove `HypotheticalDepthRequest` for `HypotheticalFrontierRequest` * Update tests * Fix (removed wrong docstring) * Fix can_second request * Patch some dead_code errors --------- Co-authored-by: Chris Sosnin <chris125_@live.com> * Async Backing: Send Statement Distribution "Backed" messages (#6634) * Backing: Send Statement Distribution "Backed" messages Closes #6590. **TODO:** - [ ] Adjust tests * Fix compile errors * (Mostly) fix tests * Fix comment * Fix test and compile error * Test that `StatementDistributionMessage::Backed` is sent * Fix compile error * Fix some clippy errors * Add prospective parachains subsystem tests (#6454) * Add prospective parachains subsystem test * Add `should_do_no_work_if_async_backing_disabled_for_leaf` test * Implement `activate_leaf` helper, up to getting ancestry * Finish implementing `activate_leaf` * Small refactor in `activate_leaf` * Get `CandidateSeconded` working * Finish `send_candidate_and_check_if_found` test * Refactor; send more leaves & candidates * Refactor test * Implement `check_candidate_parent_leaving_view` test * Start work on `check_candidate_on_multiple_forks` test * Don’t associate specific parachains with leaf * Finish `correctly_updates_leaves` test * Fix cycle due to reused head data * Fix `check_backable_query` test * Fix `check_candidate_on_multiple_forks` test * Add `check_depth_and_pvd_queries` test * Address review comments * Remove TODO * add a new index for output head data to candidate storage * Resolve test TODOs * Fix compile errors * test candidate storage pruning, make sure new index is cleaned up --------- Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Node-side metrics for asynchronous backing (#6549) * Add metrics for `prune_view_candidate_storage` * Add metrics for `request_unblocked_collations` * Fix docstring * Couple fixes from review comments * Fix `check_depth_query` test * inclusion-emulator: mirror advancement rule check (#6361) * inclusion-emulator: mirror advancement rule check * fix build * prospective-parachains: introduce `backed_in_path_only` flag for advertisements (#6649) * Introduce `backed_in_path_only` flag for depth request * fmt * update doc comment * fmt * Add async-backing zombienet tests (#6314) * Async backing: impl guide for statement distribution (#6738) Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com> * Asynchronous backing statement distribution: Take III (#5999) * add notification types for v2 statement-distribution * improve protocol docs * add empty vstaging module * fmt * add backed candidate packet request types * start putting down structure of new logic * handle activated leaf * some sanity-checking on outbound statements * fmt * update vstaging share to use statements with PVD * tiny refactor, candidate_hash location * import local statements * refactor statement import * first stab at broadcast logic * fmt * fill out some TODOs * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Mark receiver with `vstaging` * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * fmt, fix grid test after topology change * typo Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * address review * adjust comment, make easier to understand * Fix typo --------- Co-authored-by: Marcin S <marcin@bytedude.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Chris Sosnin <chris125_@live.com> * miscellaneous fixes to make asynchronous backing work (#6791) * propagate network-protocol-staging feature * add feature to adder-collator as well * allow collation-generation of occupied cores * prospective parachains: special treatment for pending availability candidates * runtime: fetch candidates pending availability * lazily construct PVD for pending candidates * fix fallout in prospective parachains hypothetical/select_child * runtime: enact candidates when creating paras-inherent * make tests compile * test pending availability in the scope * add prospective parachains test * fix validity constraints leftovers * drop prints * Fix typos --------- Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Marcin S <marcin@realemail.net> * Remove restart from test (#6840) * Async Backing: Statement Distribution Tests (#6755) * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * Hook up request sender * Add `valid_statement_without_prior_seconded_is_ignored` test * Fix `valid_statement_without_prior_seconded_is_ignored` test * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Remove obsolete test * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Mark receiver with `vstaging` * First draft of `ensure_seconding_limit_is_respected` test * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * Fix `ensure_seconding_limit_is_respected` test * Start `backed_candidate_leads_to_advertisement` test * fmt, fix grid test after topology change * Send Backed notification * Finish `backed_candidate_leads_to_advertisement` test * Finish `peer_reported_for_duplicate_statements` test * Finish `received_advertisement_before_confirmation_leads_to_request` * Add `advertisements_rejected_from_incorrect_peers` test * Add `manifest_rejected_*` tests * Add `manifest_rejected_when_group_does_not_match_para` test * Add `local_node_sanity_checks_incoming_requests` test * Add `local_node_respects_statement_mask` test * Add tests where peer is reported for providing invalid signatures * Add `cluster_peer_allowed_to_send_incomplete_statements` test * Add `received_advertisement_after_backing_leads_to_acknowledgement` * Add `received_advertisement_after_confirmation_before_backing` test * peer_reported_for_advertisement_conflicting_with_confirmed_candidate * Add `peer_reported_for_not_enough_statements` test * Add `peer_reported_for_providing_statements_meant_to_be_masked_out` * Add `additional_statements_are_shared_after_manifest_exchange` * Add `grid_statements_imported_to_backing` test * Add `relay_parent_entering_peer_view_leads_to_advertisement` test * Add `advertisement_not_re_sent_when_peer_re_enters_view` test * Update node/network/statement-distribution/src/vstaging/tests/grid.rs Co-authored-by: asynchronous rob <rphmeier@gmail.com> * Resolve TODOs, update test * Address unused code * Add check after every test for unhandled requests * Refactor (`make_dummy_leaf` and `handle_sent_request`) * Refactor (`make_dummy_topology`) * Minor refactor --------- Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Chris Sosnin <chris125_@live.com> * Fix some clippy lints in tests * Async backing: minor fixes (#6920) * bitfield-distribution test * implicit view tests * Refactor parameters -> params * scheduler: update storage migration (#6963) * update scheduler migration * Adjust weight to account for storage read * Statement Distribution Guide Edits (#7025) * Statement distribution guide edits * Addressed Marcin's comments * Add attested candidate request retry timeouts (#6833) Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <chris125_@live.com> Fix async backing statement distribution tests (#6621) Resolve some todos in async backing statement-distribution branch (#6482) Fix clippy errors in statement distribution branch (#6720) * Async backing: add Prospective Parachains impl guide (#6933) Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> * Updates to Provisioner Guide for Async Backing (#7106) * Initial corrections and clarifications * Partial first draft * Finished first draft * Adding back wrongly removed test bit * fmt * Update roadmap/implementers-guide/src/node/utility/provisioner.md Co-authored-by: Marcin S. <marcin@realemail.net> * Addressing comments * Reorganization * fmt --------- Co-authored-by: Marcin S. <marcin@realemail.net> * fmt * Renaming Parathread Mentions (#7287) * Renaming parathreads * Renaming module to pallet * More updates * PVF: Refactor workers into separate crates, remove host dependency (#7253) * PVF: Refactor workers into separate crates, remove host dependency * Fix compile error * Remove some leftover code * Fix compile errors * Update Cargo.lock * Remove worker main.rs files I accidentally copied these from the other PR. This PR isn't intended to introduce standalone workers yet. * Address review comments * cargo fmt * Update a couple of comments * Update log targets * Update quote to 1.0.27 (#7280) Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: parity-processbot <> * pallets: implement `Default` for `GenesisConfig` in `no_std` (#7271) * pallets: implement Default for GenesisConfig in no_std This change is follow-up of: https://github.com/paritytech/substrate/pull/14108 It is a step towards: https://github.com/paritytech/substrate/issues/13334 * Cargo.lock updated * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * cli: enable BEEFY by default on test networks (#7293) We consider BEEFY mature enough to run by default on all nodes for test networks (Rococo/Wococo/Versi). Right now, most nodes are not running it since it's opt-in using --beefy flag. Switch to an opt-out model for test networks. Replace --beefy flag from CLI with --no-beefy and have BEEFY client start by default on test networks. Signed-off-by: acatangiu <adrian@parity.io> * runtime: past session slashing runtime API (#6667) * runtime/vstaging: unapplied_slashes runtime API * runtime/vstaging: key_ownership_proof runtime API * runtime/ParachainHost: submit_report_dispute_lost * fix key_ownership_proof API * runtime: submit_report_dispute_lost runtime API * nits * Update node/subsystem-types/src/messages.rs Co-authored-by: Marcin S. <marcin@bytedude.com> * revert unrelated fmt changes * post merge fixes * fix compilation --------- Co-authored-by: Marcin S. <marcin@bytedude.com> * Correcting git mishap * Document usage of `gum` crate (#7294) * Document usage of gum crate * Small fix * Add some more basic info * Update node/gum/src/lib.rs Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * Update target docs --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * XCM: Fix issue with RequestUnlock (#7278) * XCM: Fix issue with RequestUnlock * Leave API changes for v4 * Fix clippy errors * Fix tests --------- Co-authored-by: parity-processbot <> * Companion for Substrate#14228 (#7295) * Companion for Substrate#14228 https://github.com/paritytech/substrate/pull/14228 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * Companion for #14237: Use latest sp-crates (#7300) * To revert: Update substrate branch to "lexnv/bump_sp_crates" Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert "To revert: Update substrate branch to "lexnv/bump_sp_crates"" This reverts commit 5f1db84eac4a226c37b7f6ce6ee19b49dc7e2008. * Update cargo lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * bounded-collections bump to 0.1.7 (#7305) * bounded-collections bump to 0.1.7 Companion for: paritytech/substrate#14225 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * bump to quote 1.0.28 (#7306) * `RollingSessionWindow` cleanup (#7204) * Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit * Fix tests in import * Fix the rest of the tests * Remove dead code * Fix todos * Simplify session caching * Comments for `SessionInfoProvider` * Separate `SessionInfoProvider` from `State` * `cache_session_info_for_head` becomes freestanding function * Remove unneeded `mut` usage * fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider` * Fix SessionInfo retrieval * Code cleanup * Don't wrap `SessionInfoProvider` in an `Option` * Remove `earliest_session()` * Remove pre-caching -> wip * Fix some tests and code cleanup * Fix all tests * Fixes in tests * Fix comments, variable names and small style changes * Fix a warning * impl From<SessionWindowSize> for NonZeroUsize * Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG * Code review feedback * Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db * Remove `col_session_data` usages * Storage migration clearing columns w/o removing them * Remove session data column usages from `approval-voting` and `dispute-coordinator` tests * Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests * Fix formatting in initialized.rs * Fix a corner case in `SessionInfo` caching for `dispute-coordinator` * Remove `RollingSessionWindow` ;( * Revert "Fix formatting in initialized.rs" This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc. * v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it * Fix `NUM_COLUMNS` in `approval-voting` * Use `columns::v3::NUM_COLUMNS` when opening db * Update node/service/src/parachains_db/upgrade.rs Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3` * Fix `NUM+COLUMNS` in approval_voting * Fix formatting * Fix columns usage * Clarification comments about the different db versions --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * pallet-para-config: Remove remnant WeightInfo functions (#7308) * pallet-para-config: Remove remnant WeightInfo functions Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * set_config_with_weight begone Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::configuration --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: command-bot <> * XCM: PayOverXcm config (#6900) * Move XCM query functionality to trait * Fix tests * Add PayOverXcm implementation * fix the PayOverXcm trait to compile * moved doc comment out of trait implmeentation and to the trait * PayOverXCM documentation * Change documentation a bit * Added empty benchmark methods implementation and changed docs * update PayOverXCM to convert AccountIds to MultiLocations * Implement benchmarking method * Change v3 to latest * Descend origin to an asset sender (#6970) * descend origin to an asset sender * sender as tuple of dest and sender * Add more variants to the QueryResponseStatus enum * Change Beneficiary to Into<[u8; 32]> * update PayOverXcm to return concrete errors and use AccountId as sender * use polkadot-primitives for AccountId * fix dependency to use polkadot-core-primitives * force Unpaid instruction to the top of the instructions list * modify report_outcome to accept interior argument * use new_query directly for building final xcm query, instead of report_outcome * fix usage of new_query to use the XcmQueryHandler * fix usage of new_query to use the XcmQueryHandler * tiny method calling fix * xcm query handler (#7198) * drop redundant query status * rename ReportQueryStatus to OuterQueryStatus * revert rename of QueryResponseStatus * update mapping * Update xcm/xcm-builder/src/pay.rs Co-authored-by: Gavin Wood <gavin@parity.io> * Updates * Docs * Fix benchmarking stuff * Destination can be determined based on asset_kind * Tweaking API to minimise clones * Some repotting and docs --------- Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> * Companion for #14265 (#7307) * Update Cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update Cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: parity-processbot <> * bump serde to 1.0.163 (#7315) * bump serde to 1.0.163 * bump ci * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * fmt * Updated fmt * Removing changes accidentally pulled from master * fix another master pull issue * Another master pull fix * fmt * Fixing implementers guide build * Revert "Merge branch 'rh-async-backing-feature-while-frozen' of https://github.com/paritytech/polkadot into brad-rename-parathread" This reverts commit bebc24af52ab61155e3fe02cb3ce66a592bce49c, reversing changes made to 1b2de662dfb11173679d6da5bd0da9d149c85547. --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: acatangiu <adrian@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Marcin S. <marcin@bytedude.com> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> * fix bitfield distribution test * approval distribution tests * fix bridge tests * update Cargo.lock * [async-backing-branch] Optimize collator-protocol validator-side request fetching (#7457) * Optimize collator-protocol validator-side request fetching * address feedback: replace tuples with structs * feedback: add doc comments * move collation types to subfolder --------- Signed-off-by: alindima <alin@parity.io> * Update collation generation for asynchronous backing (#7405) * break candidate receipt construction and distribution into own function * update implementers' guide to include SubmitCollation * implement SubmitCollation for collation-generation * fmt * fix test compilation & remove unnecessary submodule * add some TODOs for a test suite. * Update roadmap/implementers-guide/src/types/overseer-protocol.md Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * add new test harness and first test * refactor to avoid requiring background sender * ensure collation gets packaged and distributed * tests for the fallback case with no hint * add parent rp-number hint tests * fmt * update uses of CollationGenerationConfig * fix remaining test * address review comments * use subsystemsender for background tasks * fmt * remove ValidationCodeHashHint and related tests --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * fix some more fallout from merge * fmt * remove staging APIs from Rococo & Westend (#7513) * send network messages on main protocol name (#7515) * misc async backing improvements for allowed ancestry blocks (#7532) * shared: fix acquire_info * backwards-compat test for prospective parachains * same relay parent is allowed * provisioner: request candidate receipt by relay parent (#7527) * return candidates hash from prospective parachains * update provisioner * update tests * guide changes * send a single message to backing * fix test * revert to old `handle_new_activations` logic in some cases (#7514) * revert to old `handle_new_activations` logic * gate sending messages on scheduled cores to max_depth >= 2 * fmt * 2->1 * Omnibus asynchronous backing bugfix PR (#7529) * fix a bug in backing * add some more logs * prospective parachains: take ancestry only up to session bounds * add test * fix zombienet tests (#7614) Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix runtime compilation * make bitfield distribution tests compile * attempt to fix zombienet disputes (#7618) * update metric name * update some metric names * avoid cycles when creating fake candidates * make undying collator more friendly to malformed parents * fix a bug in malus * fmt * clippy * add RUN_IN_CONTAINER to new ZombieNet tests (#7631) * remove duplicated migration happened because of master-merge --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: acatangiu <adrian@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Signed-off-by: alindima <alin@parity.io> Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at> Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com> Co-authored-by: Marcin S <marcin@bytedude.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Mattia L.V. Bradascio <28816406+bredamatt@users.noreply.github.com> Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com> Co-authored-by: BradleyOlson64 <lotrftw9@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Alin Dima <alin@parity.io>
This commit is contained in:
asynchronous rob
@@ -25,8 +25,9 @@ use polkadot_node_jaeger as jaeger;
|
||||
use polkadot_node_network_protocol::{
|
||||
self as net_protocol,
|
||||
grid_topology::{RandomRouting, RequiredRouting, SessionGridTopologies, SessionGridTopology},
|
||||
peer_set::MAX_NOTIFICATION_SIZE,
|
||||
v1 as protocol_v1, PeerId, UnifiedReputationChange as Rep, Versioned, View,
|
||||
peer_set::{ValidationVersion, MAX_NOTIFICATION_SIZE},
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, PeerId, UnifiedReputationChange as Rep,
|
||||
Versioned, VersionedValidationProtocol, View,
|
||||
};
|
||||
use polkadot_node_primitives::approval::{
|
||||
AssignmentCert, BlockApprovalMeta, IndirectAssignmentCert, IndirectSignedApprovalVote,
|
||||
@@ -159,6 +160,15 @@ enum Resend {
|
||||
No,
|
||||
}
|
||||
|
||||
/// Data stored on a per-peer basis.
|
||||
#[derive(Debug)]
|
||||
struct PeerData {
|
||||
/// The peer's view.
|
||||
view: View,
|
||||
/// The peer's protocol version.
|
||||
version: ValidationVersion,
|
||||
}
|
||||
|
||||
/// The [`State`] struct is responsible for tracking the overall state of the subsystem.
|
||||
///
|
||||
/// It tracks metadata about our view of the unfinalized chain,
|
||||
@@ -179,7 +189,7 @@ struct State {
|
||||
pending_known: HashMap<Hash, Vec<(PeerId, PendingMessage)>>,
|
||||
|
||||
/// Peer data is partially stored here, and partially inline within the [`BlockEntry`]s
|
||||
peer_views: HashMap<PeerId, View>,
|
||||
peer_data: HashMap<PeerId, PeerData>,
|
||||
|
||||
/// Keeps a topology for various different sessions.
|
||||
topologies: SessionGridTopologies,
|
||||
@@ -349,14 +359,30 @@ impl State {
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
match event {
|
||||
NetworkBridgeEvent::PeerConnected(peer_id, role, _, _) => {
|
||||
NetworkBridgeEvent::PeerConnected(peer_id, role, version, _) => {
|
||||
// insert a blank view if none already present
|
||||
gum::trace!(target: LOG_TARGET, ?peer_id, ?role, "Peer connected");
|
||||
self.peer_views.entry(peer_id).or_default();
|
||||
let version = match ValidationVersion::try_from(version).ok() {
|
||||
Some(v) => v,
|
||||
None => {
|
||||
// sanity: network bridge is supposed to detect this already.
|
||||
gum::error!(
|
||||
target: LOG_TARGET,
|
||||
?peer_id,
|
||||
?version,
|
||||
"Unsupported protocol version"
|
||||
);
|
||||
return
|
||||
},
|
||||
};
|
||||
|
||||
self.peer_data
|
||||
.entry(peer_id)
|
||||
.or_insert_with(|| PeerData { version, view: Default::default() });
|
||||
},
|
||||
NetworkBridgeEvent::PeerDisconnected(peer_id) => {
|
||||
gum::trace!(target: LOG_TARGET, ?peer_id, "Peer disconnected");
|
||||
self.peer_views.remove(&peer_id);
|
||||
self.peer_data.remove(&peer_id);
|
||||
self.blocks.iter_mut().for_each(|(_hash, entry)| {
|
||||
entry.known_by.remove(&peer_id);
|
||||
})
|
||||
@@ -393,12 +419,12 @@ impl State {
|
||||
live
|
||||
});
|
||||
},
|
||||
NetworkBridgeEvent::PeerMessage(peer_id, msg) => {
|
||||
self.process_incoming_peer_message(ctx, metrics, peer_id, msg, rng).await;
|
||||
},
|
||||
NetworkBridgeEvent::UpdatedAuthorityIds { .. } => {
|
||||
// The approval-distribution subsystem doesn't deal with `AuthorityDiscoveryId`s.
|
||||
},
|
||||
NetworkBridgeEvent::PeerMessage(peer_id, Versioned::V1(msg)) => {
|
||||
self.process_incoming_peer_message(ctx, metrics, peer_id, msg, rng).await;
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -455,16 +481,18 @@ impl State {
|
||||
|
||||
{
|
||||
let sender = ctx.sender();
|
||||
for (peer_id, view) in self.peer_views.iter() {
|
||||
let intersection = view.iter().filter(|h| new_hashes.contains(h));
|
||||
let view_intersection = View::new(intersection.cloned(), view.finalized_number);
|
||||
for (peer_id, data) in self.peer_data.iter() {
|
||||
let intersection = data.view.iter().filter(|h| new_hashes.contains(h));
|
||||
let view_intersection =
|
||||
View::new(intersection.cloned(), data.view.finalized_number);
|
||||
Self::unify_with_peer(
|
||||
sender,
|
||||
metrics,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
self.peer_views.len(),
|
||||
self.peer_data.len(),
|
||||
*peer_id,
|
||||
data.version,
|
||||
view_intersection,
|
||||
rng,
|
||||
)
|
||||
@@ -547,6 +575,7 @@ impl State {
|
||||
|
||||
adjust_required_routing_and_propagate(
|
||||
ctx,
|
||||
&self.peer_data,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
|block_entry| block_entry.session == session,
|
||||
@@ -566,13 +595,16 @@ impl State {
|
||||
ctx: &mut Context,
|
||||
metrics: &Metrics,
|
||||
peer_id: PeerId,
|
||||
msg: protocol_v1::ApprovalDistributionMessage,
|
||||
msg: net_protocol::ApprovalDistributionMessage,
|
||||
rng: &mut R,
|
||||
) where
|
||||
R: CryptoRng + Rng,
|
||||
{
|
||||
match msg {
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments) => {
|
||||
Versioned::V1(protocol_v1::ApprovalDistributionMessage::Assignments(assignments)) |
|
||||
Versioned::VStaging(protocol_vstaging::ApprovalDistributionMessage::Assignments(
|
||||
assignments,
|
||||
)) => {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
peer_id = %peer_id,
|
||||
@@ -611,7 +643,10 @@ impl State {
|
||||
.await;
|
||||
}
|
||||
},
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals) => {
|
||||
Versioned::V1(protocol_v1::ApprovalDistributionMessage::Approvals(approvals)) |
|
||||
Versioned::VStaging(protocol_vstaging::ApprovalDistributionMessage::Approvals(
|
||||
approvals,
|
||||
)) => {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
peer_id = %peer_id,
|
||||
@@ -664,9 +699,14 @@ impl State {
|
||||
{
|
||||
gum::trace!(target: LOG_TARGET, ?view, "Peer view change");
|
||||
let finalized_number = view.finalized_number;
|
||||
let old_view =
|
||||
self.peer_views.get_mut(&peer_id).map(|d| std::mem::replace(d, view.clone()));
|
||||
let old_finalized_number = old_view.map(|v| v.finalized_number).unwrap_or(0);
|
||||
let (peer_protocol_version, old_finalized_number) = match self
|
||||
.peer_data
|
||||
.get_mut(&peer_id)
|
||||
.map(|d| (d.version, std::mem::replace(&mut d.view, view.clone())))
|
||||
{
|
||||
Some((v, view)) => (v, view.finalized_number),
|
||||
None => return, // unknown peer
|
||||
};
|
||||
|
||||
// we want to prune every block known_by peer up to (including) view.finalized_number
|
||||
let blocks = &mut self.blocks;
|
||||
@@ -691,8 +731,9 @@ impl State {
|
||||
metrics,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
self.peer_views.len(),
|
||||
self.peer_data.len(),
|
||||
peer_id,
|
||||
peer_protocol_version,
|
||||
view,
|
||||
rng,
|
||||
)
|
||||
@@ -992,7 +1033,7 @@ impl State {
|
||||
// then messages will be sent when we get it.
|
||||
|
||||
let assignments = vec![(assignment, claimed_candidate_index)];
|
||||
let n_peers_total = self.peer_views.len();
|
||||
let n_peers_total = self.peer_data.len();
|
||||
let source_peer = source.peer_id();
|
||||
|
||||
let mut peer_filter = move |peer| {
|
||||
@@ -1019,31 +1060,53 @@ impl State {
|
||||
route_random
|
||||
};
|
||||
|
||||
let peers = entry.known_by.keys().filter(|p| peer_filter(p)).cloned().collect::<Vec<_>>();
|
||||
let (v1_peers, vstaging_peers) = {
|
||||
let peer_data = &self.peer_data;
|
||||
let peers = entry
|
||||
.known_by
|
||||
.keys()
|
||||
.filter_map(|p| peer_data.get_key_value(p))
|
||||
.filter(|(p, _)| peer_filter(p))
|
||||
.map(|(p, peer_data)| (*p, peer_data.version))
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for peer in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(peer_knowledge) = entry.known_by.get_mut(peer) {
|
||||
peer_knowledge.sent.insert(message_subject.clone(), message_kind);
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for (peer, _) in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(peer_knowledge) = entry.known_by.get_mut(peer) {
|
||||
peer_knowledge.sent.insert(message_subject.clone(), message_kind);
|
||||
}
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?claimed_candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an assignment to peers",
|
||||
);
|
||||
}
|
||||
|
||||
let v1_peers = filter_peers_by_version(&peers, ValidationVersion::V1);
|
||||
let vstaging_peers = filter_peers_by_version(&peers, ValidationVersion::VStaging);
|
||||
|
||||
(v1_peers, vstaging_peers)
|
||||
};
|
||||
|
||||
if !v1_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
v1_peers,
|
||||
versioned_assignments_packet(ValidationVersion::V1, assignments.clone()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?claimed_candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an assignment to peers",
|
||||
);
|
||||
|
||||
if !vstaging_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments),
|
||||
)),
|
||||
vstaging_peers,
|
||||
versioned_assignments_packet(ValidationVersion::VStaging, assignments.clone()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
@@ -1332,38 +1395,55 @@ impl State {
|
||||
in_topology || knowledge.sent.contains(message_subject, MessageKind::Assignment)
|
||||
};
|
||||
|
||||
let peers = entry
|
||||
.known_by
|
||||
.iter()
|
||||
.filter(|(p, k)| peer_filter(p, k))
|
||||
.map(|(p, _)| p)
|
||||
.cloned()
|
||||
.collect::<Vec<_>>();
|
||||
let (v1_peers, vstaging_peers) = {
|
||||
let peer_data = &self.peer_data;
|
||||
let peers = entry
|
||||
.known_by
|
||||
.iter()
|
||||
.filter_map(|(p, k)| peer_data.get(&p).map(|pd| (p, k, pd.version)))
|
||||
.filter(|(p, k, _)| peer_filter(p, k))
|
||||
.map(|(p, _, v)| (*p, v))
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for peer in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(entry) = entry.known_by.get_mut(peer) {
|
||||
entry.sent.insert(message_subject.clone(), message_kind);
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for (peer, _) in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(peer_knowledge) = entry.known_by.get_mut(peer) {
|
||||
peer_knowledge.sent.insert(message_subject.clone(), message_kind);
|
||||
}
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an approval to peers",
|
||||
);
|
||||
}
|
||||
|
||||
let v1_peers = filter_peers_by_version(&peers, ValidationVersion::V1);
|
||||
let vstaging_peers = filter_peers_by_version(&peers, ValidationVersion::VStaging);
|
||||
|
||||
(v1_peers, vstaging_peers)
|
||||
};
|
||||
|
||||
let approvals = vec![vote];
|
||||
|
||||
if !v1_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
v1_peers,
|
||||
versioned_approvals_packet(ValidationVersion::V1, approvals.clone()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
let approvals = vec![vote];
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an approval to peers",
|
||||
);
|
||||
|
||||
if !vstaging_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals),
|
||||
)),
|
||||
vstaging_peers,
|
||||
versioned_approvals_packet(ValidationVersion::VStaging, approvals),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
@@ -1427,6 +1507,7 @@ impl State {
|
||||
topologies: &SessionGridTopologies,
|
||||
total_peers: usize,
|
||||
peer_id: PeerId,
|
||||
peer_protocol_version: ValidationVersion,
|
||||
view: View,
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
@@ -1536,7 +1617,8 @@ impl State {
|
||||
"Sending assignments to unified peer",
|
||||
);
|
||||
|
||||
send_assignments_batched(sender, assignments_to_send, peer_id).await;
|
||||
send_assignments_batched(sender, assignments_to_send, peer_id, peer_protocol_version)
|
||||
.await;
|
||||
}
|
||||
|
||||
if !approvals_to_send.is_empty() {
|
||||
@@ -1547,7 +1629,7 @@ impl State {
|
||||
"Sending approvals to unified peer",
|
||||
);
|
||||
|
||||
send_approvals_batched(sender, approvals_to_send, peer_id).await;
|
||||
send_approvals_batched(sender, approvals_to_send, peer_id, peer_protocol_version).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1583,6 +1665,7 @@ impl State {
|
||||
|
||||
adjust_required_routing_and_propagate(
|
||||
ctx,
|
||||
&self.peer_data,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
|block_entry| {
|
||||
@@ -1610,6 +1693,7 @@ impl State {
|
||||
|
||||
adjust_required_routing_and_propagate(
|
||||
ctx,
|
||||
&self.peer_data,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
|block_entry| {
|
||||
@@ -1669,6 +1753,7 @@ impl State {
|
||||
#[overseer::contextbounds(ApprovalDistribution, prefix = self::overseer)]
|
||||
async fn adjust_required_routing_and_propagate<Context, BlockFilter, RoutingModifier>(
|
||||
ctx: &mut Context,
|
||||
peer_data: &HashMap<PeerId, PeerData>,
|
||||
blocks: &mut HashMap<Hash, BlockEntry>,
|
||||
topologies: &SessionGridTopologies,
|
||||
block_filter: BlockFilter,
|
||||
@@ -1758,11 +1843,22 @@ async fn adjust_required_routing_and_propagate<Context, BlockFilter, RoutingModi
|
||||
// Send messages in accumulated packets, assignments preceding approvals.
|
||||
|
||||
for (peer, assignments_packet) in peer_assignments {
|
||||
send_assignments_batched(ctx.sender(), assignments_packet, peer).await;
|
||||
let peer_protocol_version = match peer_data.get(&peer).map(|pd| pd.version) {
|
||||
None => continue,
|
||||
Some(v) => v,
|
||||
};
|
||||
|
||||
send_assignments_batched(ctx.sender(), assignments_packet, peer, peer_protocol_version)
|
||||
.await;
|
||||
}
|
||||
|
||||
for (peer, approvals_packet) in peer_approvals {
|
||||
send_approvals_batched(ctx.sender(), approvals_packet, peer).await;
|
||||
let peer_protocol_version = match peer_data.get(&peer).map(|pd| pd.version) {
|
||||
None => continue,
|
||||
Some(v) => v,
|
||||
};
|
||||
|
||||
send_approvals_batched(ctx.sender(), approvals_packet, peer, peer_protocol_version).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1912,6 +2008,49 @@ impl ApprovalDistribution {
|
||||
}
|
||||
}
|
||||
|
||||
fn versioned_approvals_packet(
|
||||
version: ValidationVersion,
|
||||
approvals: Vec<IndirectSignedApprovalVote>,
|
||||
) -> VersionedValidationProtocol {
|
||||
match version {
|
||||
ValidationVersion::V1 =>
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals),
|
||||
)),
|
||||
ValidationVersion::VStaging =>
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(approvals),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn versioned_assignments_packet(
|
||||
version: ValidationVersion,
|
||||
assignments: Vec<(IndirectAssignmentCert, CandidateIndex)>,
|
||||
) -> VersionedValidationProtocol {
|
||||
match version {
|
||||
ValidationVersion::V1 =>
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments),
|
||||
)),
|
||||
ValidationVersion::VStaging =>
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Assignments(assignments),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn filter_peers_by_version(
|
||||
peers: &[(PeerId, ValidationVersion)],
|
||||
version: ValidationVersion,
|
||||
) -> Vec<PeerId> {
|
||||
peers
|
||||
.iter()
|
||||
.filter(|(_, v)| v == &version)
|
||||
.map(|(peer_id, _)| *peer_id)
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[overseer::subsystem(ApprovalDistribution, error=SubsystemError, prefix=self::overseer)]
|
||||
impl<Context> ApprovalDistribution {
|
||||
fn start(self, ctx: Context) -> SpawnedSubsystem {
|
||||
@@ -1954,19 +2093,16 @@ pub(crate) async fn send_assignments_batched(
|
||||
sender: &mut impl overseer::ApprovalDistributionSenderTrait,
|
||||
assignments: Vec<(IndirectAssignmentCert, CandidateIndex)>,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
) {
|
||||
let mut batches = assignments.into_iter().peekable();
|
||||
|
||||
while batches.peek().is_some() {
|
||||
let batch: Vec<_> = batches.by_ref().take(MAX_ASSIGNMENT_BATCH_SIZE).collect();
|
||||
let versioned = versioned_assignments_packet(protocol_version, batch);
|
||||
|
||||
sender
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vec![peer],
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(batch),
|
||||
)),
|
||||
))
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(vec![peer], versioned))
|
||||
.await;
|
||||
}
|
||||
}
|
||||
@@ -1976,19 +2112,16 @@ pub(crate) async fn send_approvals_batched(
|
||||
sender: &mut impl overseer::ApprovalDistributionSenderTrait,
|
||||
approvals: Vec<IndirectSignedApprovalVote>,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
) {
|
||||
let mut batches = approvals.into_iter().peekable();
|
||||
|
||||
while batches.peek().is_some() {
|
||||
let batch: Vec<_> = batches.by_ref().take(MAX_APPROVAL_BATCH_SIZE).collect();
|
||||
let versioned = versioned_approvals_packet(protocol_version, batch);
|
||||
|
||||
sender
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vec![peer],
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(batch),
|
||||
)),
|
||||
))
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(vec![peer], versioned))
|
||||
.await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -219,6 +219,7 @@ async fn setup_gossip_topology(
|
||||
async fn setup_peer_with_view(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer_id: &PeerId,
|
||||
validation_version: ValidationVersion,
|
||||
view: View,
|
||||
) {
|
||||
overseer_send(
|
||||
@@ -226,7 +227,7 @@ async fn setup_peer_with_view(
|
||||
ApprovalDistributionMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerConnected(
|
||||
*peer_id,
|
||||
ObservedRole::Full,
|
||||
ValidationVersion::V1.into(),
|
||||
validation_version.into(),
|
||||
None,
|
||||
)),
|
||||
)
|
||||
@@ -243,13 +244,12 @@ async fn setup_peer_with_view(
|
||||
async fn send_message_from_peer(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer_id: &PeerId,
|
||||
msg: protocol_v1::ApprovalDistributionMessage,
|
||||
msg: net_protocol::ApprovalDistributionMessage,
|
||||
) {
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
ApprovalDistributionMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
*peer_id,
|
||||
Versioned::V1(msg),
|
||||
*peer_id, msg,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
@@ -331,9 +331,9 @@ fn try_import_the_same_assignment() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup peers
|
||||
setup_peer_with_view(overseer, &peer_a, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::V1, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -353,7 +353,7 @@ fn try_import_the_same_assignment() {
|
||||
let assignments = vec![(cert.clone(), 0u32)];
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, &peer_a, msg).await;
|
||||
send_message_from_peer(overseer, &peer_a, Versioned::V1(msg)).await;
|
||||
|
||||
expect_reputation_change(overseer, &peer_a, COST_UNEXPECTED_MESSAGE).await;
|
||||
|
||||
@@ -386,11 +386,11 @@ fn try_import_the_same_assignment() {
|
||||
);
|
||||
|
||||
// setup new peer
|
||||
setup_peer_with_view(overseer, &peer_d, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_d, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// send the same assignment from peer_d
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments);
|
||||
send_message_from_peer(overseer, &peer_d, msg).await;
|
||||
send_message_from_peer(overseer, &peer_d, Versioned::V1(msg)).await;
|
||||
|
||||
expect_reputation_change(overseer, &peer_d, COST_UNEXPECTED_MESSAGE).await;
|
||||
expect_reputation_change(overseer, &peer_d, BENEFIT_VALID_MESSAGE).await;
|
||||
@@ -413,7 +413,7 @@ fn delay_reputation_change() {
|
||||
let overseer = &mut virtual_overseer;
|
||||
|
||||
// Setup peers
|
||||
setup_peer_with_view(overseer, &peer, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -433,7 +433,7 @@ fn delay_reputation_change() {
|
||||
let assignments = vec![(cert.clone(), 0u32)];
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, &peer, msg).await;
|
||||
send_message_from_peer(overseer, &peer, Versioned::V1(msg)).await;
|
||||
|
||||
// send an `Accept` message from the Approval Voting subsystem
|
||||
assert_matches!(
|
||||
@@ -474,7 +474,7 @@ fn spam_attack_results_in_negative_reputation_change() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
let peer = &peer_a;
|
||||
setup_peer_with_view(overseer, peer, view![]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// new block `hash_b` with 20 candidates
|
||||
let candidates_count = 20;
|
||||
@@ -501,7 +501,7 @@ fn spam_attack_results_in_negative_reputation_change() {
|
||||
.collect();
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
for i in 0..candidates_count {
|
||||
expect_reputation_change(overseer, peer, COST_UNEXPECTED_MESSAGE).await;
|
||||
@@ -533,7 +533,7 @@ fn spam_attack_results_in_negative_reputation_change() {
|
||||
.await;
|
||||
|
||||
// send the assignments again
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
// each of them will incur `COST_UNEXPECTED_MESSAGE`, not only the first one
|
||||
for _ in 0..candidates_count {
|
||||
@@ -558,7 +558,7 @@ fn peer_sending_us_the_same_we_just_sent_them_is_ok() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
let peer = &peer_a;
|
||||
setup_peer_with_view(overseer, peer, view![]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// new block `hash` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -610,12 +610,12 @@ fn peer_sending_us_the_same_we_just_sent_them_is_ok() {
|
||||
// the peer could send us it as well
|
||||
let assignments = vec![(cert, candidate_index)];
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments);
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
assert!(overseer.recv().timeout(TIMEOUT).await.is_none(), "we should not punish the peer");
|
||||
|
||||
// send the assignments again
|
||||
send_message_from_peer(overseer, peer, msg).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg)).await;
|
||||
|
||||
// now we should
|
||||
expect_reputation_change(overseer, peer, COST_DUPLICATE_MESSAGE).await;
|
||||
@@ -634,9 +634,9 @@ fn import_approval_happy_path() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup peers
|
||||
setup_peer_with_view(overseer, &peer_a, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::V1, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -681,7 +681,7 @@ fn import_approval_happy_path() {
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -722,8 +722,8 @@ fn import_approval_bad() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup peers
|
||||
setup_peer_with_view(overseer, &peer_a, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::V1, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -749,14 +749,14 @@ fn import_approval_bad() {
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
expect_reputation_change(overseer, &peer_b, COST_UNEXPECTED_MESSAGE).await;
|
||||
|
||||
// now import an assignment from peer_b
|
||||
let assignments = vec![(cert.clone(), candidate_index)];
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -775,7 +775,7 @@ fn import_approval_bad() {
|
||||
|
||||
// and try again
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -916,7 +916,7 @@ fn update_peer_view() {
|
||||
overseer_send(overseer, ApprovalDistributionMessage::DistributeAssignment(cert_b, 0)).await;
|
||||
|
||||
// connect a peer
|
||||
setup_peer_with_view(overseer, peer, view![hash_a]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash_a]).await;
|
||||
|
||||
// we should send relevant assignments to the peer
|
||||
assert_matches!(
|
||||
@@ -934,7 +934,7 @@ fn update_peer_view() {
|
||||
virtual_overseer
|
||||
});
|
||||
|
||||
assert_eq!(state.peer_views.get(peer).map(|v| v.finalized_number), Some(0));
|
||||
assert_eq!(state.peer_data.get(peer).map(|data| data.view.finalized_number), Some(0));
|
||||
assert_eq!(
|
||||
state
|
||||
.blocks
|
||||
@@ -986,7 +986,7 @@ fn update_peer_view() {
|
||||
virtual_overseer
|
||||
});
|
||||
|
||||
assert_eq!(state.peer_views.get(peer).map(|v| v.finalized_number), Some(2));
|
||||
assert_eq!(state.peer_data.get(peer).map(|data| data.view.finalized_number), Some(2));
|
||||
assert_eq!(
|
||||
state
|
||||
.blocks
|
||||
@@ -1016,7 +1016,10 @@ fn update_peer_view() {
|
||||
virtual_overseer
|
||||
});
|
||||
|
||||
assert_eq!(state.peer_views.get(peer).map(|v| v.finalized_number), Some(finalized_number));
|
||||
assert_eq!(
|
||||
state.peer_data.get(peer).map(|data| data.view.finalized_number),
|
||||
Some(finalized_number)
|
||||
);
|
||||
assert!(state.blocks.get(&hash_c).unwrap().known_by.get(peer).is_none());
|
||||
}
|
||||
|
||||
@@ -1031,7 +1034,7 @@ fn import_remotely_then_locally() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup the peer
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -1051,7 +1054,7 @@ fn import_remotely_then_locally() {
|
||||
let cert = fake_assignment_cert(hash, validator_index);
|
||||
let assignments = vec![(cert.clone(), candidate_index)];
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, peer, msg).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg)).await;
|
||||
|
||||
// send an `Accept` message from the Approval Voting subsystem
|
||||
assert_matches!(
|
||||
@@ -1086,7 +1089,7 @@ fn import_remotely_then_locally() {
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, peer, msg).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -1152,7 +1155,7 @@ fn sends_assignments_even_when_state_is_approved() {
|
||||
.await;
|
||||
|
||||
// connect the peer.
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
let assignments = vec![(cert.clone(), candidate_index)];
|
||||
let approvals = vec![approval.clone()];
|
||||
@@ -1216,7 +1219,7 @@ fn race_condition_in_local_vs_remote_view_update() {
|
||||
};
|
||||
|
||||
// This will send a peer view that is ahead of our view
|
||||
setup_peer_with_view(overseer, peer, view![hash_b]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash_b]).await;
|
||||
|
||||
// Send our view update to include a new head
|
||||
overseer_send(
|
||||
@@ -1237,7 +1240,7 @@ fn race_condition_in_local_vs_remote_view_update() {
|
||||
.collect();
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
// This will handle pending messages being processed
|
||||
let msg = ApprovalDistributionMessage::NewBlocks(vec![meta]);
|
||||
@@ -1280,7 +1283,7 @@ fn propagates_locally_generated_assignment_to_both_dimensions() {
|
||||
|
||||
// Connect all peers.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// Set up a gossip topology.
|
||||
@@ -1385,7 +1388,7 @@ fn propagates_assignments_along_unshared_dimension() {
|
||||
|
||||
// Connect all peers.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// Set up a gossip topology.
|
||||
@@ -1421,7 +1424,7 @@ fn propagates_assignments_along_unshared_dimension() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -1470,7 +1473,7 @@ fn propagates_assignments_along_unshared_dimension() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -1527,7 +1530,7 @@ fn propagates_to_required_after_connect() {
|
||||
// Connect all peers except omitted.
|
||||
for (i, (peer, _)) in peers.iter().enumerate() {
|
||||
if !omitted.contains(&i) {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1616,7 +1619,7 @@ fn propagates_to_required_after_connect() {
|
||||
);
|
||||
|
||||
for i in omitted.iter().copied() {
|
||||
setup_peer_with_view(overseer, &peers[i].0, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peers[i].0, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -1665,7 +1668,7 @@ fn sends_to_more_peers_after_getting_topology() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -1817,7 +1820,7 @@ fn originator_aggression_l1() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -1976,7 +1979,7 @@ fn non_originator_aggression_l1() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -2010,7 +2013,7 @@ fn non_originator_aggression_l1() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -2081,7 +2084,7 @@ fn non_originator_aggression_l2() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -2115,7 +2118,7 @@ fn non_originator_aggression_l2() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -2246,7 +2249,7 @@ fn resends_messages_periodically() {
|
||||
|
||||
// Connect all peers.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// Set up a gossip topology.
|
||||
@@ -2281,7 +2284,7 @@ fn resends_messages_periodically() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -2372,6 +2375,126 @@ fn resends_messages_periodically() {
|
||||
});
|
||||
}
|
||||
|
||||
/// Tests that peers correctly receive versioned messages.
|
||||
#[test]
|
||||
fn import_versioned_approval() {
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
let parent_hash = Hash::repeat_byte(0xFF);
|
||||
let hash = Hash::repeat_byte(0xAA);
|
||||
|
||||
let state = state_without_reputation_delay();
|
||||
let _ = test_harness(state, |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// All peers are aware of relay parent.
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::VStaging, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, ValidationVersion::VStaging, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
hash,
|
||||
parent_hash,
|
||||
number: 1,
|
||||
candidates: vec![Default::default(); 1],
|
||||
slot: 1.into(),
|
||||
session: 1,
|
||||
};
|
||||
let msg = ApprovalDistributionMessage::NewBlocks(vec![meta]);
|
||||
overseer_send(overseer, msg).await;
|
||||
|
||||
// import an assignment related to `hash` locally
|
||||
let validator_index = ValidatorIndex(0);
|
||||
let candidate_index = 0u32;
|
||||
let cert = fake_assignment_cert(hash, validator_index);
|
||||
overseer_send(
|
||||
overseer,
|
||||
ApprovalDistributionMessage::DistributeAssignment(cert, candidate_index),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_b]);
|
||||
assert_eq!(assignments.len(), 1);
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Assignments(assignments)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers.len(), 2);
|
||||
assert!(peers.contains(&peer_a));
|
||||
assert!(peers.contains(&peer_c));
|
||||
|
||||
assert_eq!(assignments.len(), 1);
|
||||
}
|
||||
);
|
||||
|
||||
// send the an approval from peer_a
|
||||
let approval = IndirectSignedApprovalVote {
|
||||
block_hash: hash,
|
||||
candidate_index,
|
||||
validator: validator_index,
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_vstaging::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_a, Versioned::VStaging(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportApproval(
|
||||
vote,
|
||||
tx,
|
||||
)) => {
|
||||
assert_eq!(vote, approval);
|
||||
tx.send(ApprovalCheckResult::Accepted).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
expect_reputation_change(overseer, &peer_a, BENEFIT_VALID_MESSAGE_FIRST).await;
|
||||
|
||||
// Peers b and c receive versioned approval messages.
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_b]);
|
||||
assert_eq!(approvals.len(), 1);
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(approvals)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_c]);
|
||||
assert_eq!(approvals.len(), 1);
|
||||
}
|
||||
);
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
fn batch_test_round(message_count: usize) {
|
||||
use polkadot_node_subsystem::SubsystemContext;
|
||||
let pool = sp_core::testing::TaskExecutor::new();
|
||||
@@ -2402,8 +2525,9 @@ fn batch_test_round(message_count: usize) {
|
||||
.collect();
|
||||
|
||||
let peer = PeerId::random();
|
||||
send_assignments_batched(&mut sender, assignments.clone(), peer).await;
|
||||
send_approvals_batched(&mut sender, approvals.clone(), peer).await;
|
||||
send_assignments_batched(&mut sender, assignments.clone(), peer, ValidationVersion::V1)
|
||||
.await;
|
||||
send_approvals_batched(&mut sender, approvals.clone(), peer, ValidationVersion::V1).await;
|
||||
|
||||
// Check expected assignments batches.
|
||||
for assignment_index in (0..assignments.len()).step_by(super::MAX_ASSIGNMENT_BATCH_SIZE) {
|
||||
|
||||
@@ -6,6 +6,7 @@ edition.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
[dependencies]
|
||||
always-assert = "0.1"
|
||||
futures = "0.3.21"
|
||||
futures-timer = "3.0.2"
|
||||
gum = { package = "tracing-gum", path = "../../gum" }
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
|
||||
#![deny(unused_crate_dependencies)]
|
||||
|
||||
use always_assert::never;
|
||||
use futures::{channel::oneshot, FutureExt};
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
@@ -29,7 +30,9 @@ use polkadot_node_network_protocol::{
|
||||
grid_topology::{
|
||||
GridNeighbors, RandomRouting, RequiredRouting, SessionBoundGridTopologyStorage,
|
||||
},
|
||||
v1 as protocol_v1, OurView, PeerId, UnifiedReputationChange as Rep, Versioned, View,
|
||||
peer_set::{ProtocolVersion, ValidationVersion},
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, OurView, PeerId,
|
||||
UnifiedReputationChange as Rep, Versioned, View,
|
||||
};
|
||||
use polkadot_node_subsystem::{
|
||||
jaeger, messages::*, overseer, ActiveLeavesUpdate, FromOrchestra, OverseerSignal, PerLeafSpan,
|
||||
@@ -76,25 +79,63 @@ struct BitfieldGossipMessage {
|
||||
}
|
||||
|
||||
impl BitfieldGossipMessage {
|
||||
fn into_validation_protocol(self) -> net_protocol::VersionedValidationProtocol {
|
||||
self.into_network_message().into()
|
||||
fn into_validation_protocol(
|
||||
self,
|
||||
recipient_version: ProtocolVersion,
|
||||
) -> net_protocol::VersionedValidationProtocol {
|
||||
self.into_network_message(recipient_version).into()
|
||||
}
|
||||
|
||||
fn into_network_message(self) -> net_protocol::BitfieldDistributionMessage {
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
))
|
||||
fn into_network_message(
|
||||
self,
|
||||
recipient_version: ProtocolVersion,
|
||||
) -> net_protocol::BitfieldDistributionMessage {
|
||||
match ValidationVersion::try_from(recipient_version).ok() {
|
||||
Some(ValidationVersion::V1) =>
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
)),
|
||||
Some(ValidationVersion::VStaging) =>
|
||||
Versioned::VStaging(protocol_vstaging::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
)),
|
||||
None => {
|
||||
never!("Peers should only have supported protocol versions.");
|
||||
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?recipient_version,
|
||||
"Unknown protocol version provided for message recipient"
|
||||
);
|
||||
|
||||
// fall back to v1 to avoid
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
))
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Data stored on a per-peer basis.
|
||||
#[derive(Debug)]
|
||||
pub struct PeerData {
|
||||
/// The peer's view.
|
||||
view: View,
|
||||
/// The peer's protocol version.
|
||||
version: ProtocolVersion,
|
||||
}
|
||||
|
||||
/// Data used to track information of peers and relay parents the
|
||||
/// overseer ordered us to work on.
|
||||
#[derive(Default, Debug)]
|
||||
struct ProtocolState {
|
||||
/// Track all active peers and their views
|
||||
/// to determine what is relevant to them.
|
||||
peer_views: HashMap<PeerId, View>,
|
||||
peer_data: HashMap<PeerId, PeerData>,
|
||||
|
||||
/// The current and previous gossip topologies
|
||||
topologies: SessionBoundGridTopologyStorage,
|
||||
@@ -357,7 +398,7 @@ async fn handle_bitfield_distribution<Context>(
|
||||
ctx,
|
||||
job_data,
|
||||
topology,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator,
|
||||
msg,
|
||||
required_routing,
|
||||
@@ -376,7 +417,7 @@ async fn relay_message<Context>(
|
||||
ctx: &mut Context,
|
||||
job_data: &mut PerRelayParentData,
|
||||
topology_neighbors: &GridNeighbors,
|
||||
peer_views: &mut HashMap<PeerId, View>,
|
||||
peers: &mut HashMap<PeerId, PeerData>,
|
||||
validator: ValidatorId,
|
||||
message: BitfieldGossipMessage,
|
||||
required_routing: RequiredRouting,
|
||||
@@ -394,16 +435,16 @@ async fn relay_message<Context>(
|
||||
.await;
|
||||
|
||||
drop(_span);
|
||||
let total_peers = peer_views.len();
|
||||
let total_peers = peers.len();
|
||||
let mut random_routing: RandomRouting = Default::default();
|
||||
|
||||
let _span = span.child("interested-peers");
|
||||
// pass on the bitfield distribution to all interested peers
|
||||
let interested_peers = peer_views
|
||||
let interested_peers = peers
|
||||
.iter()
|
||||
.filter_map(|(peer, view)| {
|
||||
.filter_map(|(peer, data)| {
|
||||
// check interest in the peer in this message's relay parent
|
||||
if view.contains(&message.relay_parent) {
|
||||
if data.view.contains(&message.relay_parent) {
|
||||
let message_needed =
|
||||
job_data.message_from_validator_needed_by_peer(&peer, &validator);
|
||||
if message_needed {
|
||||
@@ -418,7 +459,7 @@ async fn relay_message<Context>(
|
||||
};
|
||||
|
||||
if need_routing {
|
||||
Some(*peer)
|
||||
Some((*peer, data.version))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
@@ -429,9 +470,9 @@ async fn relay_message<Context>(
|
||||
None
|
||||
}
|
||||
})
|
||||
.collect::<Vec<PeerId>>();
|
||||
.collect::<Vec<(PeerId, ProtocolVersion)>>();
|
||||
|
||||
interested_peers.iter().for_each(|peer| {
|
||||
interested_peers.iter().for_each(|(peer, _)| {
|
||||
// track the message as sent for this peer
|
||||
job_data
|
||||
.message_sent_to_peer
|
||||
@@ -450,11 +491,35 @@ async fn relay_message<Context>(
|
||||
);
|
||||
} else {
|
||||
let _span = span.child("gossip");
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
interested_peers,
|
||||
message.into_validation_protocol(),
|
||||
))
|
||||
.await;
|
||||
|
||||
let filter_by_version = |peers: &[(PeerId, ProtocolVersion)],
|
||||
version: ValidationVersion| {
|
||||
peers
|
||||
.iter()
|
||||
.filter(|(_, v)| v == &version.into())
|
||||
.map(|(peer_id, _)| *peer_id)
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
|
||||
let v1_interested_peers = filter_by_version(&interested_peers, ValidationVersion::V1);
|
||||
let vstaging_interested_peers =
|
||||
filter_by_version(&interested_peers, ValidationVersion::VStaging);
|
||||
|
||||
if !v1_interested_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
v1_interested_peers,
|
||||
message.clone().into_validation_protocol(ValidationVersion::V1.into()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
if !vstaging_interested_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vstaging_interested_peers,
|
||||
message.into_validation_protocol(ValidationVersion::VStaging.into()),
|
||||
))
|
||||
.await
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -465,10 +530,20 @@ async fn process_incoming_peer_message<Context>(
|
||||
state: &mut ProtocolState,
|
||||
metrics: &Metrics,
|
||||
origin: PeerId,
|
||||
message: protocol_v1::BitfieldDistributionMessage,
|
||||
message: net_protocol::BitfieldDistributionMessage,
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
let protocol_v1::BitfieldDistributionMessage::Bitfield(relay_parent, bitfield) = message;
|
||||
let (relay_parent, bitfield) = match message {
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
relay_parent,
|
||||
bitfield,
|
||||
)) => (relay_parent, bitfield),
|
||||
Versioned::VStaging(protocol_vstaging::BitfieldDistributionMessage::Bitfield(
|
||||
relay_parent,
|
||||
bitfield,
|
||||
)) => (relay_parent, bitfield),
|
||||
};
|
||||
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
peer = %origin,
|
||||
@@ -616,7 +691,7 @@ async fn process_incoming_peer_message<Context>(
|
||||
ctx,
|
||||
job_data,
|
||||
topology,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator,
|
||||
message,
|
||||
required_routing,
|
||||
@@ -647,15 +722,18 @@ async fn handle_network_msg<Context>(
|
||||
let _timer = metrics.time_handle_network_msg();
|
||||
|
||||
match bridge_message {
|
||||
NetworkBridgeEvent::PeerConnected(peer, role, _, _) => {
|
||||
NetworkBridgeEvent::PeerConnected(peer, role, version, _) => {
|
||||
gum::trace!(target: LOG_TARGET, ?peer, ?role, "Peer connected");
|
||||
// insert if none already present
|
||||
state.peer_views.entry(peer).or_default();
|
||||
state
|
||||
.peer_data
|
||||
.entry(peer)
|
||||
.or_insert_with(|| PeerData { view: View::default(), version });
|
||||
},
|
||||
NetworkBridgeEvent::PeerDisconnected(peer) => {
|
||||
gum::trace!(target: LOG_TARGET, ?peer, "Peer disconnected");
|
||||
// get rid of superfluous data
|
||||
state.peer_views.remove(&peer);
|
||||
state.peer_data.remove(&peer);
|
||||
},
|
||||
NetworkBridgeEvent::NewGossipTopology(gossip_topology) => {
|
||||
let session_index = gossip_topology.session;
|
||||
@@ -680,12 +758,21 @@ async fn handle_network_msg<Context>(
|
||||
);
|
||||
|
||||
for new_peer in newly_added {
|
||||
// in case we already knew that peer in the past
|
||||
// it might have had an existing view, we use to initialize
|
||||
// and minimize the delta on `PeerViewChange` to be sent
|
||||
if let Some(old_view) = state.peer_views.remove(&new_peer) {
|
||||
handle_peer_view_change(ctx, state, new_peer, old_view, rng).await;
|
||||
}
|
||||
let old_view = match state.peer_data.get_mut(&new_peer) {
|
||||
Some(d) => {
|
||||
// in case we already knew that peer in the past
|
||||
// it might have had an existing view, we use to initialize
|
||||
// and minimize the delta on `PeerViewChange` to be sent
|
||||
std::mem::replace(&mut d.view, Default::default())
|
||||
},
|
||||
None => {
|
||||
// For peers which are currently unknown, we'll send topology-related
|
||||
// messages to them when they connect and send their first view update.
|
||||
continue
|
||||
},
|
||||
};
|
||||
|
||||
handle_peer_view_change(ctx, state, new_peer, old_view, rng).await;
|
||||
}
|
||||
},
|
||||
NetworkBridgeEvent::PeerViewChange(peerid, new_view) => {
|
||||
@@ -696,7 +783,7 @@ async fn handle_network_msg<Context>(
|
||||
gum::trace!(target: LOG_TARGET, ?new_view, "Our view change");
|
||||
handle_our_view_change(state, new_view);
|
||||
},
|
||||
NetworkBridgeEvent::PeerMessage(remote, Versioned::V1(message)) =>
|
||||
NetworkBridgeEvent::PeerMessage(remote, message) =>
|
||||
process_incoming_peer_message(ctx, state, metrics, remote, message, rng).await,
|
||||
NetworkBridgeEvent::UpdatedAuthorityIds { .. } => {
|
||||
// The bitfield-distribution subsystem doesn't deal with `AuthorityDiscoveryId`s.
|
||||
@@ -728,6 +815,9 @@ fn handle_our_view_change(state: &mut ProtocolState, view: OurView) {
|
||||
|
||||
// Send the difference between two views which were not sent
|
||||
// to that particular peer.
|
||||
//
|
||||
// This requires that there is an entry in the `peer_data` field for the
|
||||
// peer.
|
||||
#[overseer::contextbounds(BitfieldDistribution, prefix=self::overseer)]
|
||||
async fn handle_peer_view_change<Context>(
|
||||
ctx: &mut Context,
|
||||
@@ -736,13 +826,20 @@ async fn handle_peer_view_change<Context>(
|
||||
view: View,
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
let added = state
|
||||
.peer_views
|
||||
.entry(origin)
|
||||
.or_default()
|
||||
.replace_difference(view)
|
||||
.cloned()
|
||||
.collect::<Vec<_>>();
|
||||
let peer_data = match state.peer_data.get_mut(&origin) {
|
||||
None => {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
peer = ?origin,
|
||||
"Attempted to update peer view for unknown peer."
|
||||
);
|
||||
|
||||
return
|
||||
},
|
||||
Some(pd) => pd,
|
||||
};
|
||||
|
||||
let added = peer_data.view.replace_difference(view).cloned().collect::<Vec<_>>();
|
||||
|
||||
let topology = state.topologies.get_current_topology().local_grid_neighbors();
|
||||
let is_gossip_peer = topology.route_to_peer(RequiredRouting::GridXY, &origin);
|
||||
@@ -808,11 +905,14 @@ async fn send_tracked_gossip_message<Context>(
|
||||
"Sending gossip message"
|
||||
);
|
||||
|
||||
let version =
|
||||
if let Some(peer_data) = state.peer_data.get(&dest) { peer_data.version } else { return };
|
||||
|
||||
job_data.message_sent_to_peer.entry(dest).or_default().insert(validator.clone());
|
||||
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vec![dest],
|
||||
message.into_validation_protocol(),
|
||||
message.into_validation_protocol(version),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
@@ -56,6 +56,10 @@ fn dummy_rng() -> ChaCha12Rng {
|
||||
rand_chacha::ChaCha12Rng::seed_from_u64(12345)
|
||||
}
|
||||
|
||||
fn peer_data_v1(view: View) -> PeerData {
|
||||
PeerData { view, version: ValidationVersion::V1.into() }
|
||||
}
|
||||
|
||||
/// A very limited state, only interested in the relay parent of the
|
||||
/// given message, which must be signed by `validator` and a set of peers
|
||||
/// which are also only interested in that relay parent.
|
||||
@@ -85,7 +89,11 @@ fn prewarmed_state(
|
||||
span: PerLeafSpan::new(Arc::new(jaeger::Span::Disabled), "test"),
|
||||
},
|
||||
},
|
||||
peer_views: peers.iter().cloned().map(|peer| (peer, view!(relay_parent))).collect(),
|
||||
peer_data: peers
|
||||
.iter()
|
||||
.cloned()
|
||||
.map(|peer| (peer, peer_data_v1(view![relay_parent])))
|
||||
.collect(),
|
||||
topologies,
|
||||
view: our_view!(relay_parent),
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -211,7 +219,10 @@ fn receive_invalid_signature() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, invalid_msg.into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
invalid_msg.into_network_message(ValidationVersion::V1.into())
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -222,7 +233,10 @@ fn receive_invalid_signature() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, invalid_msg_2.into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
invalid_msg_2.into_network_message(ValidationVersion::V1.into())
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
// reputation change due to invalid signature
|
||||
@@ -256,7 +270,7 @@ fn receive_invalid_validator_index() {
|
||||
let (mut state, signing_context, keystore, validator) =
|
||||
state_with_view(our_view![hash_a, hash_b], hash_a, ReputationAggregator::new(|_| true));
|
||||
|
||||
state.peer_views.insert(peer_b, view![hash_a]);
|
||||
state.peer_data.insert(peer_b, peer_data_v1(view![hash_a]));
|
||||
|
||||
let payload = AvailabilityBitfield(bitvec![u8, bitvec::order::Lsb0; 1u8; 32]);
|
||||
let signed = Signed::<AvailabilityBitfield>::sign(
|
||||
@@ -281,7 +295,10 @@ fn receive_invalid_validator_index() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.into_network_message(ValidationVersion::V1.into())
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -344,7 +361,10 @@ fn receive_duplicate_messages() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -377,7 +397,10 @@ fn receive_duplicate_messages() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_a, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_a,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -396,7 +419,10 @@ fn receive_duplicate_messages() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -463,7 +489,10 @@ fn delay_reputation_change() {
|
||||
handle
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: BitfieldDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(peer, msg.clone().into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
@@ -486,7 +515,10 @@ fn delay_reputation_change() {
|
||||
handle
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: BitfieldDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(peer, msg.clone().into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
@@ -546,8 +578,8 @@ fn do_not_relay_message_twice() {
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
|
||||
state.peer_views.insert(peer_b, view![hash]);
|
||||
state.peer_views.insert(peer_a, view![hash]);
|
||||
state.peer_data.insert(peer_b, peer_data_v1(view![hash]));
|
||||
state.peer_data.insert(peer_a, peer_data_v1(view![hash]));
|
||||
|
||||
let msg =
|
||||
BitfieldGossipMessage { relay_parent: hash, signed_availability: signed_bitfield.clone() };
|
||||
@@ -564,7 +596,7 @@ fn do_not_relay_message_twice() {
|
||||
&mut ctx,
|
||||
state.per_relay_parent.get_mut(&hash).unwrap(),
|
||||
&gossip_peers,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator.clone(),
|
||||
msg.clone(),
|
||||
RequiredRouting::GridXY,
|
||||
@@ -591,7 +623,7 @@ fn do_not_relay_message_twice() {
|
||||
assert_eq!(2, peers.len());
|
||||
assert!(peers.contains(&peer_a));
|
||||
assert!(peers.contains(&peer_b));
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol());
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
@@ -600,7 +632,7 @@ fn do_not_relay_message_twice() {
|
||||
&mut ctx,
|
||||
state.per_relay_parent.get_mut(&hash).unwrap(),
|
||||
&gossip_peers,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator.clone(),
|
||||
msg.clone(),
|
||||
RequiredRouting::GridXY,
|
||||
@@ -687,14 +719,17 @@ fn changing_view() {
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
assert!(state.peer_views.contains_key(&peer_b));
|
||||
assert!(state.peer_data.contains_key(&peer_b));
|
||||
|
||||
// recv a first message from the network
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -729,8 +764,11 @@ fn changing_view() {
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
assert!(state.peer_views.contains_key(&peer_b));
|
||||
assert_eq!(state.peer_views.get(&peer_b).expect("Must contain value for peer B"), &view![]);
|
||||
assert!(state.peer_data.contains_key(&peer_b));
|
||||
assert_eq!(
|
||||
&state.peer_data.get(&peer_b).expect("Must contain value for peer B").view,
|
||||
&view![]
|
||||
);
|
||||
|
||||
// on rx of the same message, since we are not interested,
|
||||
// should give penalty
|
||||
@@ -738,7 +776,10 @@ fn changing_view() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -770,7 +811,10 @@ fn changing_view() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_a, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_a,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -817,8 +861,8 @@ fn do_not_send_message_back_to_origin() {
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
|
||||
state.peer_views.insert(peer_b, view![hash]);
|
||||
state.peer_views.insert(peer_a, view![hash]);
|
||||
state.peer_data.insert(peer_b, peer_data_v1(view![hash]));
|
||||
state.peer_data.insert(peer_a, peer_data_v1(view![hash]));
|
||||
|
||||
let msg =
|
||||
BitfieldGossipMessage { relay_parent: hash, signed_availability: signed_bitfield.clone() };
|
||||
@@ -833,7 +877,10 @@ fn do_not_send_message_back_to_origin() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -855,7 +902,7 @@ fn do_not_send_message_back_to_origin() {
|
||||
) => {
|
||||
assert_eq!(1, peers.len());
|
||||
assert!(peers.contains(&peer_a));
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol());
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
@@ -932,7 +979,7 @@ fn topology_test() {
|
||||
.expect("should be signed");
|
||||
|
||||
peers_x.iter().chain(peers_y.iter()).for_each(|peer| {
|
||||
state.peer_views.insert(*peer, view![hash]);
|
||||
state.peer_data.insert(*peer, peer_data_v1(view![hash]));
|
||||
});
|
||||
|
||||
let msg =
|
||||
@@ -948,7 +995,10 @@ fn topology_test() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peers_x[0], msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peers_x[0],
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -975,7 +1025,7 @@ fn topology_test() {
|
||||
assert!(topology.peers_x.iter().filter(|peer| peers.contains(&peer)).count() == 4);
|
||||
// Must never include originator
|
||||
assert!(!peers.contains(&peers_x[0]));
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol());
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
@@ -1050,3 +1100,127 @@ fn need_message_works() {
|
||||
// also not ok for Bob
|
||||
assert!(!pretend_send(&mut state, peer_b, &validator_set[1]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning() {
|
||||
let hash_a: Hash = [0; 32].into();
|
||||
let hash_b: Hash = [1; 32].into();
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
|
||||
let peers = [
|
||||
(peer_a, ValidationVersion::VStaging),
|
||||
(peer_b, ValidationVersion::V1),
|
||||
(peer_c, ValidationVersion::VStaging),
|
||||
];
|
||||
|
||||
// validator 0 key pair
|
||||
let (mut state, signing_context, keystore, validator) =
|
||||
state_with_view(our_view![hash_a, hash_b], hash_a, ReputationAggregator::new(|_| true));
|
||||
|
||||
let pool = sp_core::testing::TaskExecutor::new();
|
||||
let (mut ctx, mut handle) = make_subsystem_context::<BitfieldDistributionMessage, _>(pool);
|
||||
let mut rng = dummy_rng();
|
||||
|
||||
executor::block_on(async move {
|
||||
// create a signed message by validator 0
|
||||
let payload = AvailabilityBitfield(bitvec![u8, bitvec::order::Lsb0; 1u8; 32]);
|
||||
let signed_bitfield = Signed::<AvailabilityBitfield>::sign(
|
||||
&keystore,
|
||||
payload,
|
||||
&signing_context,
|
||||
ValidatorIndex(0),
|
||||
&validator,
|
||||
)
|
||||
.ok()
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
let msg = BitfieldGossipMessage {
|
||||
relay_parent: hash_a,
|
||||
signed_availability: signed_bitfield.clone(),
|
||||
};
|
||||
|
||||
for (peer, protocol_version) in peers {
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Full,
|
||||
protocol_version.into(),
|
||||
None
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerViewChange(peer, view![hash_a, hash_b]),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
assert!(state.peer_data.contains_key(&peer));
|
||||
}
|
||||
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_a,
|
||||
msg.clone().into_network_message(ValidationVersion::VStaging.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
// gossip to the overseer
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::Provisioner(ProvisionerMessage::ProvisionableData(
|
||||
_,
|
||||
ProvisionableData::Bitfield(hash, signed)
|
||||
)) => {
|
||||
assert_eq!(hash, hash_a);
|
||||
assert_eq!(signed, signed_bitfield)
|
||||
}
|
||||
);
|
||||
|
||||
// v1 gossip
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendValidationMessage(peers, send_msg),
|
||||
) => {
|
||||
assert_eq!(peers, vec![peer_b]);
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
// vstaging gossip
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendValidationMessage(peers, send_msg),
|
||||
) => {
|
||||
assert_eq!(peers, vec![peer_c]);
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::VStaging.into()));
|
||||
}
|
||||
);
|
||||
|
||||
// reputation change
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer, rep))
|
||||
) => {
|
||||
assert_eq!(peer, peer_a);
|
||||
assert_eq!(rep, BENEFIT_VALID_MESSAGE_FIRST.into())
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -55,6 +55,9 @@ pub(crate) fn send_message<M>(
|
||||
) where
|
||||
M: Encode + Clone,
|
||||
{
|
||||
if peers.is_empty() {
|
||||
return
|
||||
}
|
||||
let message = {
|
||||
let encoded = message.encode();
|
||||
metrics.on_notification_sent(peer_set, version, encoded.len(), peers.len());
|
||||
@@ -65,8 +68,11 @@ pub(crate) fn send_message<M>(
|
||||
// list. The message payload can be quite large. If the underlying
|
||||
// network used `Bytes` this would not be necessary.
|
||||
let last_peer = peers.pop();
|
||||
// optimization: generate the protocol name once.
|
||||
let protocol_name = protocol_names.get_name(peer_set, version);
|
||||
|
||||
// We always send messages on the "main" name even when a negotiated
|
||||
// fallback is used. The libp2p implementation handles the fallback
|
||||
// under the hood.
|
||||
let protocol_name = protocol_names.get_main_name(peer_set);
|
||||
peers.into_iter().for_each(|peer| {
|
||||
net.write_notification(peer, protocol_name.clone(), message.clone());
|
||||
});
|
||||
|
||||
@@ -33,7 +33,8 @@ use polkadot_node_network_protocol::{
|
||||
CollationVersion, PeerSet, PeerSetProtocolNames, PerPeerSet, ProtocolVersion,
|
||||
ValidationVersion,
|
||||
},
|
||||
v1 as protocol_v1, ObservedRole, OurView, PeerId, UnifiedReputationChange as Rep, View,
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, ObservedRole, OurView, PeerId,
|
||||
UnifiedReputationChange as Rep, View,
|
||||
};
|
||||
|
||||
use polkadot_node_subsystem::{
|
||||
@@ -246,15 +247,32 @@ where
|
||||
)
|
||||
.await;
|
||||
|
||||
send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Validation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::ValidationProtocol>::ViewUpdate(local_view),
|
||||
&metrics,
|
||||
);
|
||||
match ValidationVersion::try_from(version)
|
||||
.expect("try_get_protocol has already checked version is known; qed")
|
||||
{
|
||||
ValidationVersion::V1 => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Validation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::ValidationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
ValidationVersion::VStaging => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Validation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_vstaging::ValidationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
PeerSet::Collation => {
|
||||
dispatch_collation_events_to_all(
|
||||
@@ -271,15 +289,32 @@ where
|
||||
)
|
||||
.await;
|
||||
|
||||
send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Collation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::CollationProtocol>::ViewUpdate(local_view),
|
||||
&metrics,
|
||||
);
|
||||
match CollationVersion::try_from(version)
|
||||
.expect("try_get_protocol has already checked version is known; qed")
|
||||
{
|
||||
CollationVersion::V1 => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Collation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::CollationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
CollationVersion::VStaging => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Collation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_vstaging::CollationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
}
|
||||
},
|
||||
@@ -417,30 +452,39 @@ where
|
||||
);
|
||||
|
||||
if !v_messages.is_empty() {
|
||||
let (events, reports) =
|
||||
if expected_versions[PeerSet::Validation] ==
|
||||
Some(ValidationVersion::V1.into())
|
||||
{
|
||||
handle_v1_peer_messages::<protocol_v1::ValidationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Validation,
|
||||
&mut shared.0.lock().validation_peers,
|
||||
v_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Validation],
|
||||
"Major logic bug. Peer somehow has unsupported validation protocol version."
|
||||
);
|
||||
let (events, reports) = if expected_versions[PeerSet::Validation] ==
|
||||
Some(ValidationVersion::V1.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_v1::ValidationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Validation,
|
||||
&mut shared.0.lock().validation_peers,
|
||||
v_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else if expected_versions[PeerSet::Validation] ==
|
||||
Some(ValidationVersion::VStaging.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_vstaging::ValidationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Validation,
|
||||
&mut shared.0.lock().validation_peers,
|
||||
v_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Validation],
|
||||
"Major logic bug. Peer somehow has unsupported validation protocol version."
|
||||
);
|
||||
|
||||
never!("Only version 1 is supported; peer set connection checked above; qed");
|
||||
never!("Only versions 1 and 2 are supported; peer set connection checked above; qed");
|
||||
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
|
||||
for report in reports {
|
||||
network_service.report_peer(remote, report.into());
|
||||
@@ -450,30 +494,39 @@ where
|
||||
}
|
||||
|
||||
if !c_messages.is_empty() {
|
||||
let (events, reports) =
|
||||
if expected_versions[PeerSet::Collation] ==
|
||||
Some(CollationVersion::V1.into())
|
||||
{
|
||||
handle_v1_peer_messages::<protocol_v1::CollationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Collation,
|
||||
&mut shared.0.lock().collation_peers,
|
||||
c_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Collation],
|
||||
"Major logic bug. Peer somehow has unsupported collation protocol version."
|
||||
);
|
||||
let (events, reports) = if expected_versions[PeerSet::Collation] ==
|
||||
Some(CollationVersion::V1.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_v1::CollationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Collation,
|
||||
&mut shared.0.lock().collation_peers,
|
||||
c_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else if expected_versions[PeerSet::Collation] ==
|
||||
Some(CollationVersion::VStaging.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_vstaging::CollationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Collation,
|
||||
&mut shared.0.lock().collation_peers,
|
||||
c_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Collation],
|
||||
"Major logic bug. Peer somehow has unsupported collation protocol version."
|
||||
);
|
||||
|
||||
never!("Only version 1 is supported; peer set connection checked above; qed");
|
||||
never!("Only versions 1 and 2 are supported; peer set connection checked above; qed");
|
||||
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
|
||||
for report in reports {
|
||||
network_service.report_peer(remote, report.into());
|
||||
@@ -738,14 +791,34 @@ fn update_our_view<Net, Context>(
|
||||
}
|
||||
|
||||
(
|
||||
shared.validation_peers.keys().cloned().collect::<Vec<_>>(),
|
||||
shared.collation_peers.keys().cloned().collect::<Vec<_>>(),
|
||||
shared
|
||||
.validation_peers
|
||||
.iter()
|
||||
.map(|(peer_id, data)| (*peer_id, data.version))
|
||||
.collect::<Vec<_>>(),
|
||||
shared
|
||||
.collation_peers
|
||||
.iter()
|
||||
.map(|(peer_id, data)| (*peer_id, data.version))
|
||||
.collect::<Vec<_>>(),
|
||||
)
|
||||
};
|
||||
|
||||
let filter_by_version = |peers: &[(PeerId, ProtocolVersion)], version| {
|
||||
peers.iter().filter(|(_, v)| v == &version).map(|(p, _)| *p).collect::<Vec<_>>()
|
||||
};
|
||||
|
||||
let v1_validation_peers = filter_by_version(&validation_peers, ValidationVersion::V1.into());
|
||||
let v1_collation_peers = filter_by_version(&collation_peers, CollationVersion::V1.into());
|
||||
|
||||
let vstaging_validation_peers =
|
||||
filter_by_version(&validation_peers, ValidationVersion::VStaging.into());
|
||||
let vstaging_collation_peers =
|
||||
filter_by_version(&collation_peers, ValidationVersion::VStaging.into());
|
||||
|
||||
send_validation_message_v1(
|
||||
net,
|
||||
validation_peers,
|
||||
v1_validation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view.clone()),
|
||||
metrics,
|
||||
@@ -753,7 +826,23 @@ fn update_our_view<Net, Context>(
|
||||
|
||||
send_collation_message_v1(
|
||||
net,
|
||||
collation_peers,
|
||||
v1_collation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view.clone()),
|
||||
metrics,
|
||||
);
|
||||
|
||||
send_validation_message_vstaging(
|
||||
net,
|
||||
vstaging_validation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view.clone()),
|
||||
metrics,
|
||||
);
|
||||
|
||||
send_collation_message_vstaging(
|
||||
net,
|
||||
vstaging_collation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view),
|
||||
metrics,
|
||||
@@ -777,7 +866,7 @@ fn update_our_view<Net, Context>(
|
||||
|
||||
// Handle messages on a specific v1 peer-set. The peer is expected to be connected on that
|
||||
// peer-set.
|
||||
fn handle_v1_peer_messages<RawMessage: Decode, OutMessage: From<RawMessage>>(
|
||||
fn handle_peer_messages<RawMessage: Decode, OutMessage: From<RawMessage>>(
|
||||
peer: PeerId,
|
||||
peer_set: PeerSet,
|
||||
peers: &mut HashMap<PeerId, PeerData>,
|
||||
@@ -864,6 +953,42 @@ fn send_collation_message_v1(
|
||||
);
|
||||
}
|
||||
|
||||
fn send_validation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::ValidationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Validation,
|
||||
ValidationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
fn send_collation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::CollationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Collation,
|
||||
CollationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
async fn dispatch_validation_event_to_all(
|
||||
event: NetworkBridgeEvent<net_protocol::VersionedValidationProtocol>,
|
||||
ctx: &mut impl overseer::NetworkBridgeRxSenderTrait,
|
||||
|
||||
@@ -25,6 +25,7 @@ use parking_lot::Mutex;
|
||||
use std::{
|
||||
collections::HashSet,
|
||||
sync::atomic::{AtomicBool, Ordering},
|
||||
task::Poll,
|
||||
};
|
||||
|
||||
use sc_network::{Event as NetworkEvent, IfDisconnected, ProtocolName, ReputationChange};
|
||||
@@ -46,7 +47,7 @@ use polkadot_node_subsystem_test_helpers::{
|
||||
SingleItemSink, SingleItemStream, TestSubsystemContextHandle,
|
||||
};
|
||||
use polkadot_node_subsystem_util::metered;
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, Hash};
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, CandidateHash, Hash};
|
||||
|
||||
use sc_network::Multiaddr;
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
@@ -142,8 +143,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn disconnect_peer(&self, who: PeerId, protocol: ProtocolName) {
|
||||
let (peer_set, version) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -152,8 +152,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn write_notification(&self, who: PeerId, protocol: ProtocolName, message: Vec<u8>) {
|
||||
let (peer_set, version) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -195,10 +194,17 @@ impl TestNetworkHandle {
|
||||
v
|
||||
}
|
||||
|
||||
async fn connect_peer(&mut self, peer: PeerId, peer_set: PeerSet, role: ObservedRole) {
|
||||
async fn connect_peer(
|
||||
&mut self,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
peer_set: PeerSet,
|
||||
role: ObservedRole,
|
||||
) {
|
||||
let protocol_version = ProtocolVersion::from(protocol_version);
|
||||
self.send_network_event(NetworkEvent::NotificationStreamOpened {
|
||||
remote: peer,
|
||||
protocol: self.protocol_names.get_main_name(peer_set),
|
||||
protocol: self.protocol_names.get_name(peer_set, protocol_version),
|
||||
negotiated_fallback: None,
|
||||
role: role.into(),
|
||||
received_handshake: vec![],
|
||||
@@ -432,8 +438,12 @@ fn send_our_view_upon_connection() {
|
||||
|
||||
handle.await_mode_switch().await;
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle.connect_peer(peer, PeerSet::Collation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
|
||||
@@ -478,10 +488,10 @@ fn sends_view_updates_to_peers() {
|
||||
handle.await_mode_switch().await;
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
@@ -541,10 +551,10 @@ fn do_not_send_view_update_until_synced() {
|
||||
assert_ne!(peer_a, peer_b);
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
@@ -636,10 +646,10 @@ fn do_not_send_view_update_when_only_finalized_block_changed() {
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 2, 0).await;
|
||||
@@ -696,7 +706,9 @@ fn peer_view_updates_sent_via_overseer() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
|
||||
@@ -746,7 +758,9 @@ fn peer_messages_sent_via_overseer() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
|
||||
@@ -819,8 +833,12 @@ fn peer_disconnect_from_just_one_peerset() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle.connect_peer(peer, PeerSet::Collation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
|
||||
@@ -907,10 +925,10 @@ fn relays_collation_protocol_messages() {
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
@@ -1011,8 +1029,12 @@ fn different_views_on_different_peer_sets() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle.connect_peer(peer, PeerSet::Collation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
|
||||
@@ -1097,7 +1119,7 @@ fn sent_views_include_finalized_number_update() {
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
@@ -1140,7 +1162,7 @@ fn view_finalized_number_can_not_go_down() {
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
@@ -1225,3 +1247,164 @@ fn our_view_updates_decreasing_order_and_limited_to_max() {
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning_view_update() {
|
||||
let (oracle, handle) = make_sync_oracle(false);
|
||||
test_harness(Box::new(oracle), |test_harness| async move {
|
||||
let TestHarness { mut network_handle, mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let peer_ids: Vec<_> = (0..4).map(|_| PeerId::random()).collect();
|
||||
let peers = [
|
||||
(peer_ids[0], PeerSet::Validation, ValidationVersion::VStaging),
|
||||
(peer_ids[1], PeerSet::Collation, ValidationVersion::V1),
|
||||
(peer_ids[2], PeerSet::Validation, ValidationVersion::V1),
|
||||
(peer_ids[3], PeerSet::Collation, ValidationVersion::VStaging),
|
||||
];
|
||||
|
||||
let head = Hash::repeat_byte(1);
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Signal(OverseerSignal::ActiveLeaves(
|
||||
ActiveLeavesUpdate::start_work(ActivatedLeaf {
|
||||
hash: head,
|
||||
number: 1,
|
||||
status: LeafStatus::Fresh,
|
||||
span: Arc::new(jaeger::Span::Disabled),
|
||||
}),
|
||||
)))
|
||||
.await;
|
||||
|
||||
handle.await_mode_switch().await;
|
||||
|
||||
for &(peer_id, peer_set, version) in &peers {
|
||||
network_handle
|
||||
.connect_peer(peer_id, version, peer_set, ObservedRole::Full)
|
||||
.await;
|
||||
}
|
||||
|
||||
let view = view![head];
|
||||
let actions = network_handle.next_network_actions(4).await;
|
||||
|
||||
for &(peer_id, peer_set, version) in &peers {
|
||||
let wire_msg = match version {
|
||||
ValidationVersion::V1 =>
|
||||
WireMessage::<protocol_v1::ValidationProtocol>::ViewUpdate(view.clone())
|
||||
.encode(),
|
||||
ValidationVersion::VStaging =>
|
||||
WireMessage::<protocol_vstaging::ValidationProtocol>::ViewUpdate(view.clone())
|
||||
.encode(),
|
||||
};
|
||||
assert_network_actions_contains(
|
||||
&actions,
|
||||
&NetworkAction::WriteNotification(peer_id, peer_set, wire_msg),
|
||||
);
|
||||
}
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning_subsystem_msg() {
|
||||
let (oracle, _handle) = make_sync_oracle(false);
|
||||
test_harness(Box::new(oracle), |test_harness| async move {
|
||||
let TestHarness { mut network_handle, mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(
|
||||
peer,
|
||||
ValidationVersion::VStaging,
|
||||
PeerSet::Validation,
|
||||
ObservedRole::Full,
|
||||
)
|
||||
.await;
|
||||
|
||||
// bridge will inform about all connected peers.
|
||||
{
|
||||
assert_sends_validation_event_to_all(
|
||||
NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Full,
|
||||
ValidationVersion::VStaging.into(),
|
||||
None,
|
||||
),
|
||||
&mut virtual_overseer,
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_sends_validation_event_to_all(
|
||||
NetworkBridgeEvent::PeerViewChange(peer, View::default()),
|
||||
&mut virtual_overseer,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let approval_distribution_message =
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(Vec::new());
|
||||
|
||||
let msg = protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
approval_distribution_message.clone(),
|
||||
);
|
||||
|
||||
network_handle
|
||||
.peer_message(
|
||||
peer,
|
||||
PeerSet::Validation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ApprovalDistribution(
|
||||
ApprovalDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(p, Versioned::VStaging(m))
|
||||
)
|
||||
) => {
|
||||
assert_eq!(p, peer);
|
||||
assert_eq!(m, approval_distribution_message);
|
||||
}
|
||||
);
|
||||
|
||||
let metadata = protocol_v1::StatementMetadata {
|
||||
relay_parent: Hash::zero(),
|
||||
candidate_hash: CandidateHash::default(),
|
||||
signed_by: ValidatorIndex(0),
|
||||
signature: sp_core::crypto::UncheckedFrom::unchecked_from([1u8; 64]),
|
||||
};
|
||||
let statement_distribution_message =
|
||||
protocol_vstaging::StatementDistributionMessage::V1Compatibility(
|
||||
protocol_v1::StatementDistributionMessage::LargeStatement(metadata),
|
||||
);
|
||||
let msg = protocol_vstaging::ValidationProtocol::StatementDistribution(
|
||||
statement_distribution_message.clone(),
|
||||
);
|
||||
|
||||
network_handle
|
||||
.peer_message(
|
||||
peer,
|
||||
PeerSet::Validation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::StatementDistribution(
|
||||
StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(p, Versioned::VStaging(m))
|
||||
)
|
||||
) => {
|
||||
assert_eq!(p, peer);
|
||||
assert_eq!(m, statement_distribution_message);
|
||||
}
|
||||
);
|
||||
|
||||
// No more messages.
|
||||
assert_matches!(futures::poll!(virtual_overseer.recv().boxed()), Poll::Pending);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ use super::*;
|
||||
use polkadot_node_network_protocol::{
|
||||
peer_set::{CollationVersion, PeerSet, PeerSetProtocolNames, ValidationVersion},
|
||||
request_response::ReqProtocolNames,
|
||||
v1 as protocol_v1, PeerId, Versioned,
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, PeerId, Versioned,
|
||||
};
|
||||
|
||||
use polkadot_node_subsystem::{
|
||||
@@ -197,6 +197,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_validation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
NetworkBridgeTxMessage::SendValidationMessages(msgs) => {
|
||||
@@ -215,6 +222,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_validation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -233,6 +247,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_collation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
NetworkBridgeTxMessage::SendCollationMessages(msgs) => {
|
||||
@@ -251,6 +272,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_collation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -381,3 +409,39 @@ fn send_collation_message_v1(
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
fn send_validation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::ValidationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Validation,
|
||||
ValidationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
fn send_collation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::CollationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Collation,
|
||||
CollationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -130,8 +130,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn disconnect_peer(&self, who: PeerId, protocol: ProtocolName) {
|
||||
let (peer_set, version) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -140,8 +139,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn write_notification(&self, who: PeerId, protocol: ProtocolName, message: Vec<u8>) {
|
||||
let (peer_set, version) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -173,10 +171,17 @@ impl TestNetworkHandle {
|
||||
self.action_rx.next().await.expect("subsystem concluded early")
|
||||
}
|
||||
|
||||
async fn connect_peer(&mut self, peer: PeerId, peer_set: PeerSet, role: ObservedRole) {
|
||||
async fn connect_peer(
|
||||
&mut self,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
peer_set: PeerSet,
|
||||
role: ObservedRole,
|
||||
) {
|
||||
let protocol_version = ProtocolVersion::from(protocol_version);
|
||||
self.send_network_event(NetworkEvent::NotificationStreamOpened {
|
||||
remote: peer,
|
||||
protocol: self.peerset_protocol_names.get_main_name(peer_set),
|
||||
protocol: self.peerset_protocol_names.get_name(peer_set, protocol_version),
|
||||
negotiated_fallback: None,
|
||||
role: role.into(),
|
||||
received_handshake: vec![],
|
||||
@@ -242,7 +247,7 @@ fn send_messages_to_peers() {
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
@@ -251,7 +256,7 @@ fn send_messages_to_peers() {
|
||||
// so the single item sink has to be free explicitly
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
@@ -328,3 +333,107 @@ fn send_messages_to_peers() {
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning_send() {
|
||||
test_harness(|test_harness| async move {
|
||||
let TestHarness { mut network_handle, mut virtual_overseer } = test_harness;
|
||||
|
||||
let peer_ids: Vec<_> = (0..4).map(|_| PeerId::random()).collect();
|
||||
let peers = [
|
||||
(peer_ids[0], PeerSet::Validation, ValidationVersion::VStaging),
|
||||
(peer_ids[1], PeerSet::Collation, ValidationVersion::V1),
|
||||
(peer_ids[2], PeerSet::Validation, ValidationVersion::V1),
|
||||
(peer_ids[3], PeerSet::Collation, ValidationVersion::VStaging),
|
||||
];
|
||||
|
||||
for &(peer_id, peer_set, version) in &peers {
|
||||
network_handle
|
||||
.connect_peer(peer_id, version, peer_set, ObservedRole::Full)
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
}
|
||||
|
||||
// send a validation protocol message.
|
||||
|
||||
{
|
||||
let approval_distribution_message =
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(Vec::new());
|
||||
|
||||
let msg = protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
approval_distribution_message.clone(),
|
||||
);
|
||||
|
||||
// Note that bridge doesn't ensure neither peer's protocol version
|
||||
// or peer set match the message.
|
||||
let receivers = vec![peer_ids[0], peer_ids[3]];
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: NetworkBridgeTxMessage::SendValidationMessage(
|
||||
receivers.clone(),
|
||||
Versioned::VStaging(msg.clone()),
|
||||
),
|
||||
})
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
|
||||
for peer in &receivers {
|
||||
assert_eq!(
|
||||
network_handle
|
||||
.next_network_action()
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur"),
|
||||
NetworkAction::WriteNotification(
|
||||
*peer,
|
||||
PeerSet::Validation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// send a collation protocol message.
|
||||
|
||||
{
|
||||
let collator_protocol_message = protocol_vstaging::CollatorProtocolMessage::Declare(
|
||||
Sr25519Keyring::Alice.public().into(),
|
||||
0_u32.into(),
|
||||
dummy_collator_signature(),
|
||||
);
|
||||
|
||||
let msg = protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
collator_protocol_message.clone(),
|
||||
);
|
||||
|
||||
let receivers = vec![peer_ids[1], peer_ids[2]];
|
||||
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: NetworkBridgeTxMessage::SendCollationMessages(vec![(
|
||||
receivers.clone(),
|
||||
Versioned::VStaging(msg.clone()),
|
||||
)]),
|
||||
})
|
||||
.await;
|
||||
|
||||
for peer in &receivers {
|
||||
assert_eq!(
|
||||
network_handle
|
||||
.next_network_action()
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur"),
|
||||
NetworkAction::WriteNotification(
|
||||
*peer,
|
||||
PeerSet::Collation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
@@ -6,7 +6,6 @@ edition.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
[dependencies]
|
||||
always-assert = "0.1.2"
|
||||
bitvec = { version = "1.0.1", default-features = false, features = ["alloc"] }
|
||||
futures = "0.3.21"
|
||||
futures-timer = "3"
|
||||
@@ -23,6 +22,7 @@ polkadot-node-subsystem-util = { path = "../../subsystem-util" }
|
||||
polkadot-node-subsystem = {path = "../../subsystem" }
|
||||
fatality = "0.0.6"
|
||||
thiserror = "1.0.31"
|
||||
tokio-util = "0.7.1"
|
||||
|
||||
[dev-dependencies]
|
||||
log = "0.4.17"
|
||||
@@ -31,6 +31,7 @@ assert_matches = "1.4.0"
|
||||
|
||||
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master", features = ["std"] }
|
||||
sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sc-keystore = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sc-network = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
parity-scale-codec = { version = "3.6.1", features = ["std"] }
|
||||
|
||||
|
||||
@@ -0,0 +1,162 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Primitives for tracking collations-related data.
|
||||
|
||||
use std::collections::{HashSet, VecDeque};
|
||||
|
||||
use futures::{future::BoxFuture, stream::FuturesUnordered};
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
request_response::{
|
||||
incoming::OutgoingResponse, v1 as protocol_v1, vstaging as protocol_vstaging,
|
||||
IncomingRequest,
|
||||
},
|
||||
PeerId,
|
||||
};
|
||||
use polkadot_node_primitives::PoV;
|
||||
use polkadot_primitives::{CandidateHash, CandidateReceipt, Hash, Id as ParaId};
|
||||
|
||||
/// The status of a collation as seen from the collator.
|
||||
pub enum CollationStatus {
|
||||
/// The collation was created, but we did not advertise it to any validator.
|
||||
Created,
|
||||
/// The collation was advertised to at least one validator.
|
||||
Advertised,
|
||||
/// The collation was requested by at least one validator.
|
||||
Requested,
|
||||
}
|
||||
|
||||
impl CollationStatus {
|
||||
/// Advance to the [`Self::Advertised`] status.
|
||||
///
|
||||
/// This ensures that `self` isn't already [`Self::Requested`].
|
||||
pub fn advance_to_advertised(&mut self) {
|
||||
if !matches!(self, Self::Requested) {
|
||||
*self = Self::Advertised;
|
||||
}
|
||||
}
|
||||
|
||||
/// Advance to the [`Self::Requested`] status.
|
||||
pub fn advance_to_requested(&mut self) {
|
||||
*self = Self::Requested;
|
||||
}
|
||||
}
|
||||
|
||||
/// A collation built by the collator.
|
||||
pub struct Collation {
|
||||
/// Candidate receipt.
|
||||
pub receipt: CandidateReceipt,
|
||||
/// Parent head-data hash.
|
||||
pub parent_head_data_hash: Hash,
|
||||
/// Proof to verify the state transition of the parachain.
|
||||
pub pov: PoV,
|
||||
/// Collation status.
|
||||
pub status: CollationStatus,
|
||||
}
|
||||
|
||||
/// Stores the state for waiting collation fetches per relay parent.
|
||||
#[derive(Default)]
|
||||
pub struct WaitingCollationFetches {
|
||||
/// A flag indicating that we have an ongoing request.
|
||||
/// This limits the number of collations being sent at any moment
|
||||
/// of time to 1 for each relay parent.
|
||||
///
|
||||
/// If set to `true`, any new request will be queued.
|
||||
pub collation_fetch_active: bool,
|
||||
/// The collation fetches waiting to be fulfilled.
|
||||
pub req_queue: VecDeque<VersionedCollationRequest>,
|
||||
/// All peers that are waiting or actively uploading.
|
||||
///
|
||||
/// We will not accept multiple requests from the same peer, otherwise our DoS protection of
|
||||
/// moving on to the next peer after `MAX_UNSHARED_UPLOAD_TIME` would be pointless.
|
||||
pub waiting_peers: HashSet<(PeerId, CandidateHash)>,
|
||||
}
|
||||
|
||||
/// Backwards-compatible wrapper for incoming collations requests.
|
||||
pub enum VersionedCollationRequest {
|
||||
V1(IncomingRequest<protocol_v1::CollationFetchingRequest>),
|
||||
VStaging(IncomingRequest<protocol_vstaging::CollationFetchingRequest>),
|
||||
}
|
||||
|
||||
impl From<IncomingRequest<protocol_v1::CollationFetchingRequest>> for VersionedCollationRequest {
|
||||
fn from(req: IncomingRequest<protocol_v1::CollationFetchingRequest>) -> Self {
|
||||
Self::V1(req)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<IncomingRequest<protocol_vstaging::CollationFetchingRequest>>
|
||||
for VersionedCollationRequest
|
||||
{
|
||||
fn from(req: IncomingRequest<protocol_vstaging::CollationFetchingRequest>) -> Self {
|
||||
Self::VStaging(req)
|
||||
}
|
||||
}
|
||||
|
||||
impl VersionedCollationRequest {
|
||||
/// Returns parachain id from the request payload.
|
||||
pub fn para_id(&self) -> ParaId {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.payload.para_id,
|
||||
VersionedCollationRequest::VStaging(req) => req.payload.para_id,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns relay parent from the request payload.
|
||||
pub fn relay_parent(&self) -> Hash {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.payload.relay_parent,
|
||||
VersionedCollationRequest::VStaging(req) => req.payload.relay_parent,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns id of the peer the request was received from.
|
||||
pub fn peer_id(&self) -> PeerId {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.peer,
|
||||
VersionedCollationRequest::VStaging(req) => req.peer,
|
||||
}
|
||||
}
|
||||
|
||||
/// Sends the response back to requester.
|
||||
pub fn send_outgoing_response(
|
||||
self,
|
||||
response: OutgoingResponse<protocol_v1::CollationFetchingResponse>,
|
||||
) -> Result<(), ()> {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.send_outgoing_response(response),
|
||||
VersionedCollationRequest::VStaging(req) => req.send_outgoing_response(response),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Result of the finished background send-collation task.
|
||||
///
|
||||
/// Note that if the timeout was hit the request doesn't get
|
||||
/// aborted, it only indicates that we should start processing
|
||||
/// the next one from the queue.
|
||||
pub struct CollationSendResult {
|
||||
/// Candidate's relay parent.
|
||||
pub relay_parent: Hash,
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Peer id.
|
||||
pub peer_id: PeerId,
|
||||
/// Whether the max unshared timeout was hit.
|
||||
pub timed_out: bool,
|
||||
}
|
||||
|
||||
pub type ActiveCollationFetches = FuturesUnordered<BoxFuture<'static, CollationSendResult>>;
|
||||
@@ -20,7 +20,7 @@ use polkadot_node_subsystem_util::metrics::{self, prometheus};
|
||||
pub struct Metrics(Option<MetricsInner>);
|
||||
|
||||
impl Metrics {
|
||||
pub fn on_advertisment_made(&self) {
|
||||
pub fn on_advertisement_made(&self) {
|
||||
if let Some(metrics) = &self.0 {
|
||||
metrics.advertisements_made.inc();
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+366
-113
@@ -37,6 +37,7 @@ use polkadot_node_network_protocol::{
|
||||
};
|
||||
use polkadot_node_primitives::BlockData;
|
||||
use polkadot_node_subsystem::{
|
||||
errors::RuntimeApiError,
|
||||
jaeger,
|
||||
messages::{AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest},
|
||||
ActivatedLeaf, ActiveLeavesUpdate, LeafStatus,
|
||||
@@ -49,8 +50,13 @@ use polkadot_primitives::{
|
||||
};
|
||||
use polkadot_primitives_test_helpers::TestCandidateBuilder;
|
||||
|
||||
mod prospective_parachains;
|
||||
|
||||
const REPUTATION_CHANGE_TEST_INTERVAL: Duration = Duration::from_millis(10);
|
||||
|
||||
const ASYNC_BACKING_DISABLED_ERROR: RuntimeApiError =
|
||||
RuntimeApiError::NotSupported { runtime_api_name: "test-runtime" };
|
||||
|
||||
#[derive(Clone)]
|
||||
struct TestState {
|
||||
para_id: ParaId,
|
||||
@@ -186,6 +192,17 @@ impl TestState {
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, self.relay_parent);
|
||||
tx.send(Err(ASYNC_BACKING_DISABLED_ERROR)).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -193,7 +210,8 @@ type VirtualOverseer = test_helpers::TestSubsystemContextHandle<CollatorProtocol
|
||||
|
||||
struct TestHarness {
|
||||
virtual_overseer: VirtualOverseer,
|
||||
req_cfg: sc_network::config::RequestResponseConfig,
|
||||
req_v1_cfg: sc_network::config::RequestResponseConfig,
|
||||
req_vstaging_cfg: sc_network::config::RequestResponseConfig,
|
||||
}
|
||||
|
||||
fn test_harness<T: Future<Output = TestHarness>>(
|
||||
@@ -215,7 +233,9 @@ fn test_harness<T: Future<Output = TestHarness>>(
|
||||
let genesis_hash = Hash::repeat_byte(0xff);
|
||||
let req_protocol_names = ReqProtocolNames::new(&genesis_hash, None);
|
||||
|
||||
let (collation_req_receiver, req_cfg) =
|
||||
let (collation_req_receiver, req_v1_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (collation_req_vstaging_receiver, req_vstaging_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let subsystem = async {
|
||||
run_inner(
|
||||
@@ -223,6 +243,7 @@ fn test_harness<T: Future<Output = TestHarness>>(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
collation_req_receiver,
|
||||
collation_req_vstaging_receiver,
|
||||
Default::default(),
|
||||
reputation,
|
||||
REPUTATION_CHANGE_TEST_INTERVAL,
|
||||
@@ -231,7 +252,7 @@ fn test_harness<T: Future<Output = TestHarness>>(
|
||||
.unwrap();
|
||||
};
|
||||
|
||||
let test_fut = test(TestHarness { virtual_overseer, req_cfg });
|
||||
let test_fut = test(TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg });
|
||||
|
||||
futures::pin_mut!(test_fut);
|
||||
futures::pin_mut!(subsystem);
|
||||
@@ -305,6 +326,17 @@ async fn setup_system(virtual_overseer: &mut VirtualOverseer, test_state: &TestS
|
||||
])),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
tx.send(Err(ASYNC_BACKING_DISABLED_ERROR)).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/// Result of [`distribute_collation`]
|
||||
@@ -313,29 +345,23 @@ struct DistributeCollation {
|
||||
pov_block: PoV,
|
||||
}
|
||||
|
||||
/// Create some PoV and distribute it.
|
||||
async fn distribute_collation(
|
||||
async fn distribute_collation_with_receipt(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
// whether or not we expect a connection request or not.
|
||||
relay_parent: Hash,
|
||||
should_connect: bool,
|
||||
candidate: CandidateReceipt,
|
||||
pov: PoV,
|
||||
parent_head_data_hash: Hash,
|
||||
) -> DistributeCollation {
|
||||
// Now we want to distribute a `PoVBlock`
|
||||
let pov_block = PoV { block_data: BlockData(vec![42, 43, 44]) };
|
||||
|
||||
let pov_hash = pov_block.hash();
|
||||
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: test_state.relay_parent,
|
||||
pov_hash,
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::DistributeCollation(candidate.clone(), pov_block.clone(), None),
|
||||
CollatorProtocolMessage::DistributeCollation(
|
||||
candidate.clone(),
|
||||
parent_head_data_hash,
|
||||
pov.clone(),
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -343,10 +369,10 @@ async fn distribute_collation(
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
_relay_parent,
|
||||
RuntimeApiRequest::AvailabilityCores(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(relay_parent, _relay_parent);
|
||||
tx.send(Ok(test_state.availability_cores.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
@@ -358,7 +384,7 @@ async fn distribute_collation(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::SessionIndexForChild(tx),
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(relay_parent, relay_parent);
|
||||
tx.send(Ok(test_state.current_session_index())).unwrap();
|
||||
},
|
||||
|
||||
@@ -366,17 +392,17 @@ async fn distribute_collation(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::SessionInfo(index, tx),
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(relay_parent, relay_parent);
|
||||
assert_eq!(index, test_state.current_session_index());
|
||||
|
||||
tx.send(Ok(Some(test_state.session_info.clone()))).unwrap();
|
||||
},
|
||||
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
_relay_parent,
|
||||
RuntimeApiRequest::ValidatorGroups(tx),
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(_relay_parent, relay_parent);
|
||||
tx.send(Ok((
|
||||
test_state.session_info.validator_groups.to_vec(),
|
||||
test_state.group_rotation_info.clone(),
|
||||
@@ -400,13 +426,48 @@ async fn distribute_collation(
|
||||
);
|
||||
}
|
||||
|
||||
DistributeCollation { candidate, pov_block }
|
||||
DistributeCollation { candidate, pov_block: pov }
|
||||
}
|
||||
|
||||
/// Create some PoV and distribute it.
|
||||
async fn distribute_collation(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
relay_parent: Hash,
|
||||
// whether or not we expect a connection request or not.
|
||||
should_connect: bool,
|
||||
) -> DistributeCollation {
|
||||
// Now we want to distribute a `PoVBlock`
|
||||
let pov_block = PoV { block_data: BlockData(vec![42, 43, 44]) };
|
||||
|
||||
let pov_hash = pov_block.hash();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent,
|
||||
pov_hash,
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
test_state,
|
||||
relay_parent,
|
||||
should_connect,
|
||||
candidate,
|
||||
pov_block,
|
||||
parent_head_data_hash,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// Connect a peer
|
||||
async fn connect_peer(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
version: CollationVersion,
|
||||
authority_id: Option<AuthorityDiscoveryId>,
|
||||
) {
|
||||
overseer_send(
|
||||
@@ -414,7 +475,7 @@ async fn connect_peer(
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
polkadot_node_network_protocol::ObservedRole::Authority,
|
||||
CollationVersion::V1.into(),
|
||||
version.into(),
|
||||
authority_id.map(|v| HashSet::from([v])),
|
||||
)),
|
||||
)
|
||||
@@ -474,30 +535,65 @@ async fn expect_declare_msg(
|
||||
}
|
||||
|
||||
/// Check that the next received message is a collation advertisement message.
|
||||
///
|
||||
/// Expects vstaging message if `expected_candidate_hashes` is `Some`, v1 otherwise.
|
||||
async fn expect_advertise_collation_msg(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: &PeerId,
|
||||
expected_relay_parent: Hash,
|
||||
expected_candidate_hashes: Option<Vec<CandidateHash>>,
|
||||
) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendCollationMessage(
|
||||
to,
|
||||
Versioned::V1(protocol_v1::CollationProtocol::CollatorProtocol(wire_message)),
|
||||
)
|
||||
) => {
|
||||
assert_eq!(to[0], *peer);
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_v1::CollatorProtocolMessage::AdvertiseCollation(
|
||||
relay_parent,
|
||||
) => {
|
||||
assert_eq!(relay_parent, expected_relay_parent);
|
||||
let mut candidate_hashes: Option<HashSet<_>> =
|
||||
expected_candidate_hashes.map(|hashes| hashes.into_iter().collect());
|
||||
let iter_num = candidate_hashes.as_ref().map(|hashes| hashes.len()).unwrap_or(1);
|
||||
|
||||
for _ in 0..iter_num {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendCollationMessage(
|
||||
to,
|
||||
wire_message,
|
||||
)
|
||||
) => {
|
||||
assert_eq!(to[0], *peer);
|
||||
match (candidate_hashes.as_mut(), wire_message) {
|
||||
(None, Versioned::V1(protocol_v1::CollationProtocol::CollatorProtocol(wire_message))) => {
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_v1::CollatorProtocolMessage::AdvertiseCollation(
|
||||
relay_parent,
|
||||
) => {
|
||||
assert_eq!(relay_parent, expected_relay_parent);
|
||||
}
|
||||
);
|
||||
},
|
||||
(
|
||||
Some(candidate_hashes),
|
||||
Versioned::VStaging(protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
wire_message,
|
||||
)),
|
||||
) => {
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_vstaging::CollatorProtocolMessage::AdvertiseCollation {
|
||||
relay_parent,
|
||||
candidate_hash,
|
||||
..
|
||||
} => {
|
||||
assert_eq!(relay_parent, expected_relay_parent);
|
||||
assert!(candidate_hashes.contains(&candidate_hash));
|
||||
|
||||
// Drop the hash we've already seen.
|
||||
candidate_hashes.remove(&candidate_hash);
|
||||
}
|
||||
);
|
||||
},
|
||||
_ => panic!("Invalid advertisement"),
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Send a message that the given peer's view changed.
|
||||
@@ -528,19 +624,26 @@ fn advertise_and_send_collation() {
|
||||
ReputationAggregator::new(|_| true),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let mut req_cfg = test_harness.req_cfg;
|
||||
let mut req_v1_cfg = test_harness.req_v1_cfg;
|
||||
let req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
setup_system(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let DistributeCollation { candidate, pov_block } =
|
||||
distribute_collation(&mut virtual_overseer, &test_state, true).await;
|
||||
let DistributeCollation { candidate, pov_block } = distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
true,
|
||||
)
|
||||
.await;
|
||||
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(test_state.current_group_validator_peer_ids())
|
||||
{
|
||||
connect_peer(&mut virtual_overseer, peer, Some(val.clone())).await;
|
||||
connect_peer(&mut virtual_overseer, peer, CollationVersion::V1, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
// We declare to the connected validators that we are a collator.
|
||||
@@ -558,18 +661,23 @@ fn advertise_and_send_collation() {
|
||||
|
||||
// The peer is interested in a leaf that we have a collation for;
|
||||
// advertise it.
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
&mut virtual_overseer,
|
||||
&peer,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Request a collation.
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -582,13 +690,13 @@ fn advertise_and_send_collation() {
|
||||
{
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -613,8 +721,8 @@ fn advertise_and_send_collation() {
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(receipt, pov): CollationFetchingResponse
|
||||
= CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
@@ -632,13 +740,13 @@ fn advertise_and_send_collation() {
|
||||
// Re-request a collation.
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: old_relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -652,7 +760,8 @@ fn advertise_and_send_collation() {
|
||||
|
||||
assert!(overseer_recv_with_timeout(&mut virtual_overseer, TIMEOUT).await.is_none());
|
||||
|
||||
distribute_collation(&mut virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(&mut virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
// Send info about peer's view.
|
||||
overseer_send(
|
||||
@@ -664,9 +773,14 @@ fn advertise_and_send_collation() {
|
||||
)
|
||||
.await;
|
||||
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, test_state.relay_parent)
|
||||
.await;
|
||||
TestHarness { virtual_overseer, req_cfg }
|
||||
expect_advertise_collation_msg(
|
||||
&mut virtual_overseer,
|
||||
&peer,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg }
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -683,18 +797,26 @@ fn delay_reputation_change() {
|
||||
ReputationAggregator::new(|_| false),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let mut req_cfg = test_harness.req_cfg;
|
||||
let mut req_v1_cfg = test_harness.req_v1_cfg;
|
||||
let req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
setup_system(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let _ = distribute_collation(&mut virtual_overseer, &test_state, true).await;
|
||||
let _ = distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
true,
|
||||
)
|
||||
.await;
|
||||
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(test_state.current_group_validator_peer_ids())
|
||||
{
|
||||
connect_peer(&mut virtual_overseer, peer, Some(val.clone())).await;
|
||||
connect_peer(&mut virtual_overseer, peer, CollationVersion::V1, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
// We declare to the connected validators that we are a collator.
|
||||
@@ -712,18 +834,23 @@ fn delay_reputation_change() {
|
||||
|
||||
// The peer is interested in a leaf that we have a collation for;
|
||||
// advertise it.
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
&mut virtual_overseer,
|
||||
&peer,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Request a collation.
|
||||
let (pending_response, _rx) = oneshot::channel();
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -736,13 +863,13 @@ fn delay_reputation_change() {
|
||||
{
|
||||
let (pending_response, _rx) = oneshot::channel();
|
||||
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -767,7 +894,90 @@ fn delay_reputation_change() {
|
||||
);
|
||||
}
|
||||
|
||||
TestHarness { virtual_overseer, req_cfg }
|
||||
TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg }
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/// Tests that collator side works with vstaging network protocol
|
||||
/// before async backing is enabled.
|
||||
#[test]
|
||||
fn advertise_collation_vstaging_protocol() {
|
||||
let test_state = TestState::default();
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
let DistributeCollation { candidate, .. } =
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
let validators = test_state.current_group_validator_authority_ids();
|
||||
assert!(validators.len() >= 2);
|
||||
let peer_ids = test_state.current_group_validator_peer_ids();
|
||||
|
||||
// Connect first peer with v1.
|
||||
connect_peer(
|
||||
virtual_overseer,
|
||||
peer_ids[0],
|
||||
CollationVersion::V1,
|
||||
Some(validators[0].clone()),
|
||||
)
|
||||
.await;
|
||||
// The rest with vstaging.
|
||||
for (val, peer) in validators.iter().zip(peer_ids.iter()).skip(1) {
|
||||
connect_peer(
|
||||
virtual_overseer,
|
||||
*peer,
|
||||
CollationVersion::VStaging,
|
||||
Some(val.clone()),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
// Declare messages.
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer_ids[0]).await;
|
||||
for peer_id in peer_ids.iter().skip(1) {
|
||||
prospective_parachains::expect_declare_msg_vstaging(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
&peer_id,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
// Send info about peers view.
|
||||
for peer in peer_ids.iter() {
|
||||
send_peer_view_change(virtual_overseer, peer, vec![test_state.relay_parent]).await;
|
||||
}
|
||||
|
||||
// Versioned advertisements work.
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&peer_ids[0],
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
for peer_id in peer_ids.iter().skip(1) {
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
peer_id,
|
||||
test_state.relay_parent,
|
||||
Some(vec![candidate.hash()]), // This is `Some`, advertisement is vstaging.
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
test_harness
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -814,7 +1024,13 @@ fn collators_declare_to_connected_peers() {
|
||||
setup_system(&mut test_harness.virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(&mut test_harness.virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(
|
||||
&mut test_harness.virtual_overseer,
|
||||
peer,
|
||||
CollationVersion::V1,
|
||||
Some(validator_id),
|
||||
)
|
||||
.await;
|
||||
expect_declare_msg(&mut test_harness.virtual_overseer, &test_state, &peer).await;
|
||||
test_harness
|
||||
},
|
||||
@@ -843,10 +1059,10 @@ fn collations_are_only_advertised_to_validators_with_correct_view() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
|
||||
// Connect the second validator
|
||||
connect_peer(virtual_overseer, peer2, Some(validator_id2)).await;
|
||||
connect_peer(virtual_overseer, peer2, CollationVersion::V1, Some(validator_id2)).await;
|
||||
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer2).await;
|
||||
@@ -854,15 +1070,18 @@ fn collations_are_only_advertised_to_validators_with_correct_view() {
|
||||
// And let it tell us that it is has the same view.
|
||||
send_peer_view_change(virtual_overseer, &peer2, vec![test_state.relay_parent]).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent, None)
|
||||
.await;
|
||||
|
||||
// The other validator announces that it changed its view.
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![test_state.relay_parent]).await;
|
||||
|
||||
// After changing the view we should receive the advertisement
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent, None)
|
||||
.await;
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
@@ -890,15 +1109,16 @@ fn collate_on_two_different_relay_chain_blocks() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
|
||||
// Connect the second validator
|
||||
connect_peer(virtual_overseer, peer2, Some(validator_id2)).await;
|
||||
connect_peer(virtual_overseer, peer2, CollationVersion::V1, Some(validator_id2)).await;
|
||||
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer2).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
let old_relay_parent = test_state.relay_parent;
|
||||
|
||||
@@ -906,14 +1126,16 @@ fn collate_on_two_different_relay_chain_blocks() {
|
||||
// parent are active.
|
||||
test_state.advance_to_new_round(virtual_overseer, true).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![old_relay_parent]).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, old_relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, old_relay_parent, None).await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer2, vec![test_state.relay_parent]).await;
|
||||
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent, None)
|
||||
.await;
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
@@ -938,17 +1160,20 @@ fn validator_reconnect_does_not_advertise_a_second_time() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id.clone())).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id.clone()))
|
||||
.await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![test_state.relay_parent]).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent, None)
|
||||
.await;
|
||||
|
||||
// Disconnect and reconnect directly
|
||||
disconnect_peer(virtual_overseer, peer).await;
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![test_state.relay_parent]).await;
|
||||
@@ -979,7 +1204,7 @@ fn collators_reject_declare_messages() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
overseer_send(
|
||||
@@ -1031,19 +1256,20 @@ where
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
let req_cfg = &mut test_harness.req_cfg;
|
||||
let req_cfg = &mut test_harness.req_v1_cfg;
|
||||
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
let DistributeCollation { candidate, pov_block } =
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(test_state.current_group_validator_peer_ids())
|
||||
{
|
||||
connect_peer(virtual_overseer, peer, Some(val.clone())).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(val.clone())).await;
|
||||
}
|
||||
|
||||
// We declare to the connected validators that we are a collator.
|
||||
@@ -1064,10 +1290,20 @@ where
|
||||
|
||||
// The peer is interested in a leaf that we have a collation for;
|
||||
// advertise it.
|
||||
expect_advertise_collation_msg(virtual_overseer, &validator_0, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &validator_1, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&validator_0,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&validator_1,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Request a collation.
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
@@ -1077,7 +1313,7 @@ where
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer: validator_0,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -1092,8 +1328,8 @@ where
|
||||
let feedback_tx = assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(receipt, pov): CollationFetchingResponse
|
||||
= CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
@@ -1113,7 +1349,7 @@ where
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer: validator_1,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -1129,8 +1365,8 @@ where
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(receipt, pov): CollationFetchingResponse
|
||||
= CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
@@ -1159,7 +1395,8 @@ fn connect_to_buffered_groups() {
|
||||
ReputationAggregator::new(|_| true),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let mut req_cfg = test_harness.req_cfg;
|
||||
let mut req_cfg = test_harness.req_v1_cfg;
|
||||
let req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
setup_system(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
@@ -1167,7 +1404,13 @@ fn connect_to_buffered_groups() {
|
||||
let peers_a = test_state.current_group_validator_peer_ids();
|
||||
assert!(group_a.len() > 1);
|
||||
|
||||
distribute_collation(&mut virtual_overseer, &test_state, false).await;
|
||||
distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
false,
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
@@ -1181,7 +1424,8 @@ fn connect_to_buffered_groups() {
|
||||
let head_a = test_state.relay_parent;
|
||||
|
||||
for (val, peer) in group_a.iter().zip(&peers_a) {
|
||||
connect_peer(&mut virtual_overseer, *peer, Some(val.clone())).await;
|
||||
connect_peer(&mut virtual_overseer, *peer, CollationVersion::V1, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
for peer_id in &peers_a {
|
||||
@@ -1191,7 +1435,7 @@ fn connect_to_buffered_groups() {
|
||||
// Update views.
|
||||
for peed_id in &peers_a {
|
||||
send_peer_view_change(&mut virtual_overseer, peed_id, vec![head_a]).await;
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, peed_id, head_a).await;
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, peed_id, head_a, None).await;
|
||||
}
|
||||
|
||||
let peer = peers_a[0];
|
||||
@@ -1203,7 +1447,7 @@ fn connect_to_buffered_groups() {
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: head_a,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -1215,14 +1459,17 @@ fn connect_to_buffered_groups() {
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(..): CollationFetchingResponse =
|
||||
CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(..) =
|
||||
request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result.expect("We should have a proper answer").as_ref(),
|
||||
)
|
||||
.expect("Decoding should work");
|
||||
}
|
||||
);
|
||||
|
||||
// Let the subsystem process process the collation event.
|
||||
test_helpers::Yield::new().await;
|
||||
|
||||
test_state.advance_to_new_round(&mut virtual_overseer, true).await;
|
||||
test_state.group_rotation_info = test_state.group_rotation_info.bump_rotation();
|
||||
|
||||
@@ -1231,7 +1478,13 @@ fn connect_to_buffered_groups() {
|
||||
assert_ne!(head_a, head_b);
|
||||
assert_ne!(group_a, group_b);
|
||||
|
||||
distribute_collation(&mut virtual_overseer, &test_state, false).await;
|
||||
distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
false,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Should be connected to both groups except for the validator that fetched advertised
|
||||
// collation.
|
||||
@@ -1248,7 +1501,7 @@ fn connect_to_buffered_groups() {
|
||||
}
|
||||
);
|
||||
|
||||
TestHarness { virtual_overseer, req_cfg }
|
||||
TestHarness { virtual_overseer, req_v1_cfg: req_cfg, req_vstaging_cfg }
|
||||
},
|
||||
);
|
||||
}
|
||||
+575
@@ -0,0 +1,575 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Tests for the collator side with enabled prospective parachains.
|
||||
|
||||
use super::*;
|
||||
|
||||
use polkadot_node_subsystem::messages::{ChainApiMessage, ProspectiveParachainsMessage};
|
||||
use polkadot_primitives::{vstaging as vstaging_primitives, Header, OccupiedCore};
|
||||
|
||||
const ASYNC_BACKING_PARAMETERS: vstaging_primitives::AsyncBackingParams =
|
||||
vstaging_primitives::AsyncBackingParams { max_candidate_depth: 4, allowed_ancestry_len: 3 };
|
||||
|
||||
fn get_parent_hash(hash: Hash) -> Hash {
|
||||
Hash::from_low_u64_be(hash.to_low_u64_be() + 1)
|
||||
}
|
||||
|
||||
/// Handle a view update.
|
||||
async fn update_view(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
new_view: Vec<(Hash, u32)>, // Hash and block number.
|
||||
activated: u8, // How many new heads does this update contain?
|
||||
) {
|
||||
let new_view: HashMap<Hash, u32> = HashMap::from_iter(new_view);
|
||||
|
||||
let our_view =
|
||||
OurView::new(new_view.keys().map(|hash| (*hash, Arc::new(jaeger::Span::Disabled))), 0);
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(our_view)),
|
||||
)
|
||||
.await;
|
||||
|
||||
let mut next_overseer_message = None;
|
||||
for _ in 0..activated {
|
||||
let (leaf_hash, leaf_number) = assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx),
|
||||
)) => {
|
||||
tx.send(Ok(ASYNC_BACKING_PARAMETERS)).unwrap();
|
||||
(parent, new_view.get(&parent).copied().expect("Unknown parent requested"))
|
||||
}
|
||||
);
|
||||
|
||||
let min_number = leaf_number.saturating_sub(ASYNC_BACKING_PARAMETERS.allowed_ancestry_len);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetMinimumRelayParents(parent, tx),
|
||||
) if parent == leaf_hash => {
|
||||
tx.send(vec![(test_state.para_id, min_number)]).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let ancestry_len = leaf_number + 1 - min_number;
|
||||
let ancestry_hashes = std::iter::successors(Some(leaf_hash), |h| Some(get_parent_hash(*h)))
|
||||
.take(ancestry_len as usize);
|
||||
let ancestry_numbers = (min_number..=leaf_number).rev();
|
||||
let mut ancestry_iter = ancestry_hashes.clone().zip(ancestry_numbers).peekable();
|
||||
|
||||
while let Some((hash, number)) = ancestry_iter.next() {
|
||||
// May be `None` for the last element.
|
||||
let parent_hash =
|
||||
ancestry_iter.peek().map(|(h, _)| *h).unwrap_or_else(|| get_parent_hash(hash));
|
||||
|
||||
let msg = match next_overseer_message.take() {
|
||||
Some(msg) => Some(msg),
|
||||
None =>
|
||||
overseer_recv_with_timeout(virtual_overseer, Duration::from_millis(50)).await,
|
||||
};
|
||||
|
||||
let msg = match msg {
|
||||
Some(msg) => msg,
|
||||
None => {
|
||||
// We're done.
|
||||
return
|
||||
},
|
||||
};
|
||||
|
||||
if !matches!(
|
||||
&msg,
|
||||
AllMessages::ChainApi(ChainApiMessage::BlockHeader(_hash, ..))
|
||||
if *_hash == hash
|
||||
) {
|
||||
// Ancestry has already been cached for this leaf.
|
||||
next_overseer_message.replace(msg);
|
||||
break
|
||||
}
|
||||
|
||||
assert_matches!(
|
||||
msg,
|
||||
AllMessages::ChainApi(ChainApiMessage::BlockHeader(.., tx)) => {
|
||||
let header = Header {
|
||||
parent_hash,
|
||||
number,
|
||||
state_root: Hash::zero(),
|
||||
extrinsics_root: Hash::zero(),
|
||||
digest: Default::default(),
|
||||
};
|
||||
|
||||
tx.send(Ok(Some(header))).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Check that the next received message is a `Declare` message.
|
||||
pub(super) async fn expect_declare_msg_vstaging(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
peer: &PeerId,
|
||||
) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendCollationMessage(
|
||||
to,
|
||||
Versioned::VStaging(protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
wire_message,
|
||||
)),
|
||||
)) => {
|
||||
assert_eq!(to[0], *peer);
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_vstaging::CollatorProtocolMessage::Declare(
|
||||
collator_id,
|
||||
para_id,
|
||||
signature,
|
||||
) => {
|
||||
assert!(signature.verify(
|
||||
&*protocol_vstaging::declare_signature_payload(&test_state.local_peer_id),
|
||||
&collator_id),
|
||||
);
|
||||
assert_eq!(collator_id, test_state.collator_pair.public());
|
||||
assert_eq!(para_id, test_state.para_id);
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/// Test that a collator distributes a collation from the allowed ancestry
|
||||
/// to correct validators group.
|
||||
#[test]
|
||||
fn distribute_collation_from_implicit_view() {
|
||||
let head_a = Hash::from_low_u64_be(126);
|
||||
let head_a_num: u32 = 66;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 64;
|
||||
|
||||
// Grandparent of head `b`.
|
||||
let head_c = Hash::from_low_u64_be(130);
|
||||
let head_c_num = 62;
|
||||
|
||||
let group_rotation_info = GroupRotationInfo {
|
||||
session_start_block: head_c_num - 2,
|
||||
group_rotation_frequency: 3,
|
||||
now: head_c_num,
|
||||
};
|
||||
|
||||
let mut test_state = TestState::default();
|
||||
test_state.group_rotation_info = group_rotation_info;
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(virtual_overseer, CollatorProtocolMessage::CollateOn(test_state.para_id))
|
||||
.await;
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let validator_peer_ids = test_state.current_group_validator_peer_ids();
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(validator_peer_ids.clone())
|
||||
{
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::VStaging, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
// Collator declared itself to each peer.
|
||||
for peer_id in &validator_peer_ids {
|
||||
expect_declare_msg_vstaging(virtual_overseer, &test_state, peer_id).await;
|
||||
}
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1, 2, 3]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xAA);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_c,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
let DistributeCollation { candidate, pov_block: _ } =
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
head_c,
|
||||
false, // Check the group manually.
|
||||
candidate,
|
||||
pov,
|
||||
parent_head_data_hash,
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ConnectToValidators { validator_ids, .. }
|
||||
) => {
|
||||
let expected_validators = test_state.current_group_validator_authority_ids();
|
||||
|
||||
assert_eq!(expected_validators, validator_ids);
|
||||
}
|
||||
);
|
||||
|
||||
let candidate_hash = candidate.hash();
|
||||
|
||||
// Update peer views.
|
||||
for peed_id in &validator_peer_ids {
|
||||
send_peer_view_change(virtual_overseer, peed_id, vec![head_b]).await;
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
peed_id,
|
||||
head_c,
|
||||
Some(vec![candidate_hash]),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
// Head `c` goes out of view.
|
||||
// Build a different candidate for this relay parent and attempt to distribute it.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![4, 5, 6]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xBB);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_c,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::DistributeCollation(
|
||||
candidate.clone(),
|
||||
parent_head_data_hash,
|
||||
pov.clone(),
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Parent out of view, nothing happens.
|
||||
assert!(overseer_recv_with_timeout(virtual_overseer, Duration::from_millis(100))
|
||||
.await
|
||||
.is_none());
|
||||
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Tests that collator can distribute up to `MAX_CANDIDATE_DEPTH + 1` candidates
|
||||
/// per relay parent.
|
||||
#[test]
|
||||
fn distribute_collation_up_to_limit() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
let head_a = Hash::from_low_u64_be(128);
|
||||
let head_a_num: u32 = 64;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(130);
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(virtual_overseer, CollatorProtocolMessage::CollateOn(test_state.para_id))
|
||||
.await;
|
||||
// Activated leaf is `a`, but the collation will be based on `b`.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
for i in 0..(ASYNC_BACKING_PARAMETERS.max_candidate_depth + 1) {
|
||||
let pov = PoV { block_data: BlockData(vec![i as u8]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xAA);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
head_b,
|
||||
true,
|
||||
candidate,
|
||||
pov,
|
||||
parent_head_data_hash,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![10, 12, 6]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xBB);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::DistributeCollation(
|
||||
candidate.clone(),
|
||||
parent_head_data_hash,
|
||||
pov.clone(),
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Limit has been reached.
|
||||
assert!(overseer_recv_with_timeout(virtual_overseer, Duration::from_millis(100))
|
||||
.await
|
||||
.is_none());
|
||||
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Tests that collator correctly handles peer V2 requests.
|
||||
#[test]
|
||||
fn advertise_and_send_collation_by_hash() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let req_v1_cfg = test_harness.req_v1_cfg;
|
||||
let mut req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
let head_a = Hash::from_low_u64_be(128);
|
||||
let head_a_num: u32 = 64;
|
||||
|
||||
// Parent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(129);
|
||||
let head_b_num: u32 = 63;
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::CollateOn(test_state.para_id),
|
||||
)
|
||||
.await;
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
let candidates: Vec<_> = (0..2)
|
||||
.map(|i| {
|
||||
let pov = PoV { block_data: BlockData(vec![i as u8]) };
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
(candidate, pov)
|
||||
})
|
||||
.collect();
|
||||
for (candidate, pov) in &candidates {
|
||||
distribute_collation_with_receipt(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
head_b,
|
||||
true,
|
||||
candidate.clone(),
|
||||
pov.clone(),
|
||||
Hash::zero(),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let peer = test_state.validator_peer_id[0];
|
||||
let validator_id = test_state.current_group_validator_authority_ids()[0].clone();
|
||||
connect_peer(
|
||||
&mut virtual_overseer,
|
||||
peer,
|
||||
CollationVersion::VStaging,
|
||||
Some(validator_id.clone()),
|
||||
)
|
||||
.await;
|
||||
expect_declare_msg_vstaging(&mut virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
// Head `b` is not a leaf, but both advertisements are still relevant.
|
||||
send_peer_view_change(&mut virtual_overseer, &peer, vec![head_b]).await;
|
||||
let hashes: Vec<_> = candidates.iter().map(|(candidate, _)| candidate.hash()).collect();
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, head_b, Some(hashes))
|
||||
.await;
|
||||
|
||||
for (candidate, pov_block) in candidates {
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
req_vstaging_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: request_vstaging::CollationFetchingRequest {
|
||||
relay_parent: head_b,
|
||||
para_id: test_state.para_id,
|
||||
candidate_hash: candidate.hash(),
|
||||
}
|
||||
.encode(),
|
||||
pending_response,
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
// Response is the same for vstaging.
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
.expect("Decoding should work");
|
||||
assert_eq!(receipt, candidate);
|
||||
assert_eq!(pov, pov_block);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg }
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Tests that collator distributes collation built on top of occupied core.
|
||||
#[test]
|
||||
fn advertise_core_occupied() {
|
||||
let mut test_state = TestState::default();
|
||||
let candidate =
|
||||
TestCandidateBuilder { para_id: test_state.para_id, ..Default::default() }.build();
|
||||
test_state.availability_cores[0] = CoreState::Occupied(OccupiedCore {
|
||||
next_up_on_available: None,
|
||||
occupied_since: 0,
|
||||
time_out_at: 0,
|
||||
next_up_on_time_out: None,
|
||||
availability: BitVec::default(),
|
||||
group_responsible: GroupIndex(0),
|
||||
candidate_hash: candidate.hash(),
|
||||
candidate_descriptor: candidate.descriptor,
|
||||
});
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
let head_a = Hash::from_low_u64_be(128);
|
||||
let head_a_num: u32 = 64;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(130);
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(virtual_overseer, CollatorProtocolMessage::CollateOn(test_state.para_id))
|
||||
.await;
|
||||
// Activated leaf is `a`, but the collation will be based on `b`.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1, 2, 3]) };
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
let candidate_hash = candidate.hash();
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
head_b,
|
||||
true,
|
||||
candidate,
|
||||
pov,
|
||||
Hash::zero(),
|
||||
)
|
||||
.await;
|
||||
|
||||
let validators = test_state.current_group_validator_authority_ids();
|
||||
let peer_ids = test_state.current_group_validator_peer_ids();
|
||||
|
||||
connect_peer(
|
||||
virtual_overseer,
|
||||
peer_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
Some(validators[0].clone()),
|
||||
)
|
||||
.await;
|
||||
expect_declare_msg_vstaging(virtual_overseer, &test_state, &peer_ids[0]).await;
|
||||
// Peer is aware of the leaf.
|
||||
send_peer_view_change(virtual_overseer, &peer_ids[0], vec![head_a]).await;
|
||||
|
||||
// Collation is advertised.
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&peer_ids[0],
|
||||
head_b,
|
||||
Some(vec![candidate_hash]),
|
||||
)
|
||||
.await;
|
||||
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
}
|
||||
@@ -31,13 +31,19 @@
|
||||
|
||||
use std::{
|
||||
collections::{HashMap, VecDeque},
|
||||
future::Future,
|
||||
num::NonZeroUsize,
|
||||
ops::Range,
|
||||
pin::Pin,
|
||||
task::{Context, Poll},
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
use bitvec::{bitvec, vec::BitVec};
|
||||
use futures::FutureExt;
|
||||
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, GroupIndex, Hash, SessionIndex};
|
||||
use polkadot_node_network_protocol::PeerId;
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, CandidateHash, GroupIndex, SessionIndex};
|
||||
|
||||
/// The ring buffer stores at most this many unique validator groups.
|
||||
///
|
||||
@@ -66,9 +72,9 @@ pub struct ValidatorGroupsBuffer {
|
||||
group_infos: VecDeque<ValidatorsGroupInfo>,
|
||||
/// Continuous buffer of validators discovery keys.
|
||||
validators: VecDeque<AuthorityDiscoveryId>,
|
||||
/// Mapping from relay-parent to bit-vectors with bits for all `validators`.
|
||||
/// Mapping from candidate hashes to bit-vectors with bits for all `validators`.
|
||||
/// Invariants kept: All bit-vectors are guaranteed to have the same size.
|
||||
should_be_connected: HashMap<Hash, BitVec>,
|
||||
should_be_connected: HashMap<CandidateHash, BitVec>,
|
||||
/// Buffer capacity, limits the number of **groups** tracked.
|
||||
cap: NonZeroUsize,
|
||||
}
|
||||
@@ -107,7 +113,7 @@ impl ValidatorGroupsBuffer {
|
||||
/// of the buffer.
|
||||
pub fn note_collation_advertised(
|
||||
&mut self,
|
||||
relay_parent: Hash,
|
||||
candidate_hash: CandidateHash,
|
||||
session_index: SessionIndex,
|
||||
group_index: GroupIndex,
|
||||
validators: &[AuthorityDiscoveryId],
|
||||
@@ -121,19 +127,19 @@ impl ValidatorGroupsBuffer {
|
||||
}) {
|
||||
Some((idx, group)) => {
|
||||
let group_start_idx = self.group_lengths_iter().take(idx).sum();
|
||||
self.set_bits(relay_parent, group_start_idx..(group_start_idx + group.len));
|
||||
self.set_bits(candidate_hash, group_start_idx..(group_start_idx + group.len));
|
||||
},
|
||||
None => self.push(relay_parent, session_index, group_index, validators),
|
||||
None => self.push(candidate_hash, session_index, group_index, validators),
|
||||
}
|
||||
}
|
||||
|
||||
/// Note that a validator is no longer interested in a given relay parent.
|
||||
pub fn reset_validator_interest(
|
||||
&mut self,
|
||||
relay_parent: Hash,
|
||||
candidate_hash: CandidateHash,
|
||||
authority_id: &AuthorityDiscoveryId,
|
||||
) {
|
||||
let bits = match self.should_be_connected.get_mut(&relay_parent) {
|
||||
let bits = match self.should_be_connected.get_mut(&candidate_hash) {
|
||||
Some(bits) => bits,
|
||||
None => return,
|
||||
};
|
||||
@@ -145,17 +151,12 @@ impl ValidatorGroupsBuffer {
|
||||
}
|
||||
}
|
||||
|
||||
/// Remove relay parent from the buffer.
|
||||
/// Remove advertised candidate from the buffer.
|
||||
///
|
||||
/// The buffer will no longer track which validators are interested in a corresponding
|
||||
/// advertisement.
|
||||
pub fn remove_relay_parent(&mut self, relay_parent: &Hash) {
|
||||
self.should_be_connected.remove(relay_parent);
|
||||
}
|
||||
|
||||
/// Removes all advertisements from the buffer.
|
||||
pub fn clear_advertisements(&mut self) {
|
||||
self.should_be_connected.clear();
|
||||
pub fn remove_candidate(&mut self, candidate_hash: &CandidateHash) {
|
||||
self.should_be_connected.remove(candidate_hash);
|
||||
}
|
||||
|
||||
/// Pushes a new group to the buffer along with advertisement, setting all validators
|
||||
@@ -164,7 +165,7 @@ impl ValidatorGroupsBuffer {
|
||||
/// If the buffer is full, drops group from the tail.
|
||||
fn push(
|
||||
&mut self,
|
||||
relay_parent: Hash,
|
||||
candidate_hash: CandidateHash,
|
||||
session_index: SessionIndex,
|
||||
group_index: GroupIndex,
|
||||
validators: &[AuthorityDiscoveryId],
|
||||
@@ -193,17 +194,17 @@ impl ValidatorGroupsBuffer {
|
||||
self.should_be_connected
|
||||
.values_mut()
|
||||
.for_each(|bits| bits.resize(new_len, false));
|
||||
self.set_bits(relay_parent, group_start_idx..(group_start_idx + validators.len()));
|
||||
self.set_bits(candidate_hash, group_start_idx..(group_start_idx + validators.len()));
|
||||
}
|
||||
|
||||
/// Sets advertisement bits to 1 in a given range (usually corresponding to some group).
|
||||
/// If the relay parent is unknown, inserts 0-initialized bitvec first.
|
||||
///
|
||||
/// The range must be ensured to be within bounds.
|
||||
fn set_bits(&mut self, relay_parent: Hash, range: Range<usize>) {
|
||||
fn set_bits(&mut self, candidate_hash: CandidateHash, range: Range<usize>) {
|
||||
let bits = self
|
||||
.should_be_connected
|
||||
.entry(relay_parent)
|
||||
.entry(candidate_hash)
|
||||
.or_insert_with(|| bitvec![0; self.validators.len()]);
|
||||
|
||||
bits[range].fill(true);
|
||||
@@ -217,9 +218,40 @@ impl ValidatorGroupsBuffer {
|
||||
}
|
||||
}
|
||||
|
||||
/// A timeout for resetting validators' interests in collations.
|
||||
pub const RESET_INTEREST_TIMEOUT: Duration = Duration::from_secs(6);
|
||||
|
||||
/// A future that returns a candidate hash along with validator discovery
|
||||
/// keys once a timeout hit.
|
||||
///
|
||||
/// If a validator doesn't manage to fetch a collation within this timeout
|
||||
/// we should reset its interest in this advertisement in a buffer. For example,
|
||||
/// when the PoV was already requested from another peer.
|
||||
pub struct ResetInterestTimeout {
|
||||
fut: futures_timer::Delay,
|
||||
candidate_hash: CandidateHash,
|
||||
peer_id: PeerId,
|
||||
}
|
||||
|
||||
impl ResetInterestTimeout {
|
||||
/// Returns new `ResetInterestTimeout` that resolves after given timeout.
|
||||
pub fn new(candidate_hash: CandidateHash, peer_id: PeerId, delay: Duration) -> Self {
|
||||
Self { fut: futures_timer::Delay::new(delay), candidate_hash, peer_id }
|
||||
}
|
||||
}
|
||||
|
||||
impl Future for ResetInterestTimeout {
|
||||
type Output = (CandidateHash, PeerId);
|
||||
|
||||
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
self.fut.poll_unpin(cx).map(|_| (self.candidate_hash, self.peer_id))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use polkadot_primitives::Hash;
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
|
||||
#[test]
|
||||
@@ -227,8 +259,8 @@ mod tests {
|
||||
let cap = NonZeroUsize::new(1).unwrap();
|
||||
let mut buf = ValidatorGroupsBuffer::with_capacity(cap);
|
||||
|
||||
let hash_a = Hash::repeat_byte(0x1);
|
||||
let hash_b = Hash::repeat_byte(0x2);
|
||||
let hash_a = CandidateHash(Hash::repeat_byte(0x1));
|
||||
let hash_b = CandidateHash(Hash::repeat_byte(0x2));
|
||||
|
||||
let validators: Vec<_> = [
|
||||
Sr25519Keyring::Alice,
|
||||
@@ -263,7 +295,7 @@ mod tests {
|
||||
let cap = NonZeroUsize::new(3).unwrap();
|
||||
let mut buf = ValidatorGroupsBuffer::with_capacity(cap);
|
||||
|
||||
let hashes: Vec<_> = (0..5).map(Hash::repeat_byte).collect();
|
||||
let hashes: Vec<_> = (0..5).map(|i| CandidateHash(Hash::repeat_byte(i))).collect();
|
||||
|
||||
let validators: Vec<_> = [
|
||||
Sr25519Keyring::Alice,
|
||||
|
||||
@@ -17,10 +17,12 @@
|
||||
|
||||
//! Error handling related code and Error/Result definitions.
|
||||
|
||||
use futures::channel::oneshot;
|
||||
|
||||
use polkadot_node_network_protocol::request_response::incoming;
|
||||
use polkadot_node_primitives::UncheckedSignedFullStatement;
|
||||
use polkadot_node_subsystem::errors::SubsystemError;
|
||||
use polkadot_node_subsystem_util::runtime;
|
||||
use polkadot_node_subsystem::{errors::SubsystemError, RuntimeApiError};
|
||||
use polkadot_node_subsystem_util::{backing_implicit_view, runtime};
|
||||
|
||||
use crate::LOG_TARGET;
|
||||
|
||||
@@ -44,10 +46,78 @@ pub enum Error {
|
||||
#[error("Error while accessing runtime information")]
|
||||
Runtime(#[from] runtime::Error),
|
||||
|
||||
#[error("Error while accessing Runtime API")]
|
||||
RuntimeApi(#[from] RuntimeApiError),
|
||||
|
||||
#[error(transparent)]
|
||||
ImplicitViewFetchError(backing_implicit_view::FetchError),
|
||||
|
||||
#[error("Response receiver for active validators request cancelled")]
|
||||
CancelledActiveValidators(oneshot::Canceled),
|
||||
|
||||
#[error("Response receiver for validator groups request cancelled")]
|
||||
CancelledValidatorGroups(oneshot::Canceled),
|
||||
|
||||
#[error("Response receiver for availability cores request cancelled")]
|
||||
CancelledAvailabilityCores(oneshot::Canceled),
|
||||
|
||||
#[error("CollationSeconded contained statement with invalid signature")]
|
||||
InvalidStatementSignature(UncheckedSignedFullStatement),
|
||||
}
|
||||
|
||||
/// An error happened on the validator side of the protocol when attempting
|
||||
/// to start seconding a candidate.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum SecondingError {
|
||||
#[error("Error while accessing Runtime API")]
|
||||
RuntimeApi(#[from] RuntimeApiError),
|
||||
|
||||
#[error("Response receiver for persisted validation data request cancelled")]
|
||||
CancelledRuntimePersistedValidationData(oneshot::Canceled),
|
||||
|
||||
#[error("Response receiver for prospective validation data request cancelled")]
|
||||
CancelledProspectiveValidationData(oneshot::Canceled),
|
||||
|
||||
#[error("Persisted validation data is not available")]
|
||||
PersistedValidationDataNotFound,
|
||||
|
||||
#[error("Persisted validation data hash doesn't match one in the candidate receipt.")]
|
||||
PersistedValidationDataMismatch,
|
||||
|
||||
#[error("Candidate hash doesn't match the advertisement")]
|
||||
CandidateHashMismatch,
|
||||
|
||||
#[error("Received duplicate collation from the peer")]
|
||||
Duplicate,
|
||||
}
|
||||
|
||||
impl SecondingError {
|
||||
/// Returns true if an error indicates that a peer is malicious.
|
||||
pub fn is_malicious(&self) -> bool {
|
||||
use SecondingError::*;
|
||||
matches!(self, PersistedValidationDataMismatch | CandidateHashMismatch | Duplicate)
|
||||
}
|
||||
}
|
||||
|
||||
/// A validator failed to request a collation due to an error.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum FetchError {
|
||||
#[error("Collation was not previously advertised")]
|
||||
NotAdvertised,
|
||||
|
||||
#[error("Peer is unknown")]
|
||||
UnknownPeer,
|
||||
|
||||
#[error("Collation was already requested")]
|
||||
AlreadyRequested,
|
||||
|
||||
#[error("Relay parent went out of view")]
|
||||
RelayParentOutOfView,
|
||||
|
||||
#[error("Peer's protocol doesn't match the advertisement")]
|
||||
ProtocolMismatch,
|
||||
}
|
||||
|
||||
/// Utility for eating top level errors and log them.
|
||||
///
|
||||
/// We basically always want to try and continue on error. This utility function is meant to
|
||||
|
||||
@@ -32,7 +32,7 @@ use polkadot_node_subsystem_util::reputation::ReputationAggregator;
|
||||
use sp_keystore::KeystorePtr;
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
request_response::{v1 as request_v1, IncomingRequestReceiver},
|
||||
request_response::{v1 as request_v1, vstaging as protocol_vstaging, IncomingRequestReceiver},
|
||||
PeerId, UnifiedReputationChange as Rep,
|
||||
};
|
||||
use polkadot_primitives::CollatorPair;
|
||||
@@ -76,12 +76,19 @@ pub enum ProtocolSide {
|
||||
metrics: validator_side::Metrics,
|
||||
},
|
||||
/// Collators operate on a parachain.
|
||||
Collator(
|
||||
PeerId,
|
||||
CollatorPair,
|
||||
IncomingRequestReceiver<request_v1::CollationFetchingRequest>,
|
||||
collator_side::Metrics,
|
||||
),
|
||||
Collator {
|
||||
/// Local peer id.
|
||||
peer_id: PeerId,
|
||||
/// Parachain collator pair.
|
||||
collator_pair: CollatorPair,
|
||||
/// Receiver for v1 collation fetching requests.
|
||||
request_receiver_v1: IncomingRequestReceiver<request_v1::CollationFetchingRequest>,
|
||||
/// Receiver for vstaging collation fetching requests.
|
||||
request_receiver_vstaging:
|
||||
IncomingRequestReceiver<protocol_vstaging::CollationFetchingRequest>,
|
||||
/// Metrics.
|
||||
metrics: collator_side::Metrics,
|
||||
},
|
||||
/// No protocol side, just disable it.
|
||||
None,
|
||||
}
|
||||
@@ -110,10 +117,22 @@ impl<Context> CollatorProtocolSubsystem {
|
||||
validator_side::run(ctx, keystore, eviction_policy, metrics)
|
||||
.map_err(|e| SubsystemError::with_origin("collator-protocol", e))
|
||||
.boxed(),
|
||||
ProtocolSide::Collator(local_peer_id, collator_pair, req_receiver, metrics) =>
|
||||
collator_side::run(ctx, local_peer_id, collator_pair, req_receiver, metrics)
|
||||
.map_err(|e| SubsystemError::with_origin("collator-protocol", e))
|
||||
.boxed(),
|
||||
ProtocolSide::Collator {
|
||||
peer_id,
|
||||
collator_pair,
|
||||
request_receiver_v1,
|
||||
request_receiver_vstaging,
|
||||
metrics,
|
||||
} => collator_side::run(
|
||||
ctx,
|
||||
peer_id,
|
||||
collator_pair,
|
||||
request_receiver_v1,
|
||||
request_receiver_vstaging,
|
||||
metrics,
|
||||
)
|
||||
.map_err(|e| SubsystemError::with_origin("collator-protocol", e))
|
||||
.boxed(),
|
||||
ProtocolSide::None => return DummySubsystem.start(ctx),
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,366 @@
|
||||
// Copyright 2017-2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Primitives for tracking collations-related data.
|
||||
//!
|
||||
//! Usually a path of collations is as follows:
|
||||
//! 1. First, collation must be advertised by collator.
|
||||
//! 2. If the advertisement was accepted, it's queued for fetch (per relay parent).
|
||||
//! 3. Once it's requested, the collation is said to be Pending.
|
||||
//! 4. Pending collation becomes Fetched once received, we send it to backing for validation.
|
||||
//! 5. If it turns to be invalid or async backing allows seconding another candidate, carry on
|
||||
//! with the next advertisement, otherwise we're done with this relay parent.
|
||||
//!
|
||||
//! ┌──────────────────────────────────────────┐
|
||||
//! └─▶Advertised ─▶ Pending ─▶ Fetched ─▶ Validated
|
||||
|
||||
use std::{collections::VecDeque, future::Future, pin::Pin, task::Poll};
|
||||
|
||||
use futures::{future::BoxFuture, FutureExt};
|
||||
use polkadot_node_network_protocol::{
|
||||
request_response::{outgoing::RequestError, v1 as request_v1, OutgoingResult},
|
||||
PeerId,
|
||||
};
|
||||
use polkadot_node_primitives::PoV;
|
||||
use polkadot_node_subsystem::jaeger;
|
||||
use polkadot_node_subsystem_util::{
|
||||
metrics::prometheus::prometheus::HistogramTimer, runtime::ProspectiveParachainsMode,
|
||||
};
|
||||
use polkadot_primitives::{
|
||||
CandidateHash, CandidateReceipt, CollatorId, Hash, Id as ParaId, PersistedValidationData,
|
||||
};
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
use crate::{error::SecondingError, LOG_TARGET};
|
||||
|
||||
/// Candidate supplied with a para head it's built on top of.
|
||||
#[derive(Debug, Copy, Clone, Hash, Eq, PartialEq)]
|
||||
pub struct ProspectiveCandidate {
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Parent head-data hash as supplied in advertisement.
|
||||
pub parent_head_data_hash: Hash,
|
||||
}
|
||||
|
||||
impl ProspectiveCandidate {
|
||||
pub fn candidate_hash(&self) -> CandidateHash {
|
||||
self.candidate_hash
|
||||
}
|
||||
}
|
||||
|
||||
/// Identifier of a fetched collation.
|
||||
#[derive(Debug, Clone, Hash, Eq, PartialEq)]
|
||||
pub struct FetchedCollation {
|
||||
/// Candidate's relay parent.
|
||||
pub relay_parent: Hash,
|
||||
/// Parachain id.
|
||||
pub para_id: ParaId,
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Id of the collator the collation was fetched from.
|
||||
pub collator_id: CollatorId,
|
||||
}
|
||||
|
||||
impl From<&CandidateReceipt<Hash>> for FetchedCollation {
|
||||
fn from(receipt: &CandidateReceipt<Hash>) -> Self {
|
||||
let descriptor = receipt.descriptor();
|
||||
Self {
|
||||
relay_parent: descriptor.relay_parent,
|
||||
para_id: descriptor.para_id,
|
||||
candidate_hash: receipt.hash(),
|
||||
collator_id: descriptor.collator.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Identifier of a collation being requested.
|
||||
#[derive(Debug, Copy, Clone, Hash, Eq, PartialEq)]
|
||||
pub struct PendingCollation {
|
||||
/// Candidate's relay parent.
|
||||
pub relay_parent: Hash,
|
||||
/// Parachain id.
|
||||
pub para_id: ParaId,
|
||||
/// Peer that advertised this collation.
|
||||
pub peer_id: PeerId,
|
||||
/// Optional candidate hash and parent head-data hash if were
|
||||
/// supplied in advertisement.
|
||||
pub prospective_candidate: Option<ProspectiveCandidate>,
|
||||
/// Hash of the candidate's commitments.
|
||||
pub commitments_hash: Option<Hash>,
|
||||
}
|
||||
|
||||
impl PendingCollation {
|
||||
pub fn new(
|
||||
relay_parent: Hash,
|
||||
para_id: ParaId,
|
||||
peer_id: &PeerId,
|
||||
prospective_candidate: Option<ProspectiveCandidate>,
|
||||
) -> Self {
|
||||
Self {
|
||||
relay_parent,
|
||||
para_id,
|
||||
peer_id: *peer_id,
|
||||
prospective_candidate,
|
||||
commitments_hash: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// vstaging advertisement that was rejected by the backing
|
||||
/// subsystem. Validator may fetch it later if its fragment
|
||||
/// membership gets recognized before relay parent goes out of view.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct BlockedAdvertisement {
|
||||
/// Peer that advertised the collation.
|
||||
pub peer_id: PeerId,
|
||||
/// Collator id.
|
||||
pub collator_id: CollatorId,
|
||||
/// The relay-parent of the candidate.
|
||||
pub candidate_relay_parent: Hash,
|
||||
/// Hash of the candidate.
|
||||
pub candidate_hash: CandidateHash,
|
||||
}
|
||||
|
||||
/// Performs a sanity check between advertised and fetched collations.
|
||||
///
|
||||
/// Since the persisted validation data is constructed using the advertised
|
||||
/// parent head data hash, the latter doesn't require an additional check.
|
||||
pub fn fetched_collation_sanity_check(
|
||||
advertised: &PendingCollation,
|
||||
fetched: &CandidateReceipt,
|
||||
persisted_validation_data: &PersistedValidationData,
|
||||
) -> Result<(), SecondingError> {
|
||||
if persisted_validation_data.hash() != fetched.descriptor().persisted_validation_data_hash {
|
||||
Err(SecondingError::PersistedValidationDataMismatch)
|
||||
} else if advertised
|
||||
.prospective_candidate
|
||||
.map_or(false, |pc| pc.candidate_hash() != fetched.hash())
|
||||
{
|
||||
Err(SecondingError::CandidateHashMismatch)
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Identifier for a requested collation and the respective collator that advertised it.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CollationEvent {
|
||||
/// Collator id.
|
||||
pub collator_id: CollatorId,
|
||||
/// The requested collation data.
|
||||
pub pending_collation: PendingCollation,
|
||||
}
|
||||
|
||||
/// Fetched collation data.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PendingCollationFetch {
|
||||
/// Collation identifier.
|
||||
pub collation_event: CollationEvent,
|
||||
/// Candidate receipt.
|
||||
pub candidate_receipt: CandidateReceipt,
|
||||
/// Proof of validity.
|
||||
pub pov: PoV,
|
||||
}
|
||||
|
||||
/// The status of the collations in [`CollationsPerRelayParent`].
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub enum CollationStatus {
|
||||
/// We are waiting for a collation to be advertised to us.
|
||||
Waiting,
|
||||
/// We are currently fetching a collation.
|
||||
Fetching,
|
||||
/// We are waiting that a collation is being validated.
|
||||
WaitingOnValidation,
|
||||
/// We have seconded a collation.
|
||||
Seconded,
|
||||
}
|
||||
|
||||
impl Default for CollationStatus {
|
||||
fn default() -> Self {
|
||||
Self::Waiting
|
||||
}
|
||||
}
|
||||
|
||||
impl CollationStatus {
|
||||
/// Downgrades to `Waiting`, but only if `self != Seconded`.
|
||||
fn back_to_waiting(&mut self, relay_parent_mode: ProspectiveParachainsMode) {
|
||||
match self {
|
||||
Self::Seconded =>
|
||||
if relay_parent_mode.is_enabled() {
|
||||
// With async backing enabled it's allowed to
|
||||
// second more candidates.
|
||||
*self = Self::Waiting
|
||||
},
|
||||
_ => *self = Self::Waiting,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Information about collations per relay parent.
|
||||
#[derive(Default)]
|
||||
pub struct Collations {
|
||||
/// What is the current status in regards to a collation for this relay parent?
|
||||
pub status: CollationStatus,
|
||||
/// Collator we're fetching from, optionally which candidate was requested.
|
||||
///
|
||||
/// This is the currently last started fetch, which did not exceed `MAX_UNSHARED_DOWNLOAD_TIME`
|
||||
/// yet.
|
||||
pub fetching_from: Option<(CollatorId, Option<CandidateHash>)>,
|
||||
/// Collation that were advertised to us, but we did not yet fetch.
|
||||
pub waiting_queue: VecDeque<(PendingCollation, CollatorId)>,
|
||||
/// How many collations have been seconded.
|
||||
pub seconded_count: usize,
|
||||
}
|
||||
|
||||
impl Collations {
|
||||
/// Note a seconded collation for a given para.
|
||||
pub(super) fn note_seconded(&mut self) {
|
||||
self.seconded_count += 1
|
||||
}
|
||||
|
||||
/// Returns the next collation to fetch from the `waiting_queue`.
|
||||
///
|
||||
/// This will reset the status back to `Waiting` using [`CollationStatus::back_to_waiting`].
|
||||
///
|
||||
/// Returns `Some(_)` if there is any collation to fetch, the `status` is not `Seconded` and
|
||||
/// the passed in `finished_one` is the currently `waiting_collation`.
|
||||
pub(super) fn get_next_collation_to_fetch(
|
||||
&mut self,
|
||||
finished_one: &(CollatorId, Option<CandidateHash>),
|
||||
relay_parent_mode: ProspectiveParachainsMode,
|
||||
) -> Option<(PendingCollation, CollatorId)> {
|
||||
// If finished one does not match waiting_collation, then we already dequeued another fetch
|
||||
// to replace it.
|
||||
if let Some((collator_id, maybe_candidate_hash)) = self.fetching_from.as_ref() {
|
||||
// If a candidate hash was saved previously, `finished_one` must include this too.
|
||||
if collator_id != &finished_one.0 &&
|
||||
maybe_candidate_hash.map_or(true, |hash| Some(&hash) != finished_one.1.as_ref())
|
||||
{
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
waiting_collation = ?self.fetching_from,
|
||||
?finished_one,
|
||||
"Not proceeding to the next collation - has already been done."
|
||||
);
|
||||
return None
|
||||
}
|
||||
}
|
||||
self.status.back_to_waiting(relay_parent_mode);
|
||||
|
||||
match self.status {
|
||||
// We don't need to fetch any other collation when we already have seconded one.
|
||||
CollationStatus::Seconded => None,
|
||||
CollationStatus::Waiting =>
|
||||
if !self.is_seconded_limit_reached(relay_parent_mode) {
|
||||
None
|
||||
} else {
|
||||
self.waiting_queue.pop_front()
|
||||
},
|
||||
CollationStatus::WaitingOnValidation | CollationStatus::Fetching =>
|
||||
unreachable!("We have reset the status above!"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Checks the limit of seconded candidates for a given para.
|
||||
pub(super) fn is_seconded_limit_reached(
|
||||
&self,
|
||||
relay_parent_mode: ProspectiveParachainsMode,
|
||||
) -> bool {
|
||||
let seconded_limit =
|
||||
if let ProspectiveParachainsMode::Enabled { max_candidate_depth, .. } =
|
||||
relay_parent_mode
|
||||
{
|
||||
max_candidate_depth + 1
|
||||
} else {
|
||||
1
|
||||
};
|
||||
self.seconded_count < seconded_limit
|
||||
}
|
||||
}
|
||||
|
||||
// Any error that can occur when awaiting a collation fetch response.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub(super) enum CollationFetchError {
|
||||
#[error("Future was cancelled.")]
|
||||
Cancelled,
|
||||
#[error("{0}")]
|
||||
Request(#[from] RequestError),
|
||||
}
|
||||
|
||||
/// Future that concludes when the collator has responded to our collation fetch request
|
||||
/// or the request was cancelled by the validator.
|
||||
pub(super) struct CollationFetchRequest {
|
||||
/// Info about the requested collation.
|
||||
pub pending_collation: PendingCollation,
|
||||
/// Collator id.
|
||||
pub collator_id: CollatorId,
|
||||
/// Responses from collator.
|
||||
pub from_collator: BoxFuture<'static, OutgoingResult<request_v1::CollationFetchingResponse>>,
|
||||
/// Handle used for checking if this request was cancelled.
|
||||
pub cancellation_token: CancellationToken,
|
||||
/// A jaeger span corresponding to the lifetime of the request.
|
||||
pub span: Option<jaeger::Span>,
|
||||
/// A metric histogram for the lifetime of the request
|
||||
pub _lifetime_timer: Option<HistogramTimer>,
|
||||
}
|
||||
|
||||
impl Future for CollationFetchRequest {
|
||||
type Output = (
|
||||
CollationEvent,
|
||||
std::result::Result<request_v1::CollationFetchingResponse, CollationFetchError>,
|
||||
);
|
||||
|
||||
fn poll(mut self: Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll<Self::Output> {
|
||||
// First check if this fetch request was cancelled.
|
||||
let cancelled = match std::pin::pin!(self.cancellation_token.cancelled()).poll(cx) {
|
||||
Poll::Ready(()) => true,
|
||||
Poll::Pending => false,
|
||||
};
|
||||
|
||||
if cancelled {
|
||||
self.span.as_mut().map(|s| s.add_string_tag("success", "false"));
|
||||
return Poll::Ready((
|
||||
CollationEvent {
|
||||
collator_id: self.collator_id.clone(),
|
||||
pending_collation: self.pending_collation,
|
||||
},
|
||||
Err(CollationFetchError::Cancelled),
|
||||
))
|
||||
}
|
||||
|
||||
let res = self.from_collator.poll_unpin(cx).map(|res| {
|
||||
(
|
||||
CollationEvent {
|
||||
collator_id: self.collator_id.clone(),
|
||||
pending_collation: self.pending_collation,
|
||||
},
|
||||
res.map_err(CollationFetchError::Request),
|
||||
)
|
||||
});
|
||||
|
||||
match &res {
|
||||
Poll::Ready((_, Ok(request_v1::CollationFetchingResponse::Collation(..)))) => {
|
||||
self.span.as_mut().map(|s| s.add_string_tag("success", "true"));
|
||||
},
|
||||
Poll::Ready((_, Err(_))) => {
|
||||
self.span.as_mut().map(|s| s.add_string_tag("success", "false"));
|
||||
},
|
||||
_ => {},
|
||||
};
|
||||
|
||||
res
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
// Copyright 2017-2023 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
use polkadot_node_subsystem_util::metrics::{self, prometheus};
|
||||
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Metrics(Option<MetricsInner>);
|
||||
|
||||
impl Metrics {
|
||||
pub fn on_request(&self, succeeded: std::result::Result<(), ()>) {
|
||||
if let Some(metrics) = &self.0 {
|
||||
match succeeded {
|
||||
Ok(()) => metrics.collation_requests.with_label_values(&["succeeded"]).inc(),
|
||||
Err(()) => metrics.collation_requests.with_label_values(&["failed"]).inc(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Provide a timer for `process_msg` which observes on drop.
|
||||
pub fn time_process_msg(&self) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0.as_ref().map(|metrics| metrics.process_msg.start_timer())
|
||||
}
|
||||
|
||||
/// Provide a timer for `handle_collation_request_result` which observes on drop.
|
||||
pub fn time_handle_collation_request_result(
|
||||
&self,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0
|
||||
.as_ref()
|
||||
.map(|metrics| metrics.handle_collation_request_result.start_timer())
|
||||
}
|
||||
|
||||
/// Note the current number of collator peers.
|
||||
pub fn note_collator_peer_count(&self, collator_peers: usize) {
|
||||
self.0
|
||||
.as_ref()
|
||||
.map(|metrics| metrics.collator_peer_count.set(collator_peers as u64));
|
||||
}
|
||||
|
||||
/// Provide a timer for `CollationFetchRequest` structure which observes on drop.
|
||||
pub fn time_collation_request_duration(
|
||||
&self,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0.as_ref().map(|metrics| metrics.collation_request_duration.start_timer())
|
||||
}
|
||||
|
||||
/// Provide a timer for `request_unblocked_collations` which observes on drop.
|
||||
pub fn time_request_unblocked_collations(
|
||||
&self,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0
|
||||
.as_ref()
|
||||
.map(|metrics| metrics.request_unblocked_collations.start_timer())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct MetricsInner {
|
||||
collation_requests: prometheus::CounterVec<prometheus::U64>,
|
||||
process_msg: prometheus::Histogram,
|
||||
handle_collation_request_result: prometheus::Histogram,
|
||||
collator_peer_count: prometheus::Gauge<prometheus::U64>,
|
||||
collation_request_duration: prometheus::Histogram,
|
||||
request_unblocked_collations: prometheus::Histogram,
|
||||
}
|
||||
|
||||
impl metrics::Metrics for Metrics {
|
||||
fn try_register(
|
||||
registry: &prometheus::Registry,
|
||||
) -> std::result::Result<Self, prometheus::PrometheusError> {
|
||||
let metrics = MetricsInner {
|
||||
collation_requests: prometheus::register(
|
||||
prometheus::CounterVec::new(
|
||||
prometheus::Opts::new(
|
||||
"polkadot_parachain_collation_requests_total",
|
||||
"Number of collations requested from Collators.",
|
||||
),
|
||||
&["success"],
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
process_msg: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_process_msg",
|
||||
"Time spent within `collator_protocol_validator::process_msg`",
|
||||
)
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
handle_collation_request_result: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_handle_collation_request_result",
|
||||
"Time spent within `collator_protocol_validator::handle_collation_request_result`",
|
||||
)
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
collator_peer_count: prometheus::register(
|
||||
prometheus::Gauge::new(
|
||||
"polkadot_parachain_collator_peer_count",
|
||||
"Amount of collator peers connected",
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
collation_request_duration: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_collation_request_duration",
|
||||
"Lifetime of the `CollationFetchRequest` structure",
|
||||
).buckets(vec![0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.75, 0.9, 1.0, 1.2, 1.5, 1.75]),
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
request_unblocked_collations: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_request_unblocked_collations",
|
||||
"Time spent within `collator_protocol_validator::request_unblocked_collations`",
|
||||
)
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
};
|
||||
|
||||
Ok(Metrics(Some(metrics)))
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
+348
-105
@@ -19,8 +19,8 @@ use assert_matches::assert_matches;
|
||||
use futures::{executor, future, Future};
|
||||
use sp_core::{crypto::Pair, Encode};
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
use sp_keystore::{testing::MemoryKeystore, Keystore};
|
||||
use std::{iter, sync::Arc, task::Poll, time::Duration};
|
||||
use sp_keystore::Keystore;
|
||||
use std::{iter, sync::Arc, time::Duration};
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
our_view,
|
||||
@@ -28,24 +28,39 @@ use polkadot_node_network_protocol::{
|
||||
request_response::{Requests, ResponseSender},
|
||||
ObservedRole,
|
||||
};
|
||||
use polkadot_node_primitives::BlockData;
|
||||
use polkadot_node_subsystem::messages::{
|
||||
AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest,
|
||||
use polkadot_node_primitives::{BlockData, PoV};
|
||||
use polkadot_node_subsystem::{
|
||||
errors::RuntimeApiError,
|
||||
messages::{AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest},
|
||||
};
|
||||
use polkadot_node_subsystem_test_helpers as test_helpers;
|
||||
use polkadot_node_subsystem_util::{reputation::add_reputation, TimeoutExt};
|
||||
use polkadot_primitives::{
|
||||
CollatorPair, CoreState, GroupIndex, GroupRotationInfo, OccupiedCore, ScheduledCore,
|
||||
ValidatorId, ValidatorIndex,
|
||||
CandidateReceipt, CollatorPair, CoreState, GroupIndex, GroupRotationInfo, HeadData,
|
||||
OccupiedCore, PersistedValidationData, ScheduledCore, ValidatorId, ValidatorIndex,
|
||||
};
|
||||
use polkadot_primitives_test_helpers::{
|
||||
dummy_candidate_descriptor, dummy_candidate_receipt_bad_sig, dummy_hash,
|
||||
};
|
||||
|
||||
mod prospective_parachains;
|
||||
|
||||
const ACTIVITY_TIMEOUT: Duration = Duration::from_millis(500);
|
||||
const DECLARE_TIMEOUT: Duration = Duration::from_millis(25);
|
||||
const REPUTATION_CHANGE_TEST_INTERVAL: Duration = Duration::from_millis(10);
|
||||
|
||||
const ASYNC_BACKING_DISABLED_ERROR: RuntimeApiError =
|
||||
RuntimeApiError::NotSupported { runtime_api_name: "test-runtime" };
|
||||
|
||||
fn dummy_pvd() -> PersistedValidationData {
|
||||
PersistedValidationData {
|
||||
parent_head: HeadData(vec![7, 8, 9]),
|
||||
relay_parent_number: 5,
|
||||
max_pov_size: 1024,
|
||||
relay_parent_storage_root: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct TestState {
|
||||
chain_ids: Vec<ParaId>,
|
||||
@@ -120,6 +135,7 @@ type VirtualOverseer = test_helpers::TestSubsystemContextHandle<CollatorProtocol
|
||||
|
||||
struct TestHarness {
|
||||
virtual_overseer: VirtualOverseer,
|
||||
keystore: KeystorePtr,
|
||||
}
|
||||
|
||||
fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
@@ -136,17 +152,17 @@ fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
|
||||
let (context, virtual_overseer) = test_helpers::make_subsystem_context(pool.clone());
|
||||
|
||||
let keystore = MemoryKeystore::new();
|
||||
keystore
|
||||
.sr25519_generate_new(
|
||||
polkadot_primitives::PARACHAIN_KEY_TYPE_ID,
|
||||
Some(&Sr25519Keyring::Alice.to_seed()),
|
||||
)
|
||||
.unwrap();
|
||||
let keystore = Arc::new(sc_keystore::LocalKeystore::in_memory());
|
||||
Keystore::sr25519_generate_new(
|
||||
&*keystore,
|
||||
polkadot_primitives::PARACHAIN_KEY_TYPE_ID,
|
||||
Some(&Sr25519Keyring::Alice.to_seed()),
|
||||
)
|
||||
.expect("Insert key into keystore");
|
||||
|
||||
let subsystem = run_inner(
|
||||
context,
|
||||
Arc::new(keystore),
|
||||
keystore.clone(),
|
||||
crate::CollatorEvictionPolicy {
|
||||
inactive_collator: ACTIVITY_TIMEOUT,
|
||||
undeclared: DECLARE_TIMEOUT,
|
||||
@@ -156,7 +172,7 @@ fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
REPUTATION_CHANGE_TEST_INTERVAL,
|
||||
);
|
||||
|
||||
let test_fut = test(TestHarness { virtual_overseer });
|
||||
let test_fut = test(TestHarness { virtual_overseer, keystore });
|
||||
|
||||
futures::pin_mut!(test_fut);
|
||||
futures::pin_mut!(subsystem);
|
||||
@@ -253,16 +269,53 @@ async fn assert_candidate_backing_second(
|
||||
expected_relay_parent: Hash,
|
||||
expected_para_id: ParaId,
|
||||
expected_pov: &PoV,
|
||||
mode: ProspectiveParachainsMode,
|
||||
) -> CandidateReceipt {
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
// Depending on relay parent mode pvd will be either requested
|
||||
// from the Runtime API or Prospective Parachains.
|
||||
let msg = overseer_recv(virtual_overseer).await;
|
||||
match mode {
|
||||
ProspectiveParachainsMode::Disabled => assert_matches!(
|
||||
msg,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(para_id, assumption, tx),
|
||||
)) => {
|
||||
assert_eq!(expected_relay_parent, hash);
|
||||
assert_eq!(expected_para_id, para_id);
|
||||
assert_eq!(OccupiedCoreAssumption::Free, assumption);
|
||||
tx.send(Ok(Some(pvd.clone()))).unwrap();
|
||||
}
|
||||
),
|
||||
ProspectiveParachainsMode::Enabled { .. } => assert_matches!(
|
||||
msg,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetProspectiveValidationData(request, tx),
|
||||
) => {
|
||||
assert_eq!(expected_relay_parent, request.candidate_relay_parent);
|
||||
assert_eq!(expected_para_id, request.para_id);
|
||||
tx.send(Some(pvd.clone())).unwrap();
|
||||
}
|
||||
),
|
||||
}
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(CandidateBackingMessage::Second(relay_parent, candidate_receipt, incoming_pov)
|
||||
) => {
|
||||
assert_eq!(expected_relay_parent, relay_parent);
|
||||
assert_eq!(expected_para_id, candidate_receipt.descriptor.para_id);
|
||||
assert_eq!(*expected_pov, incoming_pov);
|
||||
candidate_receipt
|
||||
})
|
||||
AllMessages::CandidateBacking(CandidateBackingMessage::Second(
|
||||
relay_parent,
|
||||
candidate_receipt,
|
||||
received_pvd,
|
||||
incoming_pov,
|
||||
)) => {
|
||||
assert_eq!(expected_relay_parent, relay_parent);
|
||||
assert_eq!(expected_para_id, candidate_receipt.descriptor.para_id);
|
||||
assert_eq!(*expected_pov, incoming_pov);
|
||||
assert_eq!(pvd, received_pvd);
|
||||
candidate_receipt
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
/// Assert that a collator got disconnected.
|
||||
@@ -284,6 +337,7 @@ async fn assert_fetch_collation_request(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
relay_parent: Hash,
|
||||
para_id: ParaId,
|
||||
candidate_hash: Option<CandidateHash>,
|
||||
) -> ResponseSender {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
@@ -291,14 +345,26 @@ async fn assert_fetch_collation_request(
|
||||
) => {
|
||||
let req = reqs.into_iter().next()
|
||||
.expect("There should be exactly one request");
|
||||
match req {
|
||||
Requests::CollationFetchingV1(req) => {
|
||||
let payload = req.payload;
|
||||
assert_eq!(payload.relay_parent, relay_parent);
|
||||
assert_eq!(payload.para_id, para_id);
|
||||
req.pending_response
|
||||
}
|
||||
_ => panic!("Unexpected request"),
|
||||
match candidate_hash {
|
||||
None => assert_matches!(
|
||||
req,
|
||||
Requests::CollationFetchingV1(req) => {
|
||||
let payload = req.payload;
|
||||
assert_eq!(payload.relay_parent, relay_parent);
|
||||
assert_eq!(payload.para_id, para_id);
|
||||
req.pending_response
|
||||
}
|
||||
),
|
||||
Some(candidate_hash) => assert_matches!(
|
||||
req,
|
||||
Requests::CollationFetchingVStaging(req) => {
|
||||
let payload = req.payload;
|
||||
assert_eq!(payload.relay_parent, relay_parent);
|
||||
assert_eq!(payload.para_id, para_id);
|
||||
assert_eq!(payload.candidate_hash, candidate_hash);
|
||||
req.pending_response
|
||||
}
|
||||
),
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -309,27 +375,38 @@ async fn connect_and_declare_collator(
|
||||
peer: PeerId,
|
||||
collator: CollatorPair,
|
||||
para_id: ParaId,
|
||||
version: CollationVersion,
|
||||
) {
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Full,
|
||||
CollationVersion::V1.into(),
|
||||
version.into(),
|
||||
None,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
Versioned::V1(protocol_v1::CollatorProtocolMessage::Declare(
|
||||
let wire_message = match version {
|
||||
CollationVersion::V1 => Versioned::V1(protocol_v1::CollatorProtocolMessage::Declare(
|
||||
collator.public(),
|
||||
para_id,
|
||||
collator.sign(&protocol_v1::declare_signature_payload(&peer)),
|
||||
)),
|
||||
CollationVersion::VStaging =>
|
||||
Versioned::VStaging(protocol_vstaging::CollatorProtocolMessage::Declare(
|
||||
collator.public(),
|
||||
para_id,
|
||||
collator.sign(&protocol_v1::declare_signature_payload(&peer)),
|
||||
)),
|
||||
};
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
wire_message,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
@@ -340,24 +417,48 @@ async fn advertise_collation(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
relay_parent: Hash,
|
||||
candidate: Option<(CandidateHash, Hash)>, // Candidate hash + parent head data hash.
|
||||
) {
|
||||
let wire_message = match candidate {
|
||||
Some((candidate_hash, parent_head_data_hash)) =>
|
||||
Versioned::VStaging(protocol_vstaging::CollatorProtocolMessage::AdvertiseCollation {
|
||||
relay_parent,
|
||||
candidate_hash,
|
||||
parent_head_data_hash,
|
||||
}),
|
||||
None =>
|
||||
Versioned::V1(protocol_v1::CollatorProtocolMessage::AdvertiseCollation(relay_parent)),
|
||||
};
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
Versioned::V1(protocol_v1::CollatorProtocolMessage::AdvertiseCollation(relay_parent)),
|
||||
wire_message,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn assert_async_backing_params_request(virtual_overseer: &mut VirtualOverseer, hash: Hash) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, hash);
|
||||
tx.send(Err(ASYNC_BACKING_DISABLED_ERROR)).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
// As we receive a relevant advertisement act on it and issue a collation request.
|
||||
#[test]
|
||||
fn act_on_advertisement() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
gum::trace!("activating");
|
||||
@@ -370,6 +471,7 @@ fn act_on_advertisement() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -379,15 +481,74 @@ fn act_on_advertisement() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
/// Tests that validator side works with vstaging network protocol
|
||||
/// before async backing is enabled.
|
||||
#[test]
|
||||
fn act_on_advertisement_vstaging() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
gum::trace!("activating");
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent],
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
// vstaging advertisement.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
test_state.relay_parent,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -401,7 +562,7 @@ fn collator_reporting_works() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
@@ -411,6 +572,8 @@ fn collator_reporting_works() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -421,6 +584,7 @@ fn collator_reporting_works() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -429,6 +593,7 @@ fn collator_reporting_works() {
|
||||
peer_c,
|
||||
test_state.collators[1].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -458,7 +623,7 @@ fn collator_authentication_verification_works() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
@@ -509,20 +674,24 @@ fn fetch_one_collation_at_a_time() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
let second = Hash::random();
|
||||
|
||||
let our_view = our_view![test_state.relay_parent, second];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent, second],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
// Iter over view since the order may change due to sorted invariant.
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
@@ -532,6 +701,7 @@ fn fetch_one_collation_at_a_time() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -540,16 +710,18 @@ fn fetch_one_collation_at_a_time() {
|
||||
peer_c,
|
||||
test_state.collators[1].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent, None).await;
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -563,10 +735,13 @@ fn fetch_one_collation_at_a_time() {
|
||||
dummy_candidate_receipt_bad_sig(dummy_hash(), Some(Default::default()));
|
||||
candidate_a.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate_a.descriptor.relay_parent = test_state.relay_parent;
|
||||
candidate_a.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
response_channel
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
assert_candidate_backing_second(
|
||||
@@ -574,6 +749,7 @@ fn fetch_one_collation_at_a_time() {
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Disabled,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -581,7 +757,7 @@ fn fetch_one_collation_at_a_time() {
|
||||
test_helpers::Yield::new().await;
|
||||
|
||||
// Second collation is not requested since there's already seconded one.
|
||||
assert_matches!(futures::poll!(virtual_overseer.recv().boxed()), Poll::Pending);
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
virtual_overseer
|
||||
})
|
||||
@@ -594,20 +770,24 @@ fn fetches_next_collation() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let second = Hash::random();
|
||||
|
||||
let our_view = our_view![test_state.relay_parent, second];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent, second],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
@@ -618,6 +798,7 @@ fn fetches_next_collation() {
|
||||
peer_b,
|
||||
test_state.collators[2].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -626,6 +807,7 @@ fn fetches_next_collation() {
|
||||
peer_c,
|
||||
test_state.collators[3].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -634,45 +816,64 @@ fn fetches_next_collation() {
|
||||
peer_d,
|
||||
test_state.collators[4].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, second).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, second).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_d, second).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, second, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, second, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_d, second, None).await;
|
||||
|
||||
// Dropping the response channel should lead to fetching the second collation.
|
||||
assert_fetch_collation_request(&mut virtual_overseer, second, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
let response_channel_non_exclusive =
|
||||
assert_fetch_collation_request(&mut virtual_overseer, second, test_state.chain_ids[0])
|
||||
.await;
|
||||
let response_channel_non_exclusive = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Third collator should receive response after that timeout:
|
||||
Delay::new(MAX_UNSHARED_DOWNLOAD_TIME + Duration::from_millis(50)).await;
|
||||
|
||||
let response_channel =
|
||||
assert_fetch_collation_request(&mut virtual_overseer, second, test_state.chain_ids[0])
|
||||
.await;
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1]) };
|
||||
let mut candidate_a =
|
||||
dummy_candidate_receipt_bad_sig(dummy_hash(), Some(Default::default()));
|
||||
candidate_a.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate_a.descriptor.relay_parent = second;
|
||||
candidate_a.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
|
||||
// First request finishes now:
|
||||
response_channel_non_exclusive
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
response_channel
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
assert_candidate_backing_second(
|
||||
@@ -680,6 +881,7 @@ fn fetches_next_collation() {
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Disabled,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -692,7 +894,7 @@ fn reject_connection_to_next_group() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
@@ -702,6 +904,7 @@ fn reject_connection_to_next_group() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -711,6 +914,7 @@ fn reject_connection_to_next_group() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[1], // next, not current `para_id`
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -737,20 +941,24 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let second = Hash::random();
|
||||
|
||||
let our_view = our_view![test_state.relay_parent, second];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent, second],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
@@ -760,6 +968,7 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -768,16 +977,18 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
peer_c,
|
||||
test_state.collators[1].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent, None).await;
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -786,10 +997,13 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
dummy_candidate_receipt_bad_sig(dummy_hash(), Some(Default::default()));
|
||||
candidate_a.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate_a.descriptor.relay_parent = test_state.relay_parent;
|
||||
candidate_a.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
response_channel
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
let receipt = assert_candidate_backing_second(
|
||||
@@ -797,6 +1011,7 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Disabled,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -822,6 +1037,7 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -834,7 +1050,7 @@ fn inactive_disconnected() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -848,6 +1064,7 @@ fn inactive_disconnected() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, hash_a).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -857,14 +1074,16 @@ fn inactive_disconnected() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -880,7 +1099,7 @@ fn activity_extends_life() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -888,18 +1107,20 @@ fn activity_extends_life() {
|
||||
let hash_b = Hash::repeat_byte(1);
|
||||
let hash_c = Hash::repeat_byte(2);
|
||||
|
||||
let our_view = our_view![hash_a, hash_b, hash_c];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![hash_a, hash_b, hash_c],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
// 3 heads, 3 times.
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
@@ -908,29 +1129,45 @@ fn activity_extends_life() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 2 / 3).await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_a).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_a, None).await;
|
||||
|
||||
assert_fetch_collation_request(&mut virtual_overseer, hash_a, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
hash_a,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 2 / 3).await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_b).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_b, None).await;
|
||||
|
||||
assert_fetch_collation_request(&mut virtual_overseer, hash_b, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
hash_b,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 2 / 3).await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_c).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_c, None).await;
|
||||
|
||||
assert_fetch_collation_request(&mut virtual_overseer, hash_c, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
hash_c,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 3 / 2).await;
|
||||
|
||||
@@ -945,7 +1182,7 @@ fn disconnect_if_no_declare() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
@@ -955,6 +1192,7 @@ fn disconnect_if_no_declare() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -981,7 +1219,7 @@ fn disconnect_if_wrong_declare() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -993,6 +1231,7 @@ fn disconnect_if_wrong_declare() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -1042,7 +1281,7 @@ fn delay_reputation_change() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| false), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -1054,6 +1293,7 @@ fn delay_reputation_change() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -1127,7 +1367,7 @@ fn view_change_clears_old_collators() {
|
||||
let mut test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -1139,6 +1379,7 @@ fn view_change_clears_old_collators() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -1148,6 +1389,7 @@ fn view_change_clears_old_collators() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -1162,6 +1404,7 @@ fn view_change_clears_old_collators() {
|
||||
.await;
|
||||
|
||||
test_state.group_rotation_info = test_state.group_rotation_info.bump_rotation();
|
||||
assert_async_backing_params_request(&mut virtual_overseer, hash_b).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
assert_collator_disconnect(&mut virtual_overseer, peer_b).await;
|
||||
+988
@@ -0,0 +1,988 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Tests for the validator side with enabled prospective parachains.
|
||||
|
||||
use super::*;
|
||||
|
||||
use polkadot_node_subsystem::messages::ChainApiMessage;
|
||||
use polkadot_primitives::{
|
||||
vstaging as vstaging_primitives, BlockNumber, CandidateCommitments, CommittedCandidateReceipt,
|
||||
Header, SigningContext, ValidatorId,
|
||||
};
|
||||
|
||||
const ASYNC_BACKING_PARAMETERS: vstaging_primitives::AsyncBackingParams =
|
||||
vstaging_primitives::AsyncBackingParams { max_candidate_depth: 4, allowed_ancestry_len: 3 };
|
||||
|
||||
fn get_parent_hash(hash: Hash) -> Hash {
|
||||
Hash::from_low_u64_be(hash.to_low_u64_be() + 1)
|
||||
}
|
||||
|
||||
async fn assert_assign_incoming(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
hash: Hash,
|
||||
number: BlockNumber,
|
||||
next_msg: &mut Option<AllMessages>,
|
||||
) {
|
||||
let msg = match next_msg.take() {
|
||||
Some(msg) => msg,
|
||||
None => overseer_recv(virtual_overseer).await,
|
||||
};
|
||||
assert_matches!(
|
||||
msg,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::Validators(tx))
|
||||
) if parent == hash => {
|
||||
tx.send(Ok(test_state.validator_public.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::ValidatorGroups(tx))
|
||||
) if parent == hash => {
|
||||
let validator_groups = test_state.validator_groups.clone();
|
||||
let mut group_rotation_info = test_state.group_rotation_info.clone();
|
||||
group_rotation_info.now = number;
|
||||
tx.send(Ok((validator_groups, group_rotation_info))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::AvailabilityCores(tx))
|
||||
) if parent == hash => {
|
||||
tx.send(Ok(test_state.cores.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/// Handle a view update.
|
||||
async fn update_view(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
new_view: Vec<(Hash, u32)>, // Hash and block number.
|
||||
activated: u8, // How many new heads does this update contain?
|
||||
) -> Option<AllMessages> {
|
||||
let new_view: HashMap<Hash, u32> = HashMap::from_iter(new_view);
|
||||
|
||||
let our_view =
|
||||
OurView::new(new_view.keys().map(|hash| (*hash, Arc::new(jaeger::Span::Disabled))), 0);
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(our_view)),
|
||||
)
|
||||
.await;
|
||||
|
||||
let mut next_overseer_message = None;
|
||||
for _ in 0..activated {
|
||||
let (leaf_hash, leaf_number) = assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx),
|
||||
)) => {
|
||||
tx.send(Ok(ASYNC_BACKING_PARAMETERS)).unwrap();
|
||||
(parent, new_view.get(&parent).copied().expect("Unknown parent requested"))
|
||||
}
|
||||
);
|
||||
|
||||
assert_assign_incoming(
|
||||
virtual_overseer,
|
||||
test_state,
|
||||
leaf_hash,
|
||||
leaf_number,
|
||||
&mut next_overseer_message,
|
||||
)
|
||||
.await;
|
||||
|
||||
let min_number = leaf_number.saturating_sub(ASYNC_BACKING_PARAMETERS.allowed_ancestry_len);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetMinimumRelayParents(parent, tx),
|
||||
) if parent == leaf_hash => {
|
||||
tx.send(test_state.chain_ids.iter().map(|para_id| (*para_id, min_number)).collect()).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let ancestry_len = leaf_number + 1 - min_number;
|
||||
let ancestry_hashes = std::iter::successors(Some(leaf_hash), |h| Some(get_parent_hash(*h)))
|
||||
.take(ancestry_len as usize);
|
||||
let ancestry_numbers = (min_number..=leaf_number).rev();
|
||||
let ancestry_iter = ancestry_hashes.clone().zip(ancestry_numbers).peekable();
|
||||
|
||||
// How many blocks were actually requested.
|
||||
let mut requested_len: usize = 0;
|
||||
{
|
||||
let mut ancestry_iter = ancestry_iter.clone();
|
||||
while let Some((hash, number)) = ancestry_iter.next() {
|
||||
// May be `None` for the last element.
|
||||
let parent_hash =
|
||||
ancestry_iter.peek().map(|(h, _)| *h).unwrap_or_else(|| get_parent_hash(hash));
|
||||
|
||||
let msg = match next_overseer_message.take() {
|
||||
Some(msg) => msg,
|
||||
None => overseer_recv(virtual_overseer).await,
|
||||
};
|
||||
|
||||
if !matches!(&msg, AllMessages::ChainApi(ChainApiMessage::BlockHeader(..))) {
|
||||
// Ancestry has already been cached for this leaf.
|
||||
next_overseer_message.replace(msg);
|
||||
break
|
||||
}
|
||||
|
||||
assert_matches!(
|
||||
msg,
|
||||
AllMessages::ChainApi(ChainApiMessage::BlockHeader(.., tx)) => {
|
||||
let header = Header {
|
||||
parent_hash,
|
||||
number,
|
||||
state_root: Hash::zero(),
|
||||
extrinsics_root: Hash::zero(),
|
||||
digest: Default::default(),
|
||||
};
|
||||
|
||||
tx.send(Ok(Some(header))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
requested_len += 1;
|
||||
}
|
||||
}
|
||||
|
||||
// Skip the leaf.
|
||||
for (hash, number) in ancestry_iter.skip(1).take(requested_len.saturating_sub(1)) {
|
||||
assert_assign_incoming(
|
||||
virtual_overseer,
|
||||
test_state,
|
||||
hash,
|
||||
number,
|
||||
&mut next_overseer_message,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
next_overseer_message
|
||||
}
|
||||
|
||||
async fn send_seconded_statement(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
keystore: KeystorePtr,
|
||||
candidate: &CommittedCandidateReceipt,
|
||||
) {
|
||||
let signing_context = SigningContext { session_index: 0, parent_hash: Hash::zero() };
|
||||
let stmt = SignedFullStatement::sign(
|
||||
&keystore,
|
||||
Statement::Seconded(candidate.clone()),
|
||||
&signing_context,
|
||||
ValidatorIndex(0),
|
||||
&ValidatorId::from(Sr25519Keyring::Alice.public()),
|
||||
)
|
||||
.ok()
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::Seconded(candidate.descriptor.relay_parent, stmt),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn assert_collation_seconded(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
relay_parent: Hash,
|
||||
peer_id: PeerId,
|
||||
) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::ReportPeer(
|
||||
ReportPeerMessage::Single(peer, rep)
|
||||
)) => {
|
||||
assert_eq!(peer_id, peer);
|
||||
assert_eq!(rep.value, BENEFIT_NOTIFY_GOOD.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendCollationMessage(
|
||||
peers,
|
||||
Versioned::VStaging(protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
protocol_vstaging::CollatorProtocolMessage::CollationSeconded(
|
||||
_relay_parent,
|
||||
..,
|
||||
),
|
||||
)),
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_id]);
|
||||
assert_eq!(relay_parent, _relay_parent);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn v1_advertisement_rejected() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 0;
|
||||
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
// Accept both collators from the implicit view.
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_a, head_b, None).await;
|
||||
|
||||
// Not reported.
|
||||
test_helpers::Yield::new().await;
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accept_advertisements_from_implicit_view() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
let pair_b = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
let head_c = get_parent_hash(head_b);
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `d` we're assigned
|
||||
// to the first para.
|
||||
let head_d = get_parent_hash(head_c);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
// Accept both collators from the implicit view.
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair_b.clone(),
|
||||
test_state.chain_ids[1],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[1]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[1],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
// Advertise with different para.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_d, // Note different relay parent.
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_d,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn second_multiple_candidates_per_relay_parent() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, keystore } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `c` we're assigned
|
||||
// to the first para.
|
||||
let head_c = Hash::from_low_u64_be(130);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
for i in 0..(ASYNC_BACKING_PARAMETERS.max_candidate_depth + 1) {
|
||||
let mut candidate = dummy_candidate_receipt_bad_sig(head_c, Some(Default::default()));
|
||||
candidate.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
let commitments = CandidateCommitments {
|
||||
head_data: HeadData(vec![i as u8]),
|
||||
horizontal_messages: Default::default(),
|
||||
upward_messages: Default::default(),
|
||||
new_validation_code: None,
|
||||
processed_downward_messages: 0,
|
||||
hrmp_watermark: 0,
|
||||
};
|
||||
candidate.commitments_hash = commitments.hash();
|
||||
|
||||
let candidate_hash = candidate.hash();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1]) };
|
||||
|
||||
response_channel
|
||||
.send(Ok(request_vstaging::CollationFetchingResponse::Collation(
|
||||
candidate.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
assert_candidate_backing_second(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Enabled {
|
||||
max_candidate_depth: ASYNC_BACKING_PARAMETERS.max_candidate_depth as _,
|
||||
allowed_ancestry_len: ASYNC_BACKING_PARAMETERS.allowed_ancestry_len as _,
|
||||
},
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate =
|
||||
CommittedCandidateReceipt { descriptor: candidate.descriptor, commitments };
|
||||
|
||||
send_seconded_statement(&mut virtual_overseer, keystore.clone(), &candidate).await;
|
||||
|
||||
assert_collation_seconded(&mut virtual_overseer, head_c, peer_a).await;
|
||||
}
|
||||
|
||||
// No more advertisements can be made for this relay parent.
|
||||
let candidate_hash = CandidateHash(Hash::repeat_byte(0xAA));
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, Hash::zero())),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Reported because reached the limit of advertisements per relay parent.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer_id, rep)),
|
||||
) => {
|
||||
assert_eq!(peer_a, peer_id);
|
||||
assert_eq!(rep.value, COST_UNEXPECTED_MESSAGE.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
|
||||
// By different peer too (not reported).
|
||||
let pair_b = CollatorPair::generate().0;
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair_b.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash(Hash::repeat_byte(0xFF));
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
head_c,
|
||||
Some((candidate_hash, Hash::zero())),
|
||||
)
|
||||
.await;
|
||||
|
||||
test_helpers::Yield::new().await;
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fetched_collation_sanity_check() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `c` we're assigned
|
||||
// to the first para.
|
||||
let head_c = Hash::from_low_u64_be(130);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let mut candidate = dummy_candidate_receipt_bad_sig(head_c, Some(Default::default()));
|
||||
candidate.descriptor.para_id = test_state.chain_ids[0];
|
||||
let commitments = CandidateCommitments {
|
||||
head_data: HeadData(vec![1, 2, 3]),
|
||||
horizontal_messages: Default::default(),
|
||||
upward_messages: Default::default(),
|
||||
new_validation_code: None,
|
||||
processed_downward_messages: 0,
|
||||
hrmp_watermark: 0,
|
||||
};
|
||||
candidate.commitments_hash = commitments.hash();
|
||||
|
||||
let candidate_hash = CandidateHash(Hash::zero());
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1]) };
|
||||
|
||||
response_channel
|
||||
.send(Ok(request_vstaging::CollationFetchingResponse::Collation(
|
||||
candidate.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
// PVD request.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetProspectiveValidationData(request, tx),
|
||||
) => {
|
||||
assert_eq!(head_c, request.candidate_relay_parent);
|
||||
assert_eq!(test_state.chain_ids[0], request.para_id);
|
||||
tx.send(Some(dummy_pvd())).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
// Reported malicious.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer_id, rep)),
|
||||
) => {
|
||||
assert_eq!(peer_a, peer_id);
|
||||
assert_eq!(rep.value, COST_REPORT_BAD.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn advertisement_spam_protection() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
let head_c = get_parent_hash(head_b);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[1],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[1]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
// Reject it.
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
// Send the same advertisement again.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
// Reported.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer_id, rep)),
|
||||
) => {
|
||||
assert_eq!(peer_a, peer_id);
|
||||
assert_eq!(rep.value, COST_UNEXPECTED_MESSAGE.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn backed_candidate_unblocks_advertisements() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
let pair_b = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
let head_c = get_parent_hash(head_b);
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `d` we're assigned
|
||||
// to the first para.
|
||||
let head_d = get_parent_hash(head_c);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
// Accept both collators from the implicit view.
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair_b.clone(),
|
||||
test_state.chain_ids[1],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[1]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
// Reject it.
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
// Advertise with different para.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_d, // Note different relay parent.
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::Backed {
|
||||
para_id: test_state.chain_ids[0],
|
||||
para_head: parent_head_data_hash,
|
||||
},
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_d,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn active_leave_unblocks_advertisements() {
|
||||
let mut test_state = TestState::default();
|
||||
test_state.group_rotation_info.group_rotation_frequency = 100;
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 0;
|
||||
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peers: Vec<CollatorPair> = (0..3).map(|_| CollatorPair::generate().0).collect();
|
||||
let peer_ids: Vec<PeerId> = (0..3).map(|_| PeerId::random()).collect();
|
||||
let candidates: Vec<CandidateHash> =
|
||||
(0u8..3).map(|i| CandidateHash(Hash::repeat_byte(i))).collect();
|
||||
|
||||
for (peer, peer_id) in peers.iter().zip(&peer_ids) {
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
*peer_id,
|
||||
peer.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
for (peer, candidate) in peer_ids.iter().zip(&candidates).take(2) {
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
*peer,
|
||||
head_b,
|
||||
Some((*candidate, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, *candidate);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
// Send false.
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
let head_c = Hash::from_low_u64_be(127);
|
||||
let head_c_num: u32 = 1;
|
||||
|
||||
let next_overseer_message =
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_c, head_c_num)], 1)
|
||||
.await
|
||||
.expect("should've sent request to backing");
|
||||
|
||||
// Unblock first request.
|
||||
assert_matches!(
|
||||
next_overseer_message,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[0]);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_b,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidates[0]),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[1]);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
// Collation request was discarded.
|
||||
test_helpers::Yield::new().await;
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_ids[2],
|
||||
head_c,
|
||||
Some((candidates[2], parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[2]);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
let head_d = Hash::from_low_u64_be(126);
|
||||
let head_d_num: u32 = 2;
|
||||
|
||||
let next_overseer_message =
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_d, head_d_num)], 1)
|
||||
.await
|
||||
.expect("should've sent request to backing");
|
||||
|
||||
// Reject 2, accept 3.
|
||||
assert_matches!(
|
||||
next_overseer_message,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[1]);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[2]);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidates[2]),
|
||||
)
|
||||
.await;
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
@@ -448,8 +448,12 @@ where
|
||||
NetworkBridgeEvent::OurViewChange(_) => {},
|
||||
NetworkBridgeEvent::PeerViewChange(_, _) => {},
|
||||
NetworkBridgeEvent::NewGossipTopology { .. } => {},
|
||||
NetworkBridgeEvent::PeerMessage(_, Versioned::V1(v)) => {
|
||||
match v {};
|
||||
NetworkBridgeEvent::PeerMessage(_, message) => {
|
||||
// match void -> LLVM unreachable
|
||||
match message {
|
||||
Versioned::V1(m) => match m {},
|
||||
Versioned::VStaging(m) => match m {},
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -23,6 +23,10 @@ fatality = "0.0.6"
|
||||
rand = "0.8"
|
||||
derive_more = "0.99"
|
||||
gum = { package = "tracing-gum", path = "../../gum" }
|
||||
bitvec = "1"
|
||||
|
||||
[dev-dependencies]
|
||||
rand_chacha = "0.3.1"
|
||||
|
||||
[features]
|
||||
network-protocol-staging = []
|
||||
|
||||
@@ -253,22 +253,26 @@ impl View {
|
||||
|
||||
/// A protocol-versioned type.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Versioned<V1> {
|
||||
pub enum Versioned<V1, VStaging> {
|
||||
/// V1 type.
|
||||
V1(V1),
|
||||
/// VStaging type.
|
||||
VStaging(VStaging),
|
||||
}
|
||||
|
||||
impl<V1: Clone> Versioned<&'_ V1> {
|
||||
impl<V1: Clone, VStaging: Clone> Versioned<&'_ V1, &'_ VStaging> {
|
||||
/// Convert to a fully-owned version of the message.
|
||||
pub fn clone_inner(&self) -> Versioned<V1> {
|
||||
pub fn clone_inner(&self) -> Versioned<V1, VStaging> {
|
||||
match *self {
|
||||
Versioned::V1(inner) => Versioned::V1(inner.clone()),
|
||||
Versioned::VStaging(inner) => Versioned::VStaging(inner.clone()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// All supported versions of the validation protocol message.
|
||||
pub type VersionedValidationProtocol = Versioned<v1::ValidationProtocol>;
|
||||
pub type VersionedValidationProtocol =
|
||||
Versioned<v1::ValidationProtocol, vstaging::ValidationProtocol>;
|
||||
|
||||
impl From<v1::ValidationProtocol> for VersionedValidationProtocol {
|
||||
fn from(v1: v1::ValidationProtocol) -> Self {
|
||||
@@ -276,8 +280,14 @@ impl From<v1::ValidationProtocol> for VersionedValidationProtocol {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<vstaging::ValidationProtocol> for VersionedValidationProtocol {
|
||||
fn from(vstaging: vstaging::ValidationProtocol) -> Self {
|
||||
VersionedValidationProtocol::VStaging(vstaging)
|
||||
}
|
||||
}
|
||||
|
||||
/// All supported versions of the collation protocol message.
|
||||
pub type VersionedCollationProtocol = Versioned<v1::CollationProtocol>;
|
||||
pub type VersionedCollationProtocol = Versioned<v1::CollationProtocol, vstaging::CollationProtocol>;
|
||||
|
||||
impl From<v1::CollationProtocol> for VersionedCollationProtocol {
|
||||
fn from(v1: v1::CollationProtocol) -> Self {
|
||||
@@ -285,12 +295,19 @@ impl From<v1::CollationProtocol> for VersionedCollationProtocol {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<vstaging::CollationProtocol> for VersionedCollationProtocol {
|
||||
fn from(vstaging: vstaging::CollationProtocol) -> Self {
|
||||
VersionedCollationProtocol::VStaging(vstaging)
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! impl_versioned_full_protocol_from {
|
||||
($from:ty, $out:ty, $variant:ident) => {
|
||||
impl From<$from> for $out {
|
||||
fn from(versioned_from: $from) -> $out {
|
||||
match versioned_from {
|
||||
Versioned::V1(x) => Versioned::V1(x.into()),
|
||||
Versioned::VStaging(x) => Versioned::VStaging(x.into()),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -300,7 +317,12 @@ macro_rules! impl_versioned_full_protocol_from {
|
||||
/// Implement `TryFrom` for one versioned enum variant into the inner type.
|
||||
/// `$m_ty::$variant(inner) -> Ok(inner)`
|
||||
macro_rules! impl_versioned_try_from {
|
||||
($from:ty, $out:ty, $v1_pat:pat => $v1_out:expr) => {
|
||||
(
|
||||
$from:ty,
|
||||
$out:ty,
|
||||
$v1_pat:pat => $v1_out:expr,
|
||||
$vstaging_pat:pat => $vstaging_out:expr
|
||||
) => {
|
||||
impl TryFrom<$from> for $out {
|
||||
type Error = crate::WrongVariant;
|
||||
|
||||
@@ -308,6 +330,7 @@ macro_rules! impl_versioned_try_from {
|
||||
#[allow(unreachable_patterns)] // when there is only one variant
|
||||
match x {
|
||||
Versioned::V1($v1_pat) => Ok(Versioned::V1($v1_out)),
|
||||
Versioned::VStaging($vstaging_pat) => Ok(Versioned::VStaging($vstaging_out)),
|
||||
_ => Err(crate::WrongVariant),
|
||||
}
|
||||
}
|
||||
@@ -320,6 +343,8 @@ macro_rules! impl_versioned_try_from {
|
||||
#[allow(unreachable_patterns)] // when there is only one variant
|
||||
match x {
|
||||
Versioned::V1($v1_pat) => Ok(Versioned::V1($v1_out.clone())),
|
||||
Versioned::VStaging($vstaging_pat) =>
|
||||
Ok(Versioned::VStaging($vstaging_out.clone())),
|
||||
_ => Err(crate::WrongVariant),
|
||||
}
|
||||
}
|
||||
@@ -328,7 +353,8 @@ macro_rules! impl_versioned_try_from {
|
||||
}
|
||||
|
||||
/// Version-annotated messages used by the bitfield distribution subsystem.
|
||||
pub type BitfieldDistributionMessage = Versioned<v1::BitfieldDistributionMessage>;
|
||||
pub type BitfieldDistributionMessage =
|
||||
Versioned<v1::BitfieldDistributionMessage, vstaging::BitfieldDistributionMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
BitfieldDistributionMessage,
|
||||
VersionedValidationProtocol,
|
||||
@@ -337,11 +363,13 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedValidationProtocol,
|
||||
BitfieldDistributionMessage,
|
||||
v1::ValidationProtocol::BitfieldDistribution(x) => x
|
||||
v1::ValidationProtocol::BitfieldDistribution(x) => x,
|
||||
vstaging::ValidationProtocol::BitfieldDistribution(x) => x
|
||||
);
|
||||
|
||||
/// Version-annotated messages used by the statement distribution subsystem.
|
||||
pub type StatementDistributionMessage = Versioned<v1::StatementDistributionMessage>;
|
||||
pub type StatementDistributionMessage =
|
||||
Versioned<v1::StatementDistributionMessage, vstaging::StatementDistributionMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
StatementDistributionMessage,
|
||||
VersionedValidationProtocol,
|
||||
@@ -350,11 +378,13 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedValidationProtocol,
|
||||
StatementDistributionMessage,
|
||||
v1::ValidationProtocol::StatementDistribution(x) => x
|
||||
v1::ValidationProtocol::StatementDistribution(x) => x,
|
||||
vstaging::ValidationProtocol::StatementDistribution(x) => x
|
||||
);
|
||||
|
||||
/// Version-annotated messages used by the approval distribution subsystem.
|
||||
pub type ApprovalDistributionMessage = Versioned<v1::ApprovalDistributionMessage>;
|
||||
pub type ApprovalDistributionMessage =
|
||||
Versioned<v1::ApprovalDistributionMessage, vstaging::ApprovalDistributionMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
ApprovalDistributionMessage,
|
||||
VersionedValidationProtocol,
|
||||
@@ -363,11 +393,14 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedValidationProtocol,
|
||||
ApprovalDistributionMessage,
|
||||
v1::ValidationProtocol::ApprovalDistribution(x) => x
|
||||
v1::ValidationProtocol::ApprovalDistribution(x) => x,
|
||||
vstaging::ValidationProtocol::ApprovalDistribution(x) => x
|
||||
|
||||
);
|
||||
|
||||
/// Version-annotated messages used by the gossip-support subsystem (this is void).
|
||||
pub type GossipSupportNetworkMessage = Versioned<v1::GossipSupportNetworkMessage>;
|
||||
pub type GossipSupportNetworkMessage =
|
||||
Versioned<v1::GossipSupportNetworkMessage, vstaging::GossipSupportNetworkMessage>;
|
||||
// This is a void enum placeholder, so never gets sent over the wire.
|
||||
impl TryFrom<VersionedValidationProtocol> for GossipSupportNetworkMessage {
|
||||
type Error = WrongVariant;
|
||||
@@ -384,7 +417,8 @@ impl<'a> TryFrom<&'a VersionedValidationProtocol> for GossipSupportNetworkMessag
|
||||
}
|
||||
|
||||
/// Version-annotated messages used by the bitfield distribution subsystem.
|
||||
pub type CollatorProtocolMessage = Versioned<v1::CollatorProtocolMessage>;
|
||||
pub type CollatorProtocolMessage =
|
||||
Versioned<v1::CollatorProtocolMessage, vstaging::CollatorProtocolMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
CollatorProtocolMessage,
|
||||
VersionedCollationProtocol,
|
||||
@@ -393,7 +427,8 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedCollationProtocol,
|
||||
CollatorProtocolMessage,
|
||||
v1::CollationProtocol::CollatorProtocol(x) => x
|
||||
v1::CollationProtocol::CollatorProtocol(x) => x,
|
||||
vstaging::CollationProtocol::CollatorProtocol(x) => x
|
||||
);
|
||||
|
||||
/// v1 notification protocol types.
|
||||
@@ -553,3 +588,256 @@ pub mod v1 {
|
||||
payload
|
||||
}
|
||||
}
|
||||
|
||||
/// vstaging network protocol types.
|
||||
pub mod vstaging {
|
||||
use bitvec::{order::Lsb0, slice::BitSlice, vec::BitVec};
|
||||
use parity_scale_codec::{Decode, Encode};
|
||||
|
||||
use polkadot_primitives::vstaging::{
|
||||
CandidateHash, CandidateIndex, CollatorId, CollatorSignature, GroupIndex, Hash,
|
||||
Id as ParaId, UncheckedSignedAvailabilityBitfield, UncheckedSignedStatement,
|
||||
};
|
||||
|
||||
use polkadot_node_primitives::{
|
||||
approval::{IndirectAssignmentCert, IndirectSignedApprovalVote},
|
||||
UncheckedSignedFullStatement,
|
||||
};
|
||||
|
||||
/// Network messages used by the bitfield distribution subsystem.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum BitfieldDistributionMessage {
|
||||
/// A signed availability bitfield for a given relay-parent hash.
|
||||
#[codec(index = 0)]
|
||||
Bitfield(Hash, UncheckedSignedAvailabilityBitfield),
|
||||
}
|
||||
|
||||
/// Bitfields indicating the statements that are known or undesired
|
||||
/// about a candidate.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub struct StatementFilter {
|
||||
/// Seconded statements. '1' is known or undesired.
|
||||
pub seconded_in_group: BitVec<u8, Lsb0>,
|
||||
/// Valid statements. '1' is known or undesired.
|
||||
pub validated_in_group: BitVec<u8, Lsb0>,
|
||||
}
|
||||
|
||||
impl StatementFilter {
|
||||
/// Create a new blank filter with the given group size.
|
||||
pub fn blank(group_size: usize) -> Self {
|
||||
StatementFilter {
|
||||
seconded_in_group: BitVec::repeat(false, group_size),
|
||||
validated_in_group: BitVec::repeat(false, group_size),
|
||||
}
|
||||
}
|
||||
|
||||
/// Create a new full filter with the given group size.
|
||||
pub fn full(group_size: usize) -> Self {
|
||||
StatementFilter {
|
||||
seconded_in_group: BitVec::repeat(true, group_size),
|
||||
validated_in_group: BitVec::repeat(true, group_size),
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the filter has a specific expected length, consistent across both
|
||||
/// bitfields.
|
||||
pub fn has_len(&self, len: usize) -> bool {
|
||||
self.seconded_in_group.len() == len && self.validated_in_group.len() == len
|
||||
}
|
||||
|
||||
/// Determine the number of backing validators in the statement filter.
|
||||
pub fn backing_validators(&self) -> usize {
|
||||
self.seconded_in_group
|
||||
.iter()
|
||||
.by_vals()
|
||||
.zip(self.validated_in_group.iter().by_vals())
|
||||
.filter(|&(s, v)| s || v) // no double-counting
|
||||
.count()
|
||||
}
|
||||
|
||||
/// Whether the statement filter has at least one seconded statement.
|
||||
pub fn has_seconded(&self) -> bool {
|
||||
self.seconded_in_group.iter().by_vals().any(|x| x)
|
||||
}
|
||||
|
||||
/// Mask out `Seconded` statements in `self` according to the provided
|
||||
/// bitvec. Bits appearing in `mask` will not appear in `self` afterwards.
|
||||
pub fn mask_seconded(&mut self, mask: &BitSlice<u8, Lsb0>) {
|
||||
for (mut x, mask) in self
|
||||
.seconded_in_group
|
||||
.iter_mut()
|
||||
.zip(mask.iter().by_vals().chain(std::iter::repeat(false)))
|
||||
{
|
||||
// (x, mask) => x
|
||||
// (true, true) => false
|
||||
// (true, false) => true
|
||||
// (false, true) => false
|
||||
// (false, false) => false
|
||||
*x = *x && !mask;
|
||||
}
|
||||
}
|
||||
|
||||
/// Mask out `Valid` statements in `self` according to the provided
|
||||
/// bitvec. Bits appearing in `mask` will not appear in `self` afterwards.
|
||||
pub fn mask_valid(&mut self, mask: &BitSlice<u8, Lsb0>) {
|
||||
for (mut x, mask) in self
|
||||
.validated_in_group
|
||||
.iter_mut()
|
||||
.zip(mask.iter().by_vals().chain(std::iter::repeat(false)))
|
||||
{
|
||||
// (x, mask) => x
|
||||
// (true, true) => false
|
||||
// (true, false) => true
|
||||
// (false, true) => false
|
||||
// (false, false) => false
|
||||
*x = *x && !mask;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A manifest of a known backed candidate, along with a description
|
||||
/// of the statements backing it.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub struct BackedCandidateManifest {
|
||||
/// The relay-parent of the candidate.
|
||||
pub relay_parent: Hash,
|
||||
/// The hash of the candidate.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// The group index backing the candidate at the relay-parent.
|
||||
pub group_index: GroupIndex,
|
||||
/// The para ID of the candidate. It is illegal for this to
|
||||
/// be a para ID which is not assigned to the group indicated
|
||||
/// in this manifest.
|
||||
pub para_id: ParaId,
|
||||
/// The head-data corresponding to the candidate.
|
||||
pub parent_head_data_hash: Hash,
|
||||
/// A statement filter which indicates which validators in the
|
||||
/// para's group at the relay-parent have validated this candidate
|
||||
/// and issued statements about it, to the advertiser's knowledge.
|
||||
///
|
||||
/// This MUST have exactly the minimum amount of bytes
|
||||
/// necessary to represent the number of validators in the assigned
|
||||
/// backing group as-of the relay-parent.
|
||||
pub statement_knowledge: StatementFilter,
|
||||
}
|
||||
|
||||
/// An acknowledgement of a backed candidate being known.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub struct BackedCandidateAcknowledgement {
|
||||
/// The hash of the candidate.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// A statement filter which indicates which validators in the
|
||||
/// para's group at the relay-parent have validated this candidate
|
||||
/// and issued statements about it, to the advertiser's knowledge.
|
||||
///
|
||||
/// This MUST have exactly the minimum amount of bytes
|
||||
/// necessary to represent the number of validators in the assigned
|
||||
/// backing group as-of the relay-parent.
|
||||
pub statement_knowledge: StatementFilter,
|
||||
}
|
||||
|
||||
/// Network messages used by the statement distribution subsystem.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum StatementDistributionMessage {
|
||||
/// A notification of a signed statement in compact form, for a given relay parent.
|
||||
#[codec(index = 0)]
|
||||
Statement(Hash, UncheckedSignedStatement),
|
||||
|
||||
/// A notification of a backed candidate being known by the
|
||||
/// sending node, for the purpose of being requested by the receiving node
|
||||
/// if needed.
|
||||
#[codec(index = 1)]
|
||||
BackedCandidateManifest(BackedCandidateManifest),
|
||||
|
||||
/// A notification of a backed candidate being known by the sending node,
|
||||
/// for the purpose of informing a receiving node which already has the candidate.
|
||||
#[codec(index = 2)]
|
||||
BackedCandidateKnown(BackedCandidateAcknowledgement),
|
||||
|
||||
/// All messages for V1 for compatibility with the statement distribution
|
||||
/// protocol, for relay-parents that don't support asynchronous backing.
|
||||
///
|
||||
/// These are illegal to send to V1 peers, and illegal to send concerning relay-parents
|
||||
/// which support asynchronous backing. This backwards compatibility should be
|
||||
/// considered immediately deprecated and can be removed once the node software
|
||||
/// is not required to support logic from before asynchronous backing anymore.
|
||||
#[codec(index = 255)]
|
||||
V1Compatibility(crate::v1::StatementDistributionMessage),
|
||||
}
|
||||
|
||||
/// Network messages used by the approval distribution subsystem.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum ApprovalDistributionMessage {
|
||||
/// Assignments for candidates in recent, unfinalized blocks.
|
||||
///
|
||||
/// Actually checking the assignment may yield a different result.
|
||||
#[codec(index = 0)]
|
||||
Assignments(Vec<(IndirectAssignmentCert, CandidateIndex)>),
|
||||
/// Approvals for candidates in some recent, unfinalized block.
|
||||
#[codec(index = 1)]
|
||||
Approvals(Vec<IndirectSignedApprovalVote>),
|
||||
}
|
||||
|
||||
/// Dummy network message type, so we will receive connect/disconnect events.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum GossipSupportNetworkMessage {}
|
||||
|
||||
/// Network messages used by the collator protocol subsystem
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum CollatorProtocolMessage {
|
||||
/// Declare the intent to advertise collations under a collator ID, attaching a
|
||||
/// signature of the `PeerId` of the node using the given collator ID key.
|
||||
#[codec(index = 0)]
|
||||
Declare(CollatorId, ParaId, CollatorSignature),
|
||||
/// Advertise a collation to a validator. Can only be sent once the peer has
|
||||
/// declared that they are a collator with given ID.
|
||||
#[codec(index = 1)]
|
||||
AdvertiseCollation {
|
||||
/// Hash of the relay parent advertised collation is based on.
|
||||
relay_parent: Hash,
|
||||
/// Candidate hash.
|
||||
candidate_hash: CandidateHash,
|
||||
/// Parachain head data hash before candidate execution.
|
||||
parent_head_data_hash: Hash,
|
||||
},
|
||||
/// A collation sent to a validator was seconded.
|
||||
#[codec(index = 4)]
|
||||
CollationSeconded(Hash, UncheckedSignedFullStatement),
|
||||
}
|
||||
|
||||
/// All network messages on the validation peer-set.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq, derive_more::From)]
|
||||
pub enum ValidationProtocol {
|
||||
/// Bitfield distribution messages
|
||||
#[codec(index = 1)]
|
||||
#[from]
|
||||
BitfieldDistribution(BitfieldDistributionMessage),
|
||||
/// Statement distribution messages
|
||||
#[codec(index = 3)]
|
||||
#[from]
|
||||
StatementDistribution(StatementDistributionMessage),
|
||||
/// Approval distribution messages
|
||||
#[codec(index = 4)]
|
||||
#[from]
|
||||
ApprovalDistribution(ApprovalDistributionMessage),
|
||||
}
|
||||
|
||||
/// All network messages on the collation peer-set.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq, derive_more::From)]
|
||||
pub enum CollationProtocol {
|
||||
/// Collator protocol messages
|
||||
#[codec(index = 0)]
|
||||
#[from]
|
||||
CollatorProtocol(CollatorProtocolMessage),
|
||||
}
|
||||
|
||||
/// Get the payload that should be signed and included in a `Declare` message.
|
||||
///
|
||||
/// The payload is the local peer id of the node, which serves to prove that it
|
||||
/// controls the collator key it is declaring an intention to collate under.
|
||||
pub fn declare_signature_payload(peer_id: &sc_network::PeerId) -> Vec<u8> {
|
||||
let mut payload = peer_id.to_bytes();
|
||||
payload.extend_from_slice(b"COLL");
|
||||
payload
|
||||
}
|
||||
}
|
||||
|
||||
@@ -118,10 +118,17 @@ impl PeerSet {
|
||||
/// Networking layer relies on `get_main_version()` being the version
|
||||
/// of the main protocol name reported by [`PeerSetProtocolNames::get_main_name()`].
|
||||
pub fn get_main_version(self) -> ProtocolVersion {
|
||||
#[cfg(not(feature = "network-protocol-staging"))]
|
||||
match self {
|
||||
PeerSet::Validation => ValidationVersion::V1.into(),
|
||||
PeerSet::Collation => CollationVersion::V1.into(),
|
||||
}
|
||||
|
||||
#[cfg(feature = "network-protocol-staging")]
|
||||
match self {
|
||||
PeerSet::Validation => ValidationVersion::VStaging.into(),
|
||||
PeerSet::Collation => CollationVersion::VStaging.into(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Get the max notification size for this peer set.
|
||||
@@ -145,12 +152,16 @@ impl PeerSet {
|
||||
PeerSet::Validation =>
|
||||
if version == ValidationVersion::V1.into() {
|
||||
Some("validation/1")
|
||||
} else if version == ValidationVersion::VStaging.into() {
|
||||
Some("validation/2")
|
||||
} else {
|
||||
None
|
||||
},
|
||||
PeerSet::Collation =>
|
||||
if version == CollationVersion::V1.into() {
|
||||
Some("collation/1")
|
||||
} else if version == CollationVersion::VStaging.into() {
|
||||
Some("collation/2")
|
||||
} else {
|
||||
None
|
||||
},
|
||||
@@ -212,6 +223,8 @@ impl From<ProtocolVersion> for u32 {
|
||||
pub enum ValidationVersion {
|
||||
/// The first version.
|
||||
V1 = 1,
|
||||
/// The staging version.
|
||||
VStaging = 2,
|
||||
}
|
||||
|
||||
/// Supported collation protocol versions. Only versions defined here must be used in the codebase.
|
||||
@@ -219,6 +232,40 @@ pub enum ValidationVersion {
|
||||
pub enum CollationVersion {
|
||||
/// The first version.
|
||||
V1 = 1,
|
||||
/// The staging version.
|
||||
VStaging = 2,
|
||||
}
|
||||
|
||||
/// Marker indicating the version is unknown.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct UnknownVersion;
|
||||
|
||||
impl TryFrom<ProtocolVersion> for ValidationVersion {
|
||||
type Error = UnknownVersion;
|
||||
|
||||
fn try_from(p: ProtocolVersion) -> Result<Self, UnknownVersion> {
|
||||
for v in Self::iter() {
|
||||
if v as u32 == p.0 {
|
||||
return Ok(v)
|
||||
}
|
||||
}
|
||||
|
||||
Err(UnknownVersion)
|
||||
}
|
||||
}
|
||||
|
||||
impl TryFrom<ProtocolVersion> for CollationVersion {
|
||||
type Error = UnknownVersion;
|
||||
|
||||
fn try_from(p: ProtocolVersion) -> Result<Self, UnknownVersion> {
|
||||
for v in Self::iter() {
|
||||
if v as u32 == p.0 {
|
||||
return Ok(v)
|
||||
}
|
||||
}
|
||||
|
||||
Err(UnknownVersion)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<ValidationVersion> for ProtocolVersion {
|
||||
|
||||
@@ -51,9 +51,12 @@ pub use outgoing::{OutgoingRequest, OutgoingResult, Recipient, Requests, Respons
|
||||
///// Multiplexer for incoming requests.
|
||||
// pub mod multiplexer;
|
||||
|
||||
/// Actual versioned requests and responses, that are sent over the wire.
|
||||
/// Actual versioned requests and responses that are sent over the wire.
|
||||
pub mod v1;
|
||||
|
||||
/// Actual versioned requests and responses that are sent over the wire.
|
||||
pub mod vstaging;
|
||||
|
||||
/// A protocol per subsystem seems to make the most sense, this way we don't need any dispatching
|
||||
/// within protocols.
|
||||
#[derive(Copy, Clone, Debug, Hash, PartialEq, Eq, EnumIter)]
|
||||
@@ -62,6 +65,8 @@ pub enum Protocol {
|
||||
ChunkFetchingV1,
|
||||
/// Protocol for fetching collations from collators.
|
||||
CollationFetchingV1,
|
||||
/// Protocol for fetching collations from collators when async backing is enabled.
|
||||
CollationFetchingVStaging,
|
||||
/// Protocol for fetching seconded PoVs from validators of the same group.
|
||||
PoVFetchingV1,
|
||||
/// Protocol for fetching available data.
|
||||
@@ -70,6 +75,10 @@ pub enum Protocol {
|
||||
StatementFetchingV1,
|
||||
/// Sending of dispute statements with application level confirmations.
|
||||
DisputeSendingV1,
|
||||
|
||||
/// Protocol for requesting candidates with attestations in statement distribution
|
||||
/// when async backing is enabled.
|
||||
AttestedCandidateVStaging,
|
||||
}
|
||||
|
||||
/// Minimum bandwidth we expect for validators - 500Mbit/s is the recommendation, so approximately
|
||||
@@ -101,12 +110,30 @@ const POV_REQUEST_TIMEOUT_CONNECTED: Duration = Duration::from_millis(1200);
|
||||
/// fit statement distribution within a block of 6 seconds.)
|
||||
const STATEMENTS_TIMEOUT: Duration = Duration::from_secs(1);
|
||||
|
||||
/// We want attested candidate requests to time out relatively fast,
|
||||
/// because slow requests will bottleneck the backing system. Ideally, we'd have
|
||||
/// an adaptive timeout based on the candidate size, because there will be a lot of variance
|
||||
/// in candidate sizes: candidates with no code and no messages vs candidates with code
|
||||
/// and messages.
|
||||
///
|
||||
/// We supply leniency because there are often large candidates and asynchronous
|
||||
/// backing allows them to be included over a longer window of time. Exponential back-off
|
||||
/// up to a maximum of 10 seconds would be ideal, but isn't supported by the
|
||||
/// infrastructure here yet: see https://github.com/paritytech/polkadot/issues/6009
|
||||
const ATTESTED_CANDIDATE_TIMEOUT: Duration = Duration::from_millis(2500);
|
||||
|
||||
/// We don't want a slow peer to slow down all the others, at the same time we want to get out the
|
||||
/// data quickly in full to at least some peers (as this will reduce load on us as they then can
|
||||
/// start serving the data). So this value is a trade-off. 3 seems to be sensible. So we would need
|
||||
/// to have 3 slow nodes connected, to delay transfer for others by `STATEMENTS_TIMEOUT`.
|
||||
pub const MAX_PARALLEL_STATEMENT_REQUESTS: u32 = 3;
|
||||
|
||||
/// We don't want a slow peer to slow down all the others, at the same time we want to get out the
|
||||
/// data quickly in full to at least some peers (as this will reduce load on us as they then can
|
||||
/// start serving the data). So this value is a tradeoff. 5 seems to be sensible. So we would need
|
||||
/// to have 5 slow nodes connected, to delay transfer for others by `ATTESTED_CANDIDATE_TIMEOUT`.
|
||||
pub const MAX_PARALLEL_ATTESTED_CANDIDATE_REQUESTS: u32 = 5;
|
||||
|
||||
/// Response size limit for responses of POV like data.
|
||||
///
|
||||
/// This is larger than `MAX_POV_SIZE` to account for protocol overhead and for additional data in
|
||||
@@ -120,6 +147,12 @@ const POV_RESPONSE_SIZE: u64 = MAX_POV_SIZE as u64 + 10_000;
|
||||
/// This is `MAX_CODE_SIZE` plus some additional space for protocol overhead.
|
||||
const STATEMENT_RESPONSE_SIZE: u64 = MAX_CODE_SIZE as u64 + 10_000;
|
||||
|
||||
/// Maximum response sizes for `AttestedCandidateVStaging`.
|
||||
///
|
||||
/// This is `MAX_CODE_SIZE` plus some additional space for protocol overhead and
|
||||
/// additional backing statements.
|
||||
const ATTESTED_CANDIDATE_RESPONSE_SIZE: u64 = MAX_CODE_SIZE as u64 + 100_000;
|
||||
|
||||
/// We can have relative large timeouts here, there is no value of hitting a
|
||||
/// timeout as we want to get statements through to each node in any case.
|
||||
pub const DISPUTE_REQUEST_TIMEOUT: Duration = Duration::from_secs(12);
|
||||
@@ -166,15 +199,16 @@ impl Protocol {
|
||||
request_timeout: CHUNK_REQUEST_TIMEOUT,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::CollationFetchingV1 => RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
max_request_size: 1_000,
|
||||
max_response_size: POV_RESPONSE_SIZE,
|
||||
// Taken from initial implementation in collator protocol:
|
||||
request_timeout: POV_REQUEST_TIMEOUT_CONNECTED,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::CollationFetchingV1 | Protocol::CollationFetchingVStaging =>
|
||||
RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
max_request_size: 1_000,
|
||||
max_response_size: POV_RESPONSE_SIZE,
|
||||
// Taken from initial implementation in collator protocol:
|
||||
request_timeout: POV_REQUEST_TIMEOUT_CONNECTED,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::PoVFetchingV1 => RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
@@ -220,6 +254,14 @@ impl Protocol {
|
||||
request_timeout: DISPUTE_REQUEST_TIMEOUT,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::AttestedCandidateVStaging => RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
max_request_size: 1_000,
|
||||
max_response_size: ATTESTED_CANDIDATE_RESPONSE_SIZE,
|
||||
request_timeout: ATTESTED_CANDIDATE_TIMEOUT,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -233,7 +275,7 @@ impl Protocol {
|
||||
// as well.
|
||||
Protocol::ChunkFetchingV1 => 100,
|
||||
// 10 seems reasonable, considering group sizes of max 10 validators.
|
||||
Protocol::CollationFetchingV1 => 10,
|
||||
Protocol::CollationFetchingV1 | Protocol::CollationFetchingVStaging => 10,
|
||||
// 10 seems reasonable, considering group sizes of max 10 validators.
|
||||
Protocol::PoVFetchingV1 => 10,
|
||||
// Validators are constantly self-selecting to request available data which may lead
|
||||
@@ -264,23 +306,46 @@ impl Protocol {
|
||||
// average, so something in the ballpark of 100 should be fine. Nodes will retry on
|
||||
// failure, so having a good value here is mostly about performance tuning.
|
||||
Protocol::DisputeSendingV1 => 100,
|
||||
|
||||
Protocol::AttestedCandidateVStaging => {
|
||||
// We assume we can utilize up to 70% of the available bandwidth for statements.
|
||||
// This is just a guess/estimate, with the following considerations: If we are
|
||||
// faster than that, queue size will stay low anyway, even if not - requesters will
|
||||
// get an immediate error, but if we are slower, requesters will run in a timeout -
|
||||
// wasting precious time.
|
||||
let available_bandwidth = 7 * MIN_BANDWIDTH_BYTES / 10;
|
||||
let size = u64::saturating_sub(
|
||||
ATTESTED_CANDIDATE_TIMEOUT.as_millis() as u64 * available_bandwidth /
|
||||
(1000 * MAX_CODE_SIZE as u64),
|
||||
MAX_PARALLEL_ATTESTED_CANDIDATE_REQUESTS as u64,
|
||||
);
|
||||
debug_assert!(
|
||||
size > 0,
|
||||
"We should have a channel size greater zero, otherwise we won't accept any requests."
|
||||
);
|
||||
size as usize
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// Fallback protocol names of this protocol, as understood by substrate networking.
|
||||
fn get_fallback_names(self) -> Vec<ProtocolName> {
|
||||
std::iter::once(self.get_legacy_name().into()).collect()
|
||||
self.get_legacy_name().into_iter().map(Into::into).collect()
|
||||
}
|
||||
|
||||
/// Legacy protocol name associated with each peer set.
|
||||
const fn get_legacy_name(self) -> &'static str {
|
||||
/// Legacy protocol name associated with each peer set, if any.
|
||||
const fn get_legacy_name(self) -> Option<&'static str> {
|
||||
match self {
|
||||
Protocol::ChunkFetchingV1 => "/polkadot/req_chunk/1",
|
||||
Protocol::CollationFetchingV1 => "/polkadot/req_collation/1",
|
||||
Protocol::PoVFetchingV1 => "/polkadot/req_pov/1",
|
||||
Protocol::AvailableDataFetchingV1 => "/polkadot/req_available_data/1",
|
||||
Protocol::StatementFetchingV1 => "/polkadot/req_statement/1",
|
||||
Protocol::DisputeSendingV1 => "/polkadot/send_dispute/1",
|
||||
Protocol::ChunkFetchingV1 => Some("/polkadot/req_chunk/1"),
|
||||
Protocol::CollationFetchingV1 => Some("/polkadot/req_collation/1"),
|
||||
Protocol::PoVFetchingV1 => Some("/polkadot/req_pov/1"),
|
||||
Protocol::AvailableDataFetchingV1 => Some("/polkadot/req_available_data/1"),
|
||||
Protocol::StatementFetchingV1 => Some("/polkadot/req_statement/1"),
|
||||
Protocol::DisputeSendingV1 => Some("/polkadot/send_dispute/1"),
|
||||
|
||||
// Introduced after legacy names became legacy.
|
||||
Protocol::AttestedCandidateVStaging => None,
|
||||
Protocol::CollationFetchingVStaging => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -336,6 +401,9 @@ impl ReqProtocolNames {
|
||||
Protocol::AvailableDataFetchingV1 => "/req_available_data/1",
|
||||
Protocol::StatementFetchingV1 => "/req_statement/1",
|
||||
Protocol::DisputeSendingV1 => "/send_dispute/1",
|
||||
|
||||
Protocol::CollationFetchingVStaging => "/req_collation/2",
|
||||
Protocol::AttestedCandidateVStaging => "/req_attested_candidate/2",
|
||||
};
|
||||
|
||||
format!("{}{}", prefix, short_name).into()
|
||||
|
||||
@@ -23,7 +23,7 @@ use sc_network::PeerId;
|
||||
|
||||
use polkadot_primitives::AuthorityDiscoveryId;
|
||||
|
||||
use super::{v1, IsRequest, Protocol};
|
||||
use super::{v1, vstaging, IsRequest, Protocol};
|
||||
|
||||
/// All requests that can be sent to the network bridge via `NetworkBridgeTxMessage::SendRequest`.
|
||||
#[derive(Debug)]
|
||||
@@ -40,6 +40,12 @@ pub enum Requests {
|
||||
StatementFetchingV1(OutgoingRequest<v1::StatementFetchingRequest>),
|
||||
/// Requests for notifying about an ongoing dispute.
|
||||
DisputeSendingV1(OutgoingRequest<v1::DisputeRequest>),
|
||||
|
||||
/// Request a candidate and attestations.
|
||||
AttestedCandidateVStaging(OutgoingRequest<vstaging::AttestedCandidateRequest>),
|
||||
/// Fetch a collation from a collator which previously announced it.
|
||||
/// Compared to V1 it requires specifying which candidate is requested by its hash.
|
||||
CollationFetchingVStaging(OutgoingRequest<vstaging::CollationFetchingRequest>),
|
||||
}
|
||||
|
||||
impl Requests {
|
||||
@@ -48,10 +54,12 @@ impl Requests {
|
||||
match self {
|
||||
Self::ChunkFetchingV1(_) => Protocol::ChunkFetchingV1,
|
||||
Self::CollationFetchingV1(_) => Protocol::CollationFetchingV1,
|
||||
Self::CollationFetchingVStaging(_) => Protocol::CollationFetchingVStaging,
|
||||
Self::PoVFetchingV1(_) => Protocol::PoVFetchingV1,
|
||||
Self::AvailableDataFetchingV1(_) => Protocol::AvailableDataFetchingV1,
|
||||
Self::StatementFetchingV1(_) => Protocol::StatementFetchingV1,
|
||||
Self::DisputeSendingV1(_) => Protocol::DisputeSendingV1,
|
||||
Self::AttestedCandidateVStaging(_) => Protocol::AttestedCandidateVStaging,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,10 +74,12 @@ impl Requests {
|
||||
match self {
|
||||
Self::ChunkFetchingV1(r) => r.encode_request(),
|
||||
Self::CollationFetchingV1(r) => r.encode_request(),
|
||||
Self::CollationFetchingVStaging(r) => r.encode_request(),
|
||||
Self::PoVFetchingV1(r) => r.encode_request(),
|
||||
Self::AvailableDataFetchingV1(r) => r.encode_request(),
|
||||
Self::StatementFetchingV1(r) => r.encode_request(),
|
||||
Self::DisputeSendingV1(r) => r.encode_request(),
|
||||
Self::AttestedCandidateVStaging(r) => r.encode_request(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Requests and responses as sent over the wire for the individual protocols.
|
||||
|
||||
use parity_scale_codec::{Decode, Encode};
|
||||
|
||||
use polkadot_primitives::vstaging::{
|
||||
CandidateHash, CommittedCandidateReceipt, Hash, Id as ParaId, PersistedValidationData,
|
||||
UncheckedSignedStatement,
|
||||
};
|
||||
|
||||
use super::{IsRequest, Protocol};
|
||||
use crate::vstaging::StatementFilter;
|
||||
|
||||
/// Request a candidate with statements.
|
||||
#[derive(Debug, Clone, Encode, Decode)]
|
||||
pub struct AttestedCandidateRequest {
|
||||
/// Hash of the candidate we want to request.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Statement filter with 'OR' semantics, indicating which validators
|
||||
/// not to send statements for.
|
||||
///
|
||||
/// The filter must have exactly the minimum size required to
|
||||
/// fit all validators from the backing group.
|
||||
///
|
||||
/// The response may not contain any statements masked out by this mask.
|
||||
pub mask: StatementFilter,
|
||||
}
|
||||
|
||||
/// Response to an `AttestedCandidateRequest`.
|
||||
#[derive(Debug, Clone, Encode, Decode)]
|
||||
pub struct AttestedCandidateResponse {
|
||||
/// The candidate receipt, with commitments.
|
||||
pub candidate_receipt: CommittedCandidateReceipt,
|
||||
/// The [`PersistedValidationData`] corresponding to the candidate.
|
||||
pub persisted_validation_data: PersistedValidationData,
|
||||
/// All known statements about the candidate, in compact form,
|
||||
/// omitting `Seconded` statements which were intended to be masked
|
||||
/// out.
|
||||
pub statements: Vec<UncheckedSignedStatement>,
|
||||
}
|
||||
|
||||
impl IsRequest for AttestedCandidateRequest {
|
||||
type Response = AttestedCandidateResponse;
|
||||
const PROTOCOL: Protocol = Protocol::AttestedCandidateVStaging;
|
||||
}
|
||||
|
||||
/// Responses as sent by collators.
|
||||
pub type CollationFetchingResponse = super::v1::CollationFetchingResponse;
|
||||
|
||||
/// Request the advertised collation at that relay-parent.
|
||||
#[derive(Debug, Clone, Encode, Decode)]
|
||||
pub struct CollationFetchingRequest {
|
||||
/// Relay parent collation is built on top of.
|
||||
pub relay_parent: Hash,
|
||||
/// The `ParaId` of the collation.
|
||||
pub para_id: ParaId,
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
}
|
||||
|
||||
impl IsRequest for CollationFetchingRequest {
|
||||
// The response is the same as for V1.
|
||||
type Response = CollationFetchingResponse;
|
||||
const PROTOCOL: Protocol = Protocol::CollationFetchingVStaging;
|
||||
}
|
||||
@@ -22,10 +22,12 @@ indexmap = "1.9.1"
|
||||
parity-scale-codec = { version = "3.6.1", default-features = false, features = ["derive"] }
|
||||
thiserror = "1.0.31"
|
||||
fatality = "0.0.6"
|
||||
bitvec = "1"
|
||||
|
||||
[dev-dependencies]
|
||||
polkadot-node-subsystem-test-helpers = { path = "../../subsystem-test-helpers" }
|
||||
async-channel = "1.8.0"
|
||||
assert_matches = "1.4.0"
|
||||
polkadot-node-subsystem-test-helpers = { path = "../../subsystem-test-helpers" }
|
||||
sp-authority-discovery = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
@@ -36,3 +38,5 @@ sc-keystore = { git = "https://github.com/paritytech/substrate", branch = "maste
|
||||
sc-network = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
futures-timer = "3.0.2"
|
||||
polkadot-primitives-test-helpers = { path = "../../../primitives/test-helpers" }
|
||||
rand_chacha = "0.3"
|
||||
polkadot-node-subsystem-types = { path = "../../subsystem-types" }
|
||||
|
||||
@@ -18,9 +18,13 @@
|
||||
//! Error handling related code and Error/Result definitions.
|
||||
|
||||
use polkadot_node_network_protocol::PeerId;
|
||||
use polkadot_node_subsystem::SubsystemError;
|
||||
use polkadot_node_subsystem_util::runtime;
|
||||
use polkadot_primitives::{CandidateHash, Hash};
|
||||
use polkadot_node_subsystem::{RuntimeApiError, SubsystemError};
|
||||
use polkadot_node_subsystem_util::{
|
||||
backing_implicit_view::FetchError as ImplicitViewFetchError, runtime,
|
||||
};
|
||||
use polkadot_primitives::{CandidateHash, Hash, Id as ParaId};
|
||||
|
||||
use futures::channel::oneshot;
|
||||
|
||||
use crate::LOG_TARGET;
|
||||
|
||||
@@ -56,6 +60,27 @@ pub enum Error {
|
||||
#[error("Error while accessing runtime information")]
|
||||
Runtime(#[from] runtime::Error),
|
||||
|
||||
#[error("RuntimeAPISubsystem channel closed before receipt")]
|
||||
RuntimeApiUnavailable(#[source] oneshot::Canceled),
|
||||
|
||||
#[error("Fetching persisted validation data for para {0:?}, {1:?}")]
|
||||
FetchPersistedValidationData(ParaId, RuntimeApiError),
|
||||
|
||||
#[error("Fetching session index failed {0:?}")]
|
||||
FetchSessionIndex(RuntimeApiError),
|
||||
|
||||
#[error("Fetching session info failed {0:?}")]
|
||||
FetchSessionInfo(RuntimeApiError),
|
||||
|
||||
#[error("Fetching availability cores failed {0:?}")]
|
||||
FetchAvailabilityCores(RuntimeApiError),
|
||||
|
||||
#[error("Fetching validator groups failed {0:?}")]
|
||||
FetchValidatorGroups(RuntimeApiError),
|
||||
|
||||
#[error("Attempted to share statement when not a validator or not assigned")]
|
||||
InvalidShare,
|
||||
|
||||
#[error("Relay parent could not be found in active heads")]
|
||||
NoSuchHead(Hash),
|
||||
|
||||
@@ -76,6 +101,10 @@ pub enum Error {
|
||||
// Responder no longer waits for our data. (Should not happen right now.)
|
||||
#[error("Oneshot `GetData` channel closed")]
|
||||
ResponderGetDataCanceled,
|
||||
|
||||
// Failed to activate leaf due to a fetch error.
|
||||
#[error("Implicit view failure while activating leaf")]
|
||||
ActivateLeafFailure(ImplicitViewFetchError),
|
||||
}
|
||||
|
||||
/// Utility for eating top level errors and log them.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+4
-1
@@ -32,7 +32,10 @@ use polkadot_node_subsystem::{Span, Stage};
|
||||
use polkadot_node_subsystem_util::TimeoutExt;
|
||||
use polkadot_primitives::{CandidateHash, CommittedCandidateReceipt, Hash};
|
||||
|
||||
use crate::{metrics::Metrics, COST_WRONG_HASH, LOG_TARGET};
|
||||
use crate::{
|
||||
legacy_v1::{COST_WRONG_HASH, LOG_TARGET},
|
||||
metrics::Metrics,
|
||||
};
|
||||
|
||||
// In case we failed fetching from our known peers, how long we should wait before attempting a
|
||||
// retry, even though we have not yet discovered any new peers. Or in other words how long to
|
||||
+2
-2
@@ -48,8 +48,8 @@ pub enum ResponderMessage {
|
||||
|
||||
/// A fetching task, taking care of fetching large statements via request/response.
|
||||
///
|
||||
/// A fetch task does not know about a particular `Statement` instead it just tries fetching a
|
||||
/// `CommittedCandidateReceipt` from peers, whether this can be used to re-assemble one ore
|
||||
/// A fetch task does not know about a particular `Statement`, instead it just tries fetching a
|
||||
/// `CommittedCandidateReceipt` from peers, whether this can be used to re-assemble one or
|
||||
/// many `SignedFullStatement`s needs to be verified by the caller.
|
||||
pub async fn respond(
|
||||
mut receiver: IncomingRequestReceiver<StatementFetchingRequest>,
|
||||
+213
-28
@@ -14,7 +14,11 @@
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
use super::{metrics::Metrics, *};
|
||||
#![allow(clippy::clone_on_copy)]
|
||||
|
||||
use super::*;
|
||||
use crate::{metrics::Metrics, *};
|
||||
|
||||
use assert_matches::assert_matches;
|
||||
use futures::executor;
|
||||
use futures_timer::Delay;
|
||||
@@ -26,19 +30,21 @@ use polkadot_node_network_protocol::{
|
||||
v1::{StatementFetchingRequest, StatementFetchingResponse},
|
||||
IncomingRequest, Recipient, ReqProtocolNames, Requests,
|
||||
},
|
||||
view, ObservedRole,
|
||||
view, ObservedRole, VersionedValidationProtocol,
|
||||
};
|
||||
use polkadot_node_primitives::{
|
||||
SignedFullStatementWithPVD, Statement, UncheckedSignedFullStatement,
|
||||
};
|
||||
use polkadot_node_primitives::{Statement, UncheckedSignedFullStatement};
|
||||
use polkadot_node_subsystem::{
|
||||
jaeger,
|
||||
messages::{
|
||||
network_bridge_event, AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest,
|
||||
},
|
||||
ActivatedLeaf, LeafStatus,
|
||||
ActivatedLeaf, LeafStatus, RuntimeApiError,
|
||||
};
|
||||
use polkadot_node_subsystem_test_helpers::mock::make_ferdie_keystore;
|
||||
use polkadot_primitives::{
|
||||
GroupIndex, Hash, Id as ParaId, IndexedVec, SessionInfo, ValidationCode, ValidatorId,
|
||||
GroupIndex, Hash, HeadData, Id as ParaId, IndexedVec, SessionInfo, ValidationCode,
|
||||
};
|
||||
use polkadot_primitives_test_helpers::{
|
||||
dummy_committed_candidate_receipt, dummy_hash, AlwaysZeroRng,
|
||||
@@ -54,6 +60,30 @@ use util::reputation::add_reputation;
|
||||
// Some deterministic genesis hash for protocol names
|
||||
const GENESIS_HASH: Hash = Hash::repeat_byte(0xff);
|
||||
|
||||
const ASYNC_BACKING_DISABLED_ERROR: RuntimeApiError =
|
||||
RuntimeApiError::NotSupported { runtime_api_name: "test-runtime" };
|
||||
|
||||
fn dummy_pvd() -> PersistedValidationData {
|
||||
PersistedValidationData {
|
||||
parent_head: HeadData(vec![7, 8, 9]),
|
||||
relay_parent_number: 5,
|
||||
max_pov_size: 1024,
|
||||
relay_parent_storage_root: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
fn extend_statement_with_pvd(
|
||||
statement: SignedFullStatement,
|
||||
pvd: PersistedValidationData,
|
||||
) -> SignedFullStatementWithPVD {
|
||||
statement
|
||||
.convert_to_superpayload_with(|statement| match statement {
|
||||
Statement::Seconded(receipt) => StatementWithPVD::Seconded(receipt, pvd),
|
||||
Statement::Valid(candidate_hash) => StatementWithPVD::Valid(candidate_hash),
|
||||
})
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn active_head_accepts_only_2_seconded_per_validator() {
|
||||
let validators = vec![
|
||||
@@ -496,6 +526,7 @@ fn peer_view_update_sends_messages() {
|
||||
|
||||
let mut peer_data = PeerData {
|
||||
view: old_view,
|
||||
protocol_version: ValidationVersion::V1,
|
||||
view_knowledge: {
|
||||
let mut k = HashMap::new();
|
||||
|
||||
@@ -554,8 +585,9 @@ fn peer_view_update_sends_messages() {
|
||||
for statement in active_head.statements_about(candidate_hash) {
|
||||
let message = handle.recv().await;
|
||||
let expected_to = vec![peer];
|
||||
let expected_payload =
|
||||
statement_message(hash_c, statement.statement.clone(), &Metrics::default());
|
||||
let expected_payload = VersionedValidationProtocol::from(Versioned::V1(
|
||||
v1_statement_message(hash_c, statement.statement.clone(), &Metrics::default()),
|
||||
));
|
||||
|
||||
assert_matches!(
|
||||
message,
|
||||
@@ -596,6 +628,7 @@ fn circulated_statement_goes_to_all_peers_with_view() {
|
||||
|
||||
let peer_data_from_view = |view: View| PeerData {
|
||||
view: view.clone(),
|
||||
protocol_version: ValidationVersion::V1,
|
||||
view_knowledge: view.iter().map(|v| (*v, Default::default())).collect(),
|
||||
maybe_authority: None,
|
||||
};
|
||||
@@ -697,7 +730,7 @@ fn circulated_statement_goes_to_all_peers_with_view() {
|
||||
|
||||
assert_eq!(
|
||||
payload,
|
||||
statement_message(hash_b, statement.statement.clone(), &Metrics::default()),
|
||||
VersionedValidationProtocol::from(Versioned::V1(v1_statement_message(hash_b, statement.statement.clone(), &Metrics::default()))),
|
||||
);
|
||||
}
|
||||
)
|
||||
@@ -706,12 +739,14 @@ fn circulated_statement_goes_to_all_peers_with_view() {
|
||||
|
||||
#[test]
|
||||
fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
const PARA_ID: ParaId = ParaId::new(1);
|
||||
let hash_a = Hash::repeat_byte(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
let candidate = {
|
||||
let mut c = dummy_committed_candidate_receipt(dummy_hash());
|
||||
c.descriptor.relay_parent = hash_a;
|
||||
c.descriptor.para_id = 1.into();
|
||||
c.descriptor.para_id = PARA_ID;
|
||||
c
|
||||
};
|
||||
|
||||
@@ -733,11 +768,13 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: Arc::new(LocalKeystore::in_memory()),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -758,6 +795,17 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -862,18 +910,32 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(para_id, assumption, tx),
|
||||
)) if para_id == PARA_ID &&
|
||||
assumption == OccupiedCoreAssumption::Free &&
|
||||
hash == hash_a =>
|
||||
{
|
||||
tx.send(Ok(Some(pvd))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(p, r))
|
||||
) if p == peer_a && r == BENEFIT_VALID_STATEMENT_FIRST.into() => {}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) if r == hash_a && s == statement => {}
|
||||
) if r == hash_a && s == statement_with_pvd => {}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
@@ -902,6 +964,9 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
|
||||
#[test]
|
||||
fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
const PARA_ID: ParaId = ParaId::new(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
sp_tracing::try_init_simple();
|
||||
let hash_a = Hash::repeat_byte(1);
|
||||
let hash_b = Hash::repeat_byte(2);
|
||||
@@ -909,7 +974,7 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
let candidate = {
|
||||
let mut c = dummy_committed_candidate_receipt(dummy_hash());
|
||||
c.descriptor.relay_parent = hash_a;
|
||||
c.descriptor.para_id = 1.into();
|
||||
c.descriptor.para_id = PARA_ID;
|
||||
c.commitments.new_validation_code = Some(ValidationCode(vec![1, 2, 3]));
|
||||
c
|
||||
};
|
||||
@@ -937,11 +1002,13 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, mut req_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -962,6 +1029,17 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -1292,6 +1370,20 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
) if p == peer_c && r == BENEFIT_VALID_RESPONSE.into() => {}
|
||||
);
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(para_id, assumption, tx),
|
||||
)) if para_id == PARA_ID &&
|
||||
assumption == OccupiedCoreAssumption::Free &&
|
||||
hash == hash_a =>
|
||||
{
|
||||
tx.send(Ok(Some(pvd))).unwrap();
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
@@ -1303,7 +1395,7 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) if r == hash_a && s == statement => {}
|
||||
) if r == hash_a && s == statement_with_pvd => {}
|
||||
);
|
||||
|
||||
// Now messages should go out:
|
||||
@@ -1400,6 +1492,7 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
fn delay_reputation_changes() {
|
||||
sp_tracing::try_init_simple();
|
||||
let hash_a = Hash::repeat_byte(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
let candidate = {
|
||||
let mut c = dummy_committed_candidate_receipt(dummy_hash());
|
||||
@@ -1431,13 +1524,15 @@ fn delay_reputation_changes() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let reputation_interval = Duration::from_millis(100);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| false),
|
||||
@@ -1458,6 +1553,17 @@ fn delay_reputation_changes() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -1768,9 +1874,18 @@ fn delay_reputation_changes() {
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) if r == hash_a && s == statement => {}
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(_, assumption, tx),
|
||||
)) if assumption == OccupiedCoreAssumption::Free && hash == hash_a =>
|
||||
{
|
||||
tx.send(Ok(Some(pvd))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(CandidateBackingMessage::Statement(..))
|
||||
);
|
||||
|
||||
// Now messages should go out:
|
||||
@@ -1885,11 +2000,13 @@ fn share_prioritizes_backing_group() {
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, mut req_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -1910,6 +2027,17 @@ fn share_prioritizes_backing_group() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -2069,9 +2197,17 @@ fn share_prioritizes_backing_group() {
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
SignedFullStatement::sign(
|
||||
// note: this is ignored by legacy-v1 code.
|
||||
let pvd = PersistedValidationData {
|
||||
parent_head: HeadData::from(vec![1, 2, 3]),
|
||||
relay_parent_number: 0,
|
||||
relay_parent_storage_root: Hash::repeat_byte(42),
|
||||
max_pov_size: 100,
|
||||
};
|
||||
|
||||
SignedFullStatementWithPVD::sign(
|
||||
&keystore,
|
||||
Statement::Seconded(candidate.clone()),
|
||||
Statement::Seconded(candidate.clone()).supply_pvd(pvd),
|
||||
&signing_context,
|
||||
ValidatorIndex(4),
|
||||
&ferdie_public.into(),
|
||||
@@ -2081,14 +2217,15 @@ fn share_prioritizes_backing_group() {
|
||||
.expect("should be signed")
|
||||
};
|
||||
|
||||
let metadata = derive_metadata_assuming_seconded(hash_a, statement.clone().into());
|
||||
|
||||
handle
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::Share(hash_a, statement.clone()),
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement = StatementWithPVD::drop_pvd_from_signed(statement);
|
||||
let metadata = derive_metadata_assuming_seconded(hash_a, statement.clone().into());
|
||||
|
||||
// Messages should go out:
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
@@ -2180,10 +2317,12 @@ fn peer_cant_flood_with_large_statements() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -2204,6 +2343,17 @@ fn peer_cant_flood_with_large_statements() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -2341,6 +2491,7 @@ fn peer_cant_flood_with_large_statements() {
|
||||
#[test]
|
||||
fn handle_multiple_seconded_statements() {
|
||||
let relay_parent_hash = Hash::repeat_byte(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
let candidate = dummy_committed_candidate_receipt(relay_parent_hash);
|
||||
let candidate_hash = candidate.hash();
|
||||
@@ -2384,11 +2535,13 @@ fn handle_multiple_seconded_statements() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let virtual_overseer_fut = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: Arc::new(LocalKeystore::in_memory()),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -2409,6 +2562,17 @@ fn handle_multiple_seconded_statements() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == relay_parent_hash
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -2575,6 +2739,18 @@ fn handle_multiple_seconded_statements() {
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
_,
|
||||
RuntimeApiRequest::PersistedValidationData(_, assumption, tx),
|
||||
)) if assumption == OccupiedCoreAssumption::Free => {
|
||||
tx.send(Ok(Some(pvd.clone()))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
@@ -2592,7 +2768,7 @@ fn handle_multiple_seconded_statements() {
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) => {
|
||||
assert_eq!(r, relay_parent_hash);
|
||||
assert_eq!(s, statement);
|
||||
assert_eq!(s, statement_with_pvd);
|
||||
}
|
||||
);
|
||||
|
||||
@@ -2676,6 +2852,10 @@ fn handle_multiple_seconded_statements() {
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
// Persisted validation data is cached.
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
@@ -2692,7 +2872,7 @@ fn handle_multiple_seconded_statements() {
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) => {
|
||||
assert_eq!(r, relay_parent_hash);
|
||||
assert_eq!(s, statement);
|
||||
assert_eq!(s, statement_with_pvd);
|
||||
}
|
||||
);
|
||||
|
||||
@@ -2784,3 +2964,8 @@ fn derive_metadata_assuming_seconded(
|
||||
signature: statement.unchecked_signature().clone(),
|
||||
}
|
||||
}
|
||||
|
||||
// TODO [now]: adapt most tests to v2 messages.
|
||||
// TODO [now]: test that v2 peers send v1 messages to v1 peers
|
||||
// TODO [now]: test that v2 peers handle v1 messages from v1 peers.
|
||||
// TODO [now]: test that v2 peers send v2 messages to v2 peers.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -29,7 +29,7 @@ struct MetricsInner {
|
||||
received_responses: prometheus::CounterVec<prometheus::U64>,
|
||||
active_leaves_update: prometheus::Histogram,
|
||||
share: prometheus::Histogram,
|
||||
network_bridge_update_v1: prometheus::HistogramVec,
|
||||
network_bridge_update: prometheus::HistogramVec,
|
||||
statements_unexpected: prometheus::CounterVec<prometheus::U64>,
|
||||
created_message_size: prometheus::Gauge<prometheus::U64>,
|
||||
}
|
||||
@@ -77,16 +77,13 @@ impl Metrics {
|
||||
self.0.as_ref().map(|metrics| metrics.share.start_timer())
|
||||
}
|
||||
|
||||
/// Provide a timer for `network_bridge_update_v1` which observes on drop.
|
||||
pub fn time_network_bridge_update_v1(
|
||||
/// Provide a timer for `network_bridge_update` which observes on drop.
|
||||
pub fn time_network_bridge_update(
|
||||
&self,
|
||||
message_type: &'static str,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0.as_ref().map(|metrics| {
|
||||
metrics
|
||||
.network_bridge_update_v1
|
||||
.with_label_values(&[message_type])
|
||||
.start_timer()
|
||||
metrics.network_bridge_update.with_label_values(&[message_type]).start_timer()
|
||||
})
|
||||
}
|
||||
|
||||
@@ -168,11 +165,11 @@ impl metrics::Metrics for Metrics {
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
network_bridge_update_v1: prometheus::register(
|
||||
network_bridge_update: prometheus::register(
|
||||
prometheus::HistogramVec::new(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_statement_distribution_network_bridge_update_v1",
|
||||
"Time spent within `statement_distribution::network_bridge_update_v1`",
|
||||
"polkadot_parachain_statement_distribution_network_bridge_update",
|
||||
"Time spent within `statement_distribution::network_bridge_update`",
|
||||
)
|
||||
.buckets(HISTOGRAM_LATENCY_BUCKETS.into()),
|
||||
&["message_type"],
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,70 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! A utility for tracking groups and their members within a session.
|
||||
|
||||
use polkadot_node_primitives::minimum_votes;
|
||||
use polkadot_primitives::vstaging::{GroupIndex, IndexedVec, ValidatorIndex};
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Validator groups within a session, plus some helpful indexing for
|
||||
/// looking up groups by validator indices or authority discovery ID.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Groups {
|
||||
groups: IndexedVec<GroupIndex, Vec<ValidatorIndex>>,
|
||||
by_validator_index: HashMap<ValidatorIndex, GroupIndex>,
|
||||
}
|
||||
|
||||
impl Groups {
|
||||
/// Create a new [`Groups`] tracker with the groups and discovery keys
|
||||
/// from the session.
|
||||
pub fn new(groups: IndexedVec<GroupIndex, Vec<ValidatorIndex>>) -> Self {
|
||||
let mut by_validator_index = HashMap::new();
|
||||
|
||||
for (i, group) in groups.iter().enumerate() {
|
||||
let index = GroupIndex(i as _);
|
||||
for v in group {
|
||||
by_validator_index.insert(*v, index);
|
||||
}
|
||||
}
|
||||
|
||||
Groups { groups, by_validator_index }
|
||||
}
|
||||
|
||||
/// Access all the underlying groups.
|
||||
pub fn all(&self) -> &IndexedVec<GroupIndex, Vec<ValidatorIndex>> {
|
||||
&self.groups
|
||||
}
|
||||
|
||||
/// Get the underlying group validators by group index.
|
||||
pub fn get(&self, group_index: GroupIndex) -> Option<&[ValidatorIndex]> {
|
||||
self.groups.get(group_index).map(|x| &x[..])
|
||||
}
|
||||
|
||||
/// Get the backing group size and backing threshold.
|
||||
pub fn get_size_and_backing_threshold(
|
||||
&self,
|
||||
group_index: GroupIndex,
|
||||
) -> Option<(usize, usize)> {
|
||||
self.get(group_index).map(|g| (g.len(), minimum_votes(g.len())))
|
||||
}
|
||||
|
||||
/// Get the group index for a validator by index.
|
||||
pub fn by_validator_index(&self, validator_index: ValidatorIndex) -> Option<GroupIndex> {
|
||||
self.by_validator_index.get(&validator_index).map(|x| *x)
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,283 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! A store of all statements under a given relay-parent.
|
||||
//!
|
||||
//! This structure doesn't attempt to do any spam protection, which must
|
||||
//! be provided at a higher level.
|
||||
//!
|
||||
//! This keeps track of statements submitted with a number of different of
|
||||
//! views into this data: views based on the candidate, views based on the validator
|
||||
//! groups, and views based on the validators themselves.
|
||||
|
||||
use bitvec::{order::Lsb0 as BitOrderLsb0, vec::BitVec};
|
||||
use polkadot_node_network_protocol::vstaging::StatementFilter;
|
||||
use polkadot_primitives::vstaging::{
|
||||
CandidateHash, CompactStatement, GroupIndex, SignedStatement, ValidatorIndex,
|
||||
};
|
||||
use std::collections::hash_map::{Entry as HEntry, HashMap};
|
||||
|
||||
use super::groups::Groups;
|
||||
|
||||
/// Possible origins of a statement.
|
||||
pub enum StatementOrigin {
|
||||
/// The statement originated locally.
|
||||
Local,
|
||||
/// The statement originated from a remote peer.
|
||||
Remote,
|
||||
}
|
||||
|
||||
impl StatementOrigin {
|
||||
fn is_local(&self) -> bool {
|
||||
match *self {
|
||||
StatementOrigin::Local => true,
|
||||
StatementOrigin::Remote => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct StoredStatement {
|
||||
statement: SignedStatement,
|
||||
known_by_backing: bool,
|
||||
}
|
||||
|
||||
/// Storage for statements. Intended to be used for statements signed under
|
||||
/// the same relay-parent. See module docs for more details.
|
||||
pub struct StatementStore {
|
||||
validator_meta: HashMap<ValidatorIndex, ValidatorMeta>,
|
||||
|
||||
// we keep statements per-group because even though only one group _should_ be
|
||||
// producing statements about a candidate, until we have the candidate receipt
|
||||
// itself, we can't tell which group that is.
|
||||
group_statements: HashMap<(GroupIndex, CandidateHash), GroupStatements>,
|
||||
known_statements: HashMap<Fingerprint, StoredStatement>,
|
||||
}
|
||||
|
||||
impl StatementStore {
|
||||
/// Create a new [`StatementStore`]
|
||||
pub fn new(groups: &Groups) -> Self {
|
||||
let mut validator_meta = HashMap::new();
|
||||
for (g, group) in groups.all().iter().enumerate() {
|
||||
for (i, v) in group.iter().enumerate() {
|
||||
validator_meta.insert(
|
||||
*v,
|
||||
ValidatorMeta {
|
||||
seconded_count: 0,
|
||||
within_group_index: i,
|
||||
group: GroupIndex(g as _),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
StatementStore {
|
||||
validator_meta,
|
||||
group_statements: HashMap::new(),
|
||||
known_statements: HashMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert a statement. Returns `true` if was not known already, `false` if it was.
|
||||
/// Ignores statements by unknown validators and returns an error.
|
||||
pub fn insert(
|
||||
&mut self,
|
||||
groups: &Groups,
|
||||
statement: SignedStatement,
|
||||
origin: StatementOrigin,
|
||||
) -> Result<bool, ValidatorUnknown> {
|
||||
let validator_index = statement.validator_index();
|
||||
let validator_meta = match self.validator_meta.get_mut(&validator_index) {
|
||||
None => return Err(ValidatorUnknown),
|
||||
Some(m) => m,
|
||||
};
|
||||
|
||||
let compact = statement.payload().clone();
|
||||
let fingerprint = (validator_index, compact.clone());
|
||||
match self.known_statements.entry(fingerprint) {
|
||||
HEntry::Occupied(mut e) => {
|
||||
if let StatementOrigin::Local = origin {
|
||||
e.get_mut().known_by_backing = true;
|
||||
}
|
||||
|
||||
return Ok(false)
|
||||
},
|
||||
HEntry::Vacant(e) => {
|
||||
e.insert(StoredStatement { statement, known_by_backing: origin.is_local() });
|
||||
},
|
||||
}
|
||||
|
||||
let candidate_hash = *compact.candidate_hash();
|
||||
let seconded = if let CompactStatement::Seconded(_) = compact { true } else { false };
|
||||
|
||||
// cross-reference updates.
|
||||
{
|
||||
let group_index = validator_meta.group;
|
||||
let group = match groups.get(group_index) {
|
||||
Some(g) => g,
|
||||
None => {
|
||||
gum::error!(
|
||||
target: crate::LOG_TARGET,
|
||||
?group_index,
|
||||
"groups passed into `insert` differ from those used at store creation"
|
||||
);
|
||||
|
||||
return Err(ValidatorUnknown)
|
||||
},
|
||||
};
|
||||
|
||||
let group_statements = self
|
||||
.group_statements
|
||||
.entry((group_index, candidate_hash))
|
||||
.or_insert_with(|| GroupStatements::with_group_size(group.len()));
|
||||
|
||||
if seconded {
|
||||
validator_meta.seconded_count += 1;
|
||||
group_statements.note_seconded(validator_meta.within_group_index);
|
||||
} else {
|
||||
group_statements.note_validated(validator_meta.within_group_index);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Fill a `StatementFilter` to be used in the grid topology with all statements
|
||||
/// we are already aware of.
|
||||
pub fn fill_statement_filter(
|
||||
&self,
|
||||
group_index: GroupIndex,
|
||||
candidate_hash: CandidateHash,
|
||||
statement_filter: &mut StatementFilter,
|
||||
) {
|
||||
if let Some(statements) = self.group_statements.get(&(group_index, candidate_hash)) {
|
||||
statement_filter.seconded_in_group |= statements.seconded.as_bitslice();
|
||||
statement_filter.validated_in_group |= statements.valid.as_bitslice();
|
||||
}
|
||||
}
|
||||
|
||||
/// Get an iterator over stored signed statements by the group conforming to the
|
||||
/// given filter.
|
||||
///
|
||||
/// Seconded statements are provided first.
|
||||
pub fn group_statements<'a>(
|
||||
&'a self,
|
||||
groups: &'a Groups,
|
||||
group_index: GroupIndex,
|
||||
candidate_hash: CandidateHash,
|
||||
filter: &'a StatementFilter,
|
||||
) -> impl Iterator<Item = &'a SignedStatement> + 'a {
|
||||
let group_validators = groups.get(group_index);
|
||||
|
||||
let seconded_statements = filter
|
||||
.seconded_in_group
|
||||
.iter_ones()
|
||||
.filter_map(move |i| group_validators.as_ref().and_then(|g| g.get(i)))
|
||||
.filter_map(move |v| {
|
||||
self.known_statements.get(&(*v, CompactStatement::Seconded(candidate_hash)))
|
||||
})
|
||||
.map(|s| &s.statement);
|
||||
|
||||
let valid_statements = filter
|
||||
.validated_in_group
|
||||
.iter_ones()
|
||||
.filter_map(move |i| group_validators.as_ref().and_then(|g| g.get(i)))
|
||||
.filter_map(move |v| {
|
||||
self.known_statements.get(&(*v, CompactStatement::Valid(candidate_hash)))
|
||||
})
|
||||
.map(|s| &s.statement);
|
||||
|
||||
seconded_statements.chain(valid_statements)
|
||||
}
|
||||
|
||||
/// Get the full statement of this kind issued by this validator, if it is known.
|
||||
pub fn validator_statement(
|
||||
&self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: CompactStatement,
|
||||
) -> Option<&SignedStatement> {
|
||||
self.known_statements.get(&(validator_index, statement)).map(|s| &s.statement)
|
||||
}
|
||||
|
||||
/// Get an iterator over all statements marked as being unknown by the backing subsystem.
|
||||
pub fn fresh_statements_for_backing<'a>(
|
||||
&'a self,
|
||||
validators: &'a [ValidatorIndex],
|
||||
candidate_hash: CandidateHash,
|
||||
) -> impl Iterator<Item = &SignedStatement> + 'a {
|
||||
let s_st = CompactStatement::Seconded(candidate_hash);
|
||||
let v_st = CompactStatement::Valid(candidate_hash);
|
||||
|
||||
validators
|
||||
.iter()
|
||||
.flat_map(move |v| {
|
||||
let a = self.known_statements.get(&(*v, s_st.clone()));
|
||||
let b = self.known_statements.get(&(*v, v_st.clone()));
|
||||
|
||||
a.into_iter().chain(b)
|
||||
})
|
||||
.filter(|stored| !stored.known_by_backing)
|
||||
.map(|stored| &stored.statement)
|
||||
}
|
||||
|
||||
/// Get the amount of known `Seconded` statements by the given validator index.
|
||||
pub fn seconded_count(&self, validator_index: &ValidatorIndex) -> usize {
|
||||
self.validator_meta.get(validator_index).map_or(0, |m| m.seconded_count)
|
||||
}
|
||||
|
||||
/// Note that a statement is known by the backing subsystem.
|
||||
pub fn note_known_by_backing(
|
||||
&mut self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: CompactStatement,
|
||||
) {
|
||||
if let Some(stored) = self.known_statements.get_mut(&(validator_index, statement)) {
|
||||
stored.known_by_backing = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Error indicating that the validator was unknown.
|
||||
pub struct ValidatorUnknown;
|
||||
|
||||
type Fingerprint = (ValidatorIndex, CompactStatement);
|
||||
|
||||
struct ValidatorMeta {
|
||||
group: GroupIndex,
|
||||
within_group_index: usize,
|
||||
seconded_count: usize,
|
||||
}
|
||||
|
||||
struct GroupStatements {
|
||||
seconded: BitVec<u8, BitOrderLsb0>,
|
||||
valid: BitVec<u8, BitOrderLsb0>,
|
||||
}
|
||||
|
||||
impl GroupStatements {
|
||||
fn with_group_size(group_size: usize) -> Self {
|
||||
GroupStatements {
|
||||
seconded: BitVec::repeat(false, group_size),
|
||||
valid: BitVec::repeat(false, group_size),
|
||||
}
|
||||
}
|
||||
|
||||
fn note_seconded(&mut self, within_group_index: usize) {
|
||||
self.seconded.set(within_group_index, true);
|
||||
}
|
||||
|
||||
fn note_validated(&mut self, within_group_index: usize) {
|
||||
self.valid.set(within_group_index, true);
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,606 @@
|
||||
// Copyright 2023 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
#![allow(clippy::clone_on_copy)]
|
||||
|
||||
use super::*;
|
||||
use crate::*;
|
||||
use polkadot_node_network_protocol::{
|
||||
grid_topology::TopologyPeerInfo,
|
||||
request_response::{outgoing::Recipient, ReqProtocolNames},
|
||||
view, ObservedRole,
|
||||
};
|
||||
use polkadot_node_primitives::Statement;
|
||||
use polkadot_node_subsystem::messages::{
|
||||
network_bridge_event::NewGossipTopology, AllMessages, ChainApiMessage, FragmentTreeMembership,
|
||||
HypotheticalCandidate, NetworkBridgeEvent, ProspectiveParachainsMessage, ReportPeerMessage,
|
||||
RuntimeApiMessage, RuntimeApiRequest,
|
||||
};
|
||||
use polkadot_node_subsystem_test_helpers as test_helpers;
|
||||
use polkadot_node_subsystem_types::{jaeger, ActivatedLeaf, LeafStatus};
|
||||
use polkadot_node_subsystem_util::TimeoutExt;
|
||||
use polkadot_primitives::vstaging::{
|
||||
AssignmentPair, AsyncBackingParams, BlockNumber, CommittedCandidateReceipt, CoreState,
|
||||
GroupRotationInfo, HeadData, Header, IndexedVec, PersistedValidationData, ScheduledCore,
|
||||
SessionIndex, SessionInfo, ValidatorPair,
|
||||
};
|
||||
use sc_keystore::LocalKeystore;
|
||||
use sp_application_crypto::Pair as PairT;
|
||||
use sp_authority_discovery::AuthorityPair as AuthorityDiscoveryPair;
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
|
||||
use assert_matches::assert_matches;
|
||||
use futures::Future;
|
||||
use parity_scale_codec::Encode;
|
||||
use rand::{Rng, SeedableRng};
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
mod cluster;
|
||||
mod grid;
|
||||
mod requests;
|
||||
|
||||
type VirtualOverseer = test_helpers::TestSubsystemContextHandle<StatementDistributionMessage>;
|
||||
|
||||
const DEFAULT_ASYNC_BACKING_PARAMETERS: AsyncBackingParams =
|
||||
AsyncBackingParams { max_candidate_depth: 4, allowed_ancestry_len: 3 };
|
||||
|
||||
// Some deterministic genesis hash for req/res protocol names
|
||||
const GENESIS_HASH: Hash = Hash::repeat_byte(0xff);
|
||||
|
||||
struct TestConfig {
|
||||
validator_count: usize,
|
||||
// how many validators to place in each group.
|
||||
group_size: usize,
|
||||
// whether the local node should be a validator
|
||||
local_validator: bool,
|
||||
async_backing_params: Option<AsyncBackingParams>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
struct TestLocalValidator {
|
||||
validator_index: ValidatorIndex,
|
||||
group_index: GroupIndex,
|
||||
}
|
||||
|
||||
struct TestState {
|
||||
config: TestConfig,
|
||||
local: Option<TestLocalValidator>,
|
||||
validators: Vec<ValidatorPair>,
|
||||
session_info: SessionInfo,
|
||||
req_sender: async_channel::Sender<sc_network::config::IncomingRequest>,
|
||||
}
|
||||
|
||||
impl TestState {
|
||||
fn from_config(
|
||||
config: TestConfig,
|
||||
req_sender: async_channel::Sender<sc_network::config::IncomingRequest>,
|
||||
rng: &mut impl Rng,
|
||||
) -> Self {
|
||||
if config.group_size == 0 {
|
||||
panic!("group size cannot be 0");
|
||||
}
|
||||
|
||||
let mut validators = Vec::new();
|
||||
let mut discovery_keys = Vec::new();
|
||||
let mut assignment_keys = Vec::new();
|
||||
let mut validator_groups = Vec::new();
|
||||
|
||||
let local_validator_pos = if config.local_validator {
|
||||
// ensure local validator is always in a full group.
|
||||
Some(rng.gen_range(0..config.validator_count).saturating_sub(config.group_size - 1))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
for i in 0..config.validator_count {
|
||||
let validator_pair = if Some(i) == local_validator_pos {
|
||||
// Note: the specific key is used to ensure the keystore holds
|
||||
// this key and the subsystem can detect that it is a validator.
|
||||
Sr25519Keyring::Ferdie.pair().into()
|
||||
} else {
|
||||
ValidatorPair::generate().0
|
||||
};
|
||||
let assignment_id = AssignmentPair::generate().0.public();
|
||||
let discovery_id = AuthorityDiscoveryPair::generate().0.public();
|
||||
|
||||
let group_index = i / config.group_size;
|
||||
validators.push(validator_pair);
|
||||
discovery_keys.push(discovery_id);
|
||||
assignment_keys.push(assignment_id);
|
||||
if validator_groups.len() == group_index {
|
||||
validator_groups.push(vec![ValidatorIndex(i as _)]);
|
||||
} else {
|
||||
validator_groups.last_mut().unwrap().push(ValidatorIndex(i as _));
|
||||
}
|
||||
}
|
||||
|
||||
let local = if let Some(local_pos) = local_validator_pos {
|
||||
Some(TestLocalValidator {
|
||||
validator_index: ValidatorIndex(local_pos as _),
|
||||
group_index: GroupIndex((local_pos / config.group_size) as _),
|
||||
})
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let validator_public = validator_pubkeys(&validators);
|
||||
let session_info = SessionInfo {
|
||||
validators: validator_public,
|
||||
discovery_keys,
|
||||
validator_groups: IndexedVec::from(validator_groups),
|
||||
assignment_keys,
|
||||
n_cores: 0,
|
||||
zeroth_delay_tranche_width: 0,
|
||||
relay_vrf_modulo_samples: 0,
|
||||
n_delay_tranches: 0,
|
||||
no_show_slots: 0,
|
||||
needed_approvals: 0,
|
||||
active_validator_indices: vec![],
|
||||
dispute_period: 6,
|
||||
random_seed: [0u8; 32],
|
||||
};
|
||||
|
||||
TestState { config, local, validators, session_info, req_sender }
|
||||
}
|
||||
|
||||
fn make_dummy_leaf(&self, relay_parent: Hash) -> TestLeaf {
|
||||
TestLeaf {
|
||||
number: 1,
|
||||
hash: relay_parent,
|
||||
parent_hash: Hash::repeat_byte(0),
|
||||
session: 1,
|
||||
availability_cores: self.make_availability_cores(|i| {
|
||||
CoreState::Scheduled(ScheduledCore {
|
||||
para_id: ParaId::from(i as u32),
|
||||
collator: None,
|
||||
})
|
||||
}),
|
||||
para_data: (0..self.session_info.validator_groups.len())
|
||||
.map(|i| (ParaId::from(i as u32), PerParaData::new(1, vec![1, 2, 3].into())))
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
fn make_availability_cores(&self, f: impl Fn(usize) -> CoreState) -> Vec<CoreState> {
|
||||
(0..self.session_info.validator_groups.len()).map(f).collect()
|
||||
}
|
||||
|
||||
fn make_dummy_topology(&self) -> NewGossipTopology {
|
||||
let validator_count = self.config.validator_count;
|
||||
NewGossipTopology {
|
||||
session: 1,
|
||||
topology: SessionGridTopology::new(
|
||||
(0..validator_count).collect(),
|
||||
(0..validator_count)
|
||||
.map(|i| TopologyPeerInfo {
|
||||
peer_ids: Vec::new(),
|
||||
validator_index: ValidatorIndex(i as u32),
|
||||
discovery_id: AuthorityDiscoveryPair::generate().0.public(),
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
local_index: self.local.as_ref().map(|local| local.validator_index),
|
||||
}
|
||||
}
|
||||
|
||||
fn group_validators(
|
||||
&self,
|
||||
group_index: GroupIndex,
|
||||
exclude_local: bool,
|
||||
) -> Vec<ValidatorIndex> {
|
||||
self.session_info
|
||||
.validator_groups
|
||||
.get(group_index)
|
||||
.unwrap()
|
||||
.iter()
|
||||
.cloned()
|
||||
.filter(|&i| {
|
||||
self.local.as_ref().map_or(true, |l| !exclude_local || l.validator_index != i)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn discovery_id(&self, validator_index: ValidatorIndex) -> AuthorityDiscoveryId {
|
||||
self.session_info.discovery_keys[validator_index.0 as usize].clone()
|
||||
}
|
||||
|
||||
fn sign_statement(
|
||||
&self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: CompactStatement,
|
||||
context: &SigningContext,
|
||||
) -> SignedStatement {
|
||||
let payload = statement.signing_payload(context);
|
||||
let pair = &self.validators[validator_index.0 as usize];
|
||||
let signature = pair.sign(&payload[..]);
|
||||
|
||||
SignedStatement::new(statement, validator_index, signature, context, &pair.public())
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
fn sign_full_statement(
|
||||
&self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: Statement,
|
||||
context: &SigningContext,
|
||||
pvd: PersistedValidationData,
|
||||
) -> SignedFullStatementWithPVD {
|
||||
let payload = statement.to_compact().signing_payload(context);
|
||||
let pair = &self.validators[validator_index.0 as usize];
|
||||
let signature = pair.sign(&payload[..]);
|
||||
|
||||
SignedFullStatementWithPVD::new(
|
||||
statement.supply_pvd(pvd),
|
||||
validator_index,
|
||||
signature,
|
||||
context,
|
||||
&pair.public(),
|
||||
)
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
// send a request out, returning a future which expects a response.
|
||||
async fn send_request(
|
||||
&mut self,
|
||||
peer: PeerId,
|
||||
request: AttestedCandidateRequest,
|
||||
) -> impl Future<Output = sc_network::config::OutgoingResponse> {
|
||||
let (tx, rx) = futures::channel::oneshot::channel();
|
||||
let req = sc_network::config::IncomingRequest {
|
||||
peer,
|
||||
payload: request.encode(),
|
||||
pending_response: tx,
|
||||
};
|
||||
self.req_sender.send(req).await.unwrap();
|
||||
|
||||
rx.map(|r| r.unwrap())
|
||||
}
|
||||
}
|
||||
|
||||
fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
config: TestConfig,
|
||||
test: impl FnOnce(TestState, VirtualOverseer) -> T,
|
||||
) {
|
||||
let pool = sp_core::testing::TaskExecutor::new();
|
||||
let keystore = if config.local_validator {
|
||||
test_helpers::mock::make_ferdie_keystore()
|
||||
} else {
|
||||
Arc::new(LocalKeystore::in_memory()) as KeystorePtr
|
||||
};
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, req_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let mut rng = rand_chacha::ChaCha8Rng::seed_from_u64(0);
|
||||
|
||||
let test_state = TestState::from_config(config, req_cfg.inbound_queue.unwrap(), &mut rng);
|
||||
|
||||
let (context, virtual_overseer) = test_helpers::make_subsystem_context(pool.clone());
|
||||
let subsystem = async move {
|
||||
let subsystem = crate::StatementDistributionSubsystem {
|
||||
keystore,
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
};
|
||||
|
||||
if let Err(e) = subsystem.run(context).await {
|
||||
panic!("Fatal error: {:?}", e);
|
||||
}
|
||||
};
|
||||
|
||||
let test_fut = test(test_state, virtual_overseer);
|
||||
|
||||
futures::pin_mut!(test_fut);
|
||||
futures::pin_mut!(subsystem);
|
||||
futures::executor::block_on(future::join(
|
||||
async move {
|
||||
let mut virtual_overseer = test_fut.await;
|
||||
// Ensure we have handled all responses.
|
||||
if let Ok(Some(msg)) = virtual_overseer.rx.try_next() {
|
||||
panic!("Did not handle all responses: {:?}", msg);
|
||||
}
|
||||
// Conclude.
|
||||
virtual_overseer.send(FromOrchestra::Signal(OverseerSignal::Conclude)).await;
|
||||
},
|
||||
subsystem,
|
||||
));
|
||||
}
|
||||
|
||||
struct PerParaData {
|
||||
min_relay_parent: BlockNumber,
|
||||
head_data: HeadData,
|
||||
}
|
||||
|
||||
impl PerParaData {
|
||||
pub fn new(min_relay_parent: BlockNumber, head_data: HeadData) -> Self {
|
||||
Self { min_relay_parent, head_data }
|
||||
}
|
||||
}
|
||||
|
||||
struct TestLeaf {
|
||||
number: BlockNumber,
|
||||
hash: Hash,
|
||||
parent_hash: Hash,
|
||||
session: SessionIndex,
|
||||
availability_cores: Vec<CoreState>,
|
||||
para_data: Vec<(ParaId, PerParaData)>,
|
||||
}
|
||||
|
||||
impl TestLeaf {
|
||||
pub fn para_data(&self, para_id: ParaId) -> &PerParaData {
|
||||
self.para_data
|
||||
.iter()
|
||||
.find_map(|(p_id, data)| if *p_id == para_id { Some(data) } else { None })
|
||||
.unwrap()
|
||||
}
|
||||
}
|
||||
|
||||
async fn activate_leaf(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
leaf: &TestLeaf,
|
||||
test_state: &TestState,
|
||||
expect_session_info_request: bool,
|
||||
) {
|
||||
let activated = ActivatedLeaf {
|
||||
hash: leaf.hash,
|
||||
number: leaf.number,
|
||||
status: LeafStatus::Fresh,
|
||||
span: Arc::new(jaeger::Span::Disabled),
|
||||
};
|
||||
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Signal(OverseerSignal::ActiveLeaves(ActiveLeavesUpdate::start_work(
|
||||
activated,
|
||||
))))
|
||||
.await;
|
||||
|
||||
handle_leaf_activation(virtual_overseer, leaf, test_state, expect_session_info_request).await;
|
||||
}
|
||||
|
||||
async fn handle_leaf_activation(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
leaf: &TestLeaf,
|
||||
test_state: &TestState,
|
||||
expect_session_info_request: bool,
|
||||
) {
|
||||
let TestLeaf { number, hash, parent_hash, para_data, session, availability_cores } = leaf;
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
) if parent == *hash => {
|
||||
tx.send(Ok(test_state.config.async_backing_params.unwrap_or(DEFAULT_ASYNC_BACKING_PARAMETERS))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let mrp_response: Vec<(ParaId, BlockNumber)> = para_data
|
||||
.iter()
|
||||
.map(|(para_id, data)| (*para_id, data.min_relay_parent))
|
||||
.collect();
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetMinimumRelayParents(parent, tx)
|
||||
) if parent == *hash => {
|
||||
tx.send(mrp_response).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let header = Header {
|
||||
parent_hash: *parent_hash,
|
||||
number: *number,
|
||||
state_root: Hash::zero(),
|
||||
extrinsics_root: Hash::zero(),
|
||||
digest: Default::default(),
|
||||
};
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ChainApi(
|
||||
ChainApiMessage::BlockHeader(parent, tx)
|
||||
) if parent == *hash => {
|
||||
tx.send(Ok(Some(header))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::SessionIndexForChild(tx))) if parent == *hash => {
|
||||
tx.send(Ok(*session)).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::AvailabilityCores(tx))) if parent == *hash => {
|
||||
tx.send(Ok(availability_cores.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let validator_groups = test_state.session_info.validator_groups.to_vec();
|
||||
let group_rotation_info =
|
||||
GroupRotationInfo { session_start_block: 1, group_rotation_frequency: 12, now: 1 };
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::ValidatorGroups(tx))) if parent == *hash => {
|
||||
tx.send(Ok((validator_groups, group_rotation_info))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
if expect_session_info_request {
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::SessionInfo(s, tx))) if parent == *hash && s == *session => {
|
||||
tx.send(Ok(Some(test_state.session_info.clone()))).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Intercepts an outgoing request, checks the fields, and sends the response.
|
||||
async fn handle_sent_request(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
candidate_hash: CandidateHash,
|
||||
mask: StatementFilter,
|
||||
candidate_receipt: CommittedCandidateReceipt,
|
||||
persisted_validation_data: PersistedValidationData,
|
||||
statements: Vec<UncheckedSignedStatement>,
|
||||
) {
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendRequests(mut requests, IfDisconnected::ImmediateError)) => {
|
||||
assert_eq!(requests.len(), 1);
|
||||
assert_matches!(
|
||||
requests.pop().unwrap(),
|
||||
Requests::AttestedCandidateVStaging(outgoing) => {
|
||||
assert_eq!(outgoing.peer, Recipient::Peer(peer));
|
||||
assert_eq!(outgoing.payload.candidate_hash, candidate_hash);
|
||||
assert_eq!(outgoing.payload.mask, mask);
|
||||
|
||||
let res = AttestedCandidateResponse {
|
||||
candidate_receipt,
|
||||
persisted_validation_data,
|
||||
statements,
|
||||
};
|
||||
outgoing.pending_response.send(Ok(res.encode())).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
async fn answer_expected_hypothetical_depth_request(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
responses: Vec<(HypotheticalCandidate, FragmentTreeMembership)>,
|
||||
expected_leaf_hash: Option<Hash>,
|
||||
expected_backed_in_path_only: bool,
|
||||
) {
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetHypotheticalFrontier(req, tx)
|
||||
) => {
|
||||
assert_eq!(req.fragment_tree_relay_parent, expected_leaf_hash);
|
||||
assert_eq!(req.backed_in_path_only, expected_backed_in_path_only);
|
||||
for (i, (candidate, _)) in responses.iter().enumerate() {
|
||||
assert!(
|
||||
req.candidates.iter().any(|c| &c == &candidate),
|
||||
"did not receive request for hypothetical candidate {}",
|
||||
i,
|
||||
);
|
||||
}
|
||||
|
||||
tx.send(responses).unwrap();
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
fn validator_pubkeys(val_ids: &[ValidatorPair]) -> IndexedVec<ValidatorIndex, ValidatorId> {
|
||||
val_ids.iter().map(|v| v.public().into()).collect()
|
||||
}
|
||||
|
||||
async fn connect_peer(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
authority_ids: Option<HashSet<AuthorityDiscoveryId>>,
|
||||
) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Authority,
|
||||
ValidationVersion::VStaging.into(),
|
||||
authority_ids,
|
||||
),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
// TODO: Add some tests using this?
|
||||
#[allow(dead_code)]
|
||||
async fn disconnect_peer(virtual_overseer: &mut VirtualOverseer, peer: PeerId) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerDisconnected(peer),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn send_peer_view_change(virtual_overseer: &mut VirtualOverseer, peer: PeerId, view: View) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerViewChange(peer, view),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn send_peer_message(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
message: protocol_vstaging::StatementDistributionMessage,
|
||||
) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(peer, Versioned::VStaging(message)),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn send_new_topology(virtual_overseer: &mut VirtualOverseer, topology: NewGossipTopology) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::NewGossipTopology(topology),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn overseer_recv_with_timeout(
|
||||
overseer: &mut VirtualOverseer,
|
||||
timeout: Duration,
|
||||
) -> Option<AllMessages> {
|
||||
gum::trace!("waiting for message...");
|
||||
overseer.recv().timeout(timeout).await
|
||||
}
|
||||
|
||||
fn next_group_index(
|
||||
group_index: GroupIndex,
|
||||
validator_count: usize,
|
||||
group_size: usize,
|
||||
) -> GroupIndex {
|
||||
let next_group = group_index.0 + 1;
|
||||
let num_groups =
|
||||
validator_count / group_size + if validator_count % group_size > 0 { 1 } else { 0 };
|
||||
GroupIndex::from(next_group % num_groups as u32)
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user