mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-11-16 07:37:06 +00:00
Asynchronous Backing MegaPR (#5022)
* inclusion emulator logic for asynchronous backing (#4790) * initial stab at candidate_context * fmt * docs & more TODOs * some cleanups * reframe as inclusion_emulator * documentations yes * update types * add constraint modifications * watermark * produce modifications * v2 primitives: re-export all v1 for consistency * vstaging primitives * emulator constraints: handle code upgrades * produce outbound HRMP modifications * stack. * method for applying modifications * method just for sanity-checking modifications * fragments produce modifications, not prospectives * make linear * add some TODOs * remove stacking; handle code upgrades * take `fragment` private * reintroduce stacking. * fragment constructor * add TODO * allow validating fragments against future constraints * docs * relay-parent number and min code size checks * check code upgrade restriction * check max hrmp per candidate * fmt * remove GoAhead logic because it wasn't helpful * docs on code upgrade failure * test stacking * test modifications against constraints * fmt * test fragments * descending or duplicate test * fmt * remove unused imports in vstaging * wrong primitives * spellcheck * Runtime changes for Asynchronous Backing (#4786) * inclusion: utility for allowed relay-parents * inclusion: use prev number instead of prev hash * track most recent context of paras * inclusion: accept previous relay-parents * update dmp advancement rule for async backing * fmt * add a comment about validation outputs * clean up a couple of TODOs * weights * fix weights * fmt * Resolve dmp todo * Restore inclusion tests * Restore paras_inherent tests * MostRecentContext test * Benchmark for new paras dispatchable * Prepare check_validation_outputs for upgrade * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs * Implementers guide changes * More tests for allowed relay parents * Add a github issue link * Compute group index based on relay parent * Storage migration * Move allowed parents tracker to shared * Compile error * Get group assigned to core at the next block * Test group assignment * fmt * Error instead of panic * Update guide * Extend doc-comment * Update runtime/parachains/src/shared.rs Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Prospective Parachains Subsystem (#4913) * docs and skeleton * subsystem skeleton * main loop * fragment tree basics & fmt * begin fragment trees & view * flesh out more of view update logic * further flesh out update logic * some refcount functions for fragment trees * add fatal/non-fatal errors * use non-fatal results * clear up some TODOs * ideal format for scheduling info * add a bunch of TODOs * some more fluff * extract fragment graph to submodule * begin fragment graph API * trees, not graphs * improve docs * scope and constructor for trees * add some test TODOs * limit max ancestors and store constraints * constructor * constraints: fix bug in HRMP watermarks * fragment tree population logic * set::retain * extract population logic * implement add_and_populate * fmt * add some TODOs in tests * implement child-selection * strip out old stuff based on wrong assumptions * use fatality * implement pruning * remove unused ancestor constraints * fragment tree instantiation * remove outdated comment * add message/request types and skeleton for handling * fmt * implement handle_candidate_seconded * candidate storage: handle backed * implement handle_candidate_backed * implement answer_get_backable_candidate * remove async where not needed * implement fetch_ancestry * add logic for run_iteration * add some docs * remove global allow(unused), fix warnings * make spellcheck happy (despite English) * fmt * bump Cargo.lock * replace tracing with gum * introduce PopulateFrom trait * implement GetHypotheticalDepths * revise docs slightly * first fragment tree scope test * more scope tests * test add_candidate * fmt * test retain * refactor test code * test populate is recursive * test contiguity of depth 0 is maintained * add_and_populate tests * cycle tests * remove PopulateFrom trait * fmt * test hypothetical depths (non-recursive) * have CandidateSeconded return membership * tree membership requests * Add a ProspectiveParachainsSubsystem struct * add a staging API for base constraints * add a `From` impl * add runtime API for staging_validity_constraints * implement fetch_base_constraints * implement `fetch_upcoming_paras` * remove reconstruction of candidate receipt; no obvious usecase * fmt * export message to broader module * remove last TODO * correctly export * fix compilation and add GetMinimumRelayParent request * make provisioner into a real subsystem with proper mesage bounds * fmt * fix ChannelsOut in overseer test * fix overseer tests * fix again * fmt * Integrate prospective parachains subsystem into backing: Part 1 (#5557) * BEGIN ASYNC candidate-backing CHANGES * rename & document modes * answer prospective validation data requests * GetMinimumRelayParents request is now plural * implement an implicit view utility for backing subsystems * implicit-view: get allowed relay parents * refactorings and improvements to implicit view * add some TODOs for tests * split implicit view updates into 2 functions * backing: define State to prepare for functional refactor * add some docs * backing: implement bones of new leaf activation logic * backing: create per-relay-parent-states * use new handle_active_leaves_update * begin extracting logic from CandidateBackingJob * mostly extract statement import from job logic * handle statement imports outside of job logic * do some TODO planning for prospective parachains integration * finish rewriting backing subsystem in functional style * add prospective parachains mode to relay parent entries * fmt * add a RejectedByProspectiveParachains error * notify prospective parachains of seconded and backed candidates * always validate candidates exhaustively in backing. * return persisted_validation_data from validation * handle rejections by prospective parachains * implement seconding sanity check * invoke validate_and_second * Alter statement table to allow multiple seconded messages per validator * refactor backing to have statements carry PVD * clean up all warnings * Add tests for implicit view * Improve doc comments * Prospective parachains mode based on Runtime API version * Add a TODO * Rework seconding_sanity_check * Iterate over responses * Update backing tests * collator-protocol: load PVD from runtime * Fix validator side tests * Update statement-distribution to fetch PVD * Fix statement-distribution tests * Backing tests with prospective paras #1 * fix per_relay_parent pruning in backing * Test multiple leaves * Test seconding sanity check * Import statement order Before creating an entry in `PerCandidateState` map wait for the approval from the prospective parachains * Add a test for correct state updates * Second multiple candidates per relay parent test * Add backing tests with prospective paras * Second more than one test without prospective paras * Add a test for prospective para blocks * Update malus * typos Co-authored-by: Chris Sosnin <chris125_@live.com> * Track occupied depth in backing per parachain (#5778) * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * fmt * Network bridge changes for asynchronous backing + update subsystems to handle versioned packets (#5991) * BEGIN STATEMENT DISTRIBUTION WORK create a vstaging network protocol which is the same as v1 * mostly make network bridge amenable to vstaging * network-bridge: fully adapt to vstaging * add some TODOs for tests * fix fallout in bitfield-distribution * bitfield distribution tests + TODOs * fix fallout in gossip-support * collator-protocol: fix message fallout * collator-protocol: load PVD from runtime * add TODO for vstaging tests * make things compile * set used network protocol version using a feature * fmt * get approval-distribution building * fix approval-distribution tests * spellcheck * nits * approval distribution net protocol test * bitfield distribution net protocol test * Revert "collator-protocol: fix message fallout" This reverts commit 07cc887303e16c6b3843ecb25cdc7cc2080e2ed1. * Network bridge tests Co-authored-by: Chris Sosnin <chris125_@live.com> * remove max_pov_size requirement from prospective pvd request (#6014) * remove max_pov_size requirement from prospective pvd request * fmt * Extract legacy statement distribution to its own module (#6026) * add compatibility type to v2 statement distribution message * warning cleanup * handle compatibility layer for v2 * clean up an unimplemented!() block * circulate statements based on version * extract legacy v1 code into separate module * remove unimplemented * clean up naming of from_requester/responder * remove TODOs * have backing share seconded statements with PVD * fmt * fix warning * Quick fix unused warning for not yet implemented/used staging messages. * Fix network bridge test * Fix wrong merge. We now have 23 subsystems (network bridge split + prospective parachains) Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at> * Version 3 is already live. * Fix tests (#6055) * Fix backing tests * Fix warnings. * fmt * collator-protocol: asynchronous backing changes (#5740) * Draft collator side changes * Start working on collations management * Handle peer's view change * Versioning on advertising * Versioned collation fetching request * Handle versioned messages * Improve docs for collation requests * Add spans * Add request receiver to overseer * Fix collator side tests * Extract relay parent mode to lib * Validator side draft * Add more checks for advertisement * Request pvd based on async backing mode * review * Validator side improvements * Make old tests green * More fixes * Collator side tests draft * Send collation test * fmt * Collator side network protocol versioning * cleanup * merge artifacts * Validator side net protocol versioning * Remove fragment tree membership request * Resolve todo * Collator side core state test * Improve net protocol compatibility * Validator side tests * more improvements * style fixes * downgrade log * Track implicit assignments * Limit the number of seconded candidates per para * Add a sanity check * Handle fetched candidate * fix tests * Retry fetch * Guard against dequeueing while already fetching * Reintegrate connection management * Timeout on advertisements * fmt * spellcheck * update tests after merge * validator assignment fixes for backing and collator protocol (#6158) * Rename depth->ancestry len in tests * Refactor group assignments * Remove implicit assignments * backing: consider occupied core assignments * Track a single para on validator side * Refactor prospective parachains mode request (#6179) * Extract prospective parachains mode into util * Skip activations depending on the mode * backing: don't send backed candidate to provisioner (#6185) * backing: introduce `CanSecond` request for advertisements filtering (#6225) * Drop BoundToRelayParent * draft changes * fix backing tests * Fix genesis ancestry * Fix validator side tests * more tests * cargo generate-lockfile * Implement `StagingValidityConstraints` Runtime API method (#6258) * Implement StagingValidityConstraints * spellcheck * fix ump params * Update hrmp comment * Introduce ump per candidate limit * hypothetical earliest block * refactor primitives usage * hypothetical earliest block number test * fix build * Prepare the Runtime for asynchronous backing upgrade (#6287) * Introduce async backing params to runtime config * fix cumulus config * use config * finish runtimes * Introduce new staging API * Update collator protocol * Update provisioner * Update prospective parachains * Update backing * Move async backing params lower in the config * make naming consistent * misc * Use real prospective parachains subsystem (#6407) * Backport `HypotheticalFrontier` into the feature branch (#6605) * implement more general HypotheticalFrontier * fmt * drop unneeded request Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Resolve todo about legacy leaf activation (#6447) * fix bug/warning in handling membership answers * Remove `HypotheticalDepthRequest` in favor of `HypotheticalFrontierRequest` (#6521) * Remove `HypotheticalDepthRequest` for `HypotheticalFrontierRequest` * Update tests * Fix (removed wrong docstring) * Fix can_second request * Patch some dead_code errors --------- Co-authored-by: Chris Sosnin <chris125_@live.com> * Async Backing: Send Statement Distribution "Backed" messages (#6634) * Backing: Send Statement Distribution "Backed" messages Closes #6590. **TODO:** - [ ] Adjust tests * Fix compile errors * (Mostly) fix tests * Fix comment * Fix test and compile error * Test that `StatementDistributionMessage::Backed` is sent * Fix compile error * Fix some clippy errors * Add prospective parachains subsystem tests (#6454) * Add prospective parachains subsystem test * Add `should_do_no_work_if_async_backing_disabled_for_leaf` test * Implement `activate_leaf` helper, up to getting ancestry * Finish implementing `activate_leaf` * Small refactor in `activate_leaf` * Get `CandidateSeconded` working * Finish `send_candidate_and_check_if_found` test * Refactor; send more leaves & candidates * Refactor test * Implement `check_candidate_parent_leaving_view` test * Start work on `check_candidate_on_multiple_forks` test * Don’t associate specific parachains with leaf * Finish `correctly_updates_leaves` test * Fix cycle due to reused head data * Fix `check_backable_query` test * Fix `check_candidate_on_multiple_forks` test * Add `check_depth_and_pvd_queries` test * Address review comments * Remove TODO * add a new index for output head data to candidate storage * Resolve test TODOs * Fix compile errors * test candidate storage pruning, make sure new index is cleaned up --------- Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Node-side metrics for asynchronous backing (#6549) * Add metrics for `prune_view_candidate_storage` * Add metrics for `request_unblocked_collations` * Fix docstring * Couple fixes from review comments * Fix `check_depth_query` test * inclusion-emulator: mirror advancement rule check (#6361) * inclusion-emulator: mirror advancement rule check * fix build * prospective-parachains: introduce `backed_in_path_only` flag for advertisements (#6649) * Introduce `backed_in_path_only` flag for depth request * fmt * update doc comment * fmt * Add async-backing zombienet tests (#6314) * Async backing: impl guide for statement distribution (#6738) Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com> * Asynchronous backing statement distribution: Take III (#5999) * add notification types for v2 statement-distribution * improve protocol docs * add empty vstaging module * fmt * add backed candidate packet request types * start putting down structure of new logic * handle activated leaf * some sanity-checking on outbound statements * fmt * update vstaging share to use statements with PVD * tiny refactor, candidate_hash location * import local statements * refactor statement import * first stab at broadcast logic * fmt * fill out some TODOs * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Mark receiver with `vstaging` * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * fmt, fix grid test after topology change * typo Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * address review * adjust comment, make easier to understand * Fix typo --------- Co-authored-by: Marcin S <marcin@bytedude.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Chris Sosnin <chris125_@live.com> * miscellaneous fixes to make asynchronous backing work (#6791) * propagate network-protocol-staging feature * add feature to adder-collator as well * allow collation-generation of occupied cores * prospective parachains: special treatment for pending availability candidates * runtime: fetch candidates pending availability * lazily construct PVD for pending candidates * fix fallout in prospective parachains hypothetical/select_child * runtime: enact candidates when creating paras-inherent * make tests compile * test pending availability in the scope * add prospective parachains test * fix validity constraints leftovers * drop prints * Fix typos --------- Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Marcin S <marcin@realemail.net> * Remove restart from test (#6840) * Async Backing: Statement Distribution Tests (#6755) * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * Hook up request sender * Add `valid_statement_without_prior_seconded_is_ignored` test * Fix `valid_statement_without_prior_seconded_is_ignored` test * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Remove obsolete test * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Mark receiver with `vstaging` * First draft of `ensure_seconding_limit_is_respected` test * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * Fix `ensure_seconding_limit_is_respected` test * Start `backed_candidate_leads_to_advertisement` test * fmt, fix grid test after topology change * Send Backed notification * Finish `backed_candidate_leads_to_advertisement` test * Finish `peer_reported_for_duplicate_statements` test * Finish `received_advertisement_before_confirmation_leads_to_request` * Add `advertisements_rejected_from_incorrect_peers` test * Add `manifest_rejected_*` tests * Add `manifest_rejected_when_group_does_not_match_para` test * Add `local_node_sanity_checks_incoming_requests` test * Add `local_node_respects_statement_mask` test * Add tests where peer is reported for providing invalid signatures * Add `cluster_peer_allowed_to_send_incomplete_statements` test * Add `received_advertisement_after_backing_leads_to_acknowledgement` * Add `received_advertisement_after_confirmation_before_backing` test * peer_reported_for_advertisement_conflicting_with_confirmed_candidate * Add `peer_reported_for_not_enough_statements` test * Add `peer_reported_for_providing_statements_meant_to_be_masked_out` * Add `additional_statements_are_shared_after_manifest_exchange` * Add `grid_statements_imported_to_backing` test * Add `relay_parent_entering_peer_view_leads_to_advertisement` test * Add `advertisement_not_re_sent_when_peer_re_enters_view` test * Update node/network/statement-distribution/src/vstaging/tests/grid.rs Co-authored-by: asynchronous rob <rphmeier@gmail.com> * Resolve TODOs, update test * Address unused code * Add check after every test for unhandled requests * Refactor (`make_dummy_leaf` and `handle_sent_request`) * Refactor (`make_dummy_topology`) * Minor refactor --------- Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Chris Sosnin <chris125_@live.com> * Fix some clippy lints in tests * Async backing: minor fixes (#6920) * bitfield-distribution test * implicit view tests * Refactor parameters -> params * scheduler: update storage migration (#6963) * update scheduler migration * Adjust weight to account for storage read * Statement Distribution Guide Edits (#7025) * Statement distribution guide edits * Addressed Marcin's comments * Add attested candidate request retry timeouts (#6833) Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <chris125_@live.com> Fix async backing statement distribution tests (#6621) Resolve some todos in async backing statement-distribution branch (#6482) Fix clippy errors in statement distribution branch (#6720) * Async backing: add Prospective Parachains impl guide (#6933) Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> * Updates to Provisioner Guide for Async Backing (#7106) * Initial corrections and clarifications * Partial first draft * Finished first draft * Adding back wrongly removed test bit * fmt * Update roadmap/implementers-guide/src/node/utility/provisioner.md Co-authored-by: Marcin S. <marcin@realemail.net> * Addressing comments * Reorganization * fmt --------- Co-authored-by: Marcin S. <marcin@realemail.net> * fmt * Renaming Parathread Mentions (#7287) * Renaming parathreads * Renaming module to pallet * More updates * PVF: Refactor workers into separate crates, remove host dependency (#7253) * PVF: Refactor workers into separate crates, remove host dependency * Fix compile error * Remove some leftover code * Fix compile errors * Update Cargo.lock * Remove worker main.rs files I accidentally copied these from the other PR. This PR isn't intended to introduce standalone workers yet. * Address review comments * cargo fmt * Update a couple of comments * Update log targets * Update quote to 1.0.27 (#7280) Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: parity-processbot <> * pallets: implement `Default` for `GenesisConfig` in `no_std` (#7271) * pallets: implement Default for GenesisConfig in no_std This change is follow-up of: https://github.com/paritytech/substrate/pull/14108 It is a step towards: https://github.com/paritytech/substrate/issues/13334 * Cargo.lock updated * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * cli: enable BEEFY by default on test networks (#7293) We consider BEEFY mature enough to run by default on all nodes for test networks (Rococo/Wococo/Versi). Right now, most nodes are not running it since it's opt-in using --beefy flag. Switch to an opt-out model for test networks. Replace --beefy flag from CLI with --no-beefy and have BEEFY client start by default on test networks. Signed-off-by: acatangiu <adrian@parity.io> * runtime: past session slashing runtime API (#6667) * runtime/vstaging: unapplied_slashes runtime API * runtime/vstaging: key_ownership_proof runtime API * runtime/ParachainHost: submit_report_dispute_lost * fix key_ownership_proof API * runtime: submit_report_dispute_lost runtime API * nits * Update node/subsystem-types/src/messages.rs Co-authored-by: Marcin S. <marcin@bytedude.com> * revert unrelated fmt changes * post merge fixes * fix compilation --------- Co-authored-by: Marcin S. <marcin@bytedude.com> * Correcting git mishap * Document usage of `gum` crate (#7294) * Document usage of gum crate * Small fix * Add some more basic info * Update node/gum/src/lib.rs Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * Update target docs --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * XCM: Fix issue with RequestUnlock (#7278) * XCM: Fix issue with RequestUnlock * Leave API changes for v4 * Fix clippy errors * Fix tests --------- Co-authored-by: parity-processbot <> * Companion for Substrate#14228 (#7295) * Companion for Substrate#14228 https://github.com/paritytech/substrate/pull/14228 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * Companion for #14237: Use latest sp-crates (#7300) * To revert: Update substrate branch to "lexnv/bump_sp_crates" Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert "To revert: Update substrate branch to "lexnv/bump_sp_crates"" This reverts commit 5f1db84eac4a226c37b7f6ce6ee19b49dc7e2008. * Update cargo lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * bounded-collections bump to 0.1.7 (#7305) * bounded-collections bump to 0.1.7 Companion for: paritytech/substrate#14225 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * bump to quote 1.0.28 (#7306) * `RollingSessionWindow` cleanup (#7204) * Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit * Fix tests in import * Fix the rest of the tests * Remove dead code * Fix todos * Simplify session caching * Comments for `SessionInfoProvider` * Separate `SessionInfoProvider` from `State` * `cache_session_info_for_head` becomes freestanding function * Remove unneeded `mut` usage * fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider` * Fix SessionInfo retrieval * Code cleanup * Don't wrap `SessionInfoProvider` in an `Option` * Remove `earliest_session()` * Remove pre-caching -> wip * Fix some tests and code cleanup * Fix all tests * Fixes in tests * Fix comments, variable names and small style changes * Fix a warning * impl From<SessionWindowSize> for NonZeroUsize * Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG * Code review feedback * Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db * Remove `col_session_data` usages * Storage migration clearing columns w/o removing them * Remove session data column usages from `approval-voting` and `dispute-coordinator` tests * Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests * Fix formatting in initialized.rs * Fix a corner case in `SessionInfo` caching for `dispute-coordinator` * Remove `RollingSessionWindow` ;( * Revert "Fix formatting in initialized.rs" This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc. * v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it * Fix `NUM_COLUMNS` in `approval-voting` * Use `columns::v3::NUM_COLUMNS` when opening db * Update node/service/src/parachains_db/upgrade.rs Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3` * Fix `NUM+COLUMNS` in approval_voting * Fix formatting * Fix columns usage * Clarification comments about the different db versions --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * pallet-para-config: Remove remnant WeightInfo functions (#7308) * pallet-para-config: Remove remnant WeightInfo functions Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * set_config_with_weight begone Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::configuration --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: command-bot <> * XCM: PayOverXcm config (#6900) * Move XCM query functionality to trait * Fix tests * Add PayOverXcm implementation * fix the PayOverXcm trait to compile * moved doc comment out of trait implmeentation and to the trait * PayOverXCM documentation * Change documentation a bit * Added empty benchmark methods implementation and changed docs * update PayOverXCM to convert AccountIds to MultiLocations * Implement benchmarking method * Change v3 to latest * Descend origin to an asset sender (#6970) * descend origin to an asset sender * sender as tuple of dest and sender * Add more variants to the QueryResponseStatus enum * Change Beneficiary to Into<[u8; 32]> * update PayOverXcm to return concrete errors and use AccountId as sender * use polkadot-primitives for AccountId * fix dependency to use polkadot-core-primitives * force Unpaid instruction to the top of the instructions list * modify report_outcome to accept interior argument * use new_query directly for building final xcm query, instead of report_outcome * fix usage of new_query to use the XcmQueryHandler * fix usage of new_query to use the XcmQueryHandler * tiny method calling fix * xcm query handler (#7198) * drop redundant query status * rename ReportQueryStatus to OuterQueryStatus * revert rename of QueryResponseStatus * update mapping * Update xcm/xcm-builder/src/pay.rs Co-authored-by: Gavin Wood <gavin@parity.io> * Updates * Docs * Fix benchmarking stuff * Destination can be determined based on asset_kind * Tweaking API to minimise clones * Some repotting and docs --------- Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> * Companion for #14265 (#7307) * Update Cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update Cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: parity-processbot <> * bump serde to 1.0.163 (#7315) * bump serde to 1.0.163 * bump ci * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * fmt * Updated fmt * Removing changes accidentally pulled from master * fix another master pull issue * Another master pull fix * fmt * Fixing implementers guide build * Revert "Merge branch 'rh-async-backing-feature-while-frozen' of https://github.com/paritytech/polkadot into brad-rename-parathread" This reverts commit bebc24af52ab61155e3fe02cb3ce66a592bce49c, reversing changes made to 1b2de662dfb11173679d6da5bd0da9d149c85547. --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: acatangiu <adrian@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Marcin S. <marcin@bytedude.com> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> * fix bitfield distribution test * approval distribution tests * fix bridge tests * update Cargo.lock * [async-backing-branch] Optimize collator-protocol validator-side request fetching (#7457) * Optimize collator-protocol validator-side request fetching * address feedback: replace tuples with structs * feedback: add doc comments * move collation types to subfolder --------- Signed-off-by: alindima <alin@parity.io> * Update collation generation for asynchronous backing (#7405) * break candidate receipt construction and distribution into own function * update implementers' guide to include SubmitCollation * implement SubmitCollation for collation-generation * fmt * fix test compilation & remove unnecessary submodule * add some TODOs for a test suite. * Update roadmap/implementers-guide/src/types/overseer-protocol.md Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * add new test harness and first test * refactor to avoid requiring background sender * ensure collation gets packaged and distributed * tests for the fallback case with no hint * add parent rp-number hint tests * fmt * update uses of CollationGenerationConfig * fix remaining test * address review comments * use subsystemsender for background tasks * fmt * remove ValidationCodeHashHint and related tests --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * fix some more fallout from merge * fmt * remove staging APIs from Rococo & Westend (#7513) * send network messages on main protocol name (#7515) * misc async backing improvements for allowed ancestry blocks (#7532) * shared: fix acquire_info * backwards-compat test for prospective parachains * same relay parent is allowed * provisioner: request candidate receipt by relay parent (#7527) * return candidates hash from prospective parachains * update provisioner * update tests * guide changes * send a single message to backing * fix test * revert to old `handle_new_activations` logic in some cases (#7514) * revert to old `handle_new_activations` logic * gate sending messages on scheduled cores to max_depth >= 2 * fmt * 2->1 * Omnibus asynchronous backing bugfix PR (#7529) * fix a bug in backing * add some more logs * prospective parachains: take ancestry only up to session bounds * add test * fix zombienet tests (#7614) Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix runtime compilation * make bitfield distribution tests compile * attempt to fix zombienet disputes (#7618) * update metric name * update some metric names * avoid cycles when creating fake candidates * make undying collator more friendly to malformed parents * fix a bug in malus * fmt * clippy * add RUN_IN_CONTAINER to new ZombieNet tests (#7631) * remove duplicated migration happened because of master-merge --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: acatangiu <adrian@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Signed-off-by: alindima <alin@parity.io> Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at> Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com> Co-authored-by: Marcin S <marcin@bytedude.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Mattia L.V. Bradascio <28816406+bredamatt@users.noreply.github.com> Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com> Co-authored-by: BradleyOlson64 <lotrftw9@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Alin Dima <alin@parity.io>
This commit is contained in:
+17









asynchronous rob
Marcin S
Oliver Tale-Yazdi
Michal Kucharczyk
Adrian Catangiu
ordian
Marcin S.
Andrei Sandu
Francisco Aguirre
Bastian Köcher
Alexandru Vasile
Sam Johnson
Tsvetomir Dimitrov
Anthony Alaribe
Muharem Ismailov
Anthony Alaribe
Gavin Wood
Chris Sosnin
Parity Bot
Chris Sosnin
Robert Klotzner
Robert Klotzner
Mattia L.V. Bradascio
Bradley Olson
alexgparity
BradleyOlson64
Alin Dima authored and
GitHub committed 2023-08-18 16:11:56 +00:00
1 parent
ad539f0e41
commit
5174b9d2d7
175 files changed
+40882
-6462
No files matched your search
@@ -25,8 +25,9 @@ use polkadot_node_jaeger as jaeger;
|
||||
use polkadot_node_network_protocol::{
|
||||
self as net_protocol,
|
||||
grid_topology::{RandomRouting, RequiredRouting, SessionGridTopologies, SessionGridTopology},
|
||||
peer_set::MAX_NOTIFICATION_SIZE,
|
||||
v1 as protocol_v1, PeerId, UnifiedReputationChange as Rep, Versioned, View,
|
||||
peer_set::{ValidationVersion, MAX_NOTIFICATION_SIZE},
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, PeerId, UnifiedReputationChange as Rep,
|
||||
Versioned, VersionedValidationProtocol, View,
|
||||
};
|
||||
use polkadot_node_primitives::approval::{
|
||||
AssignmentCert, BlockApprovalMeta, IndirectAssignmentCert, IndirectSignedApprovalVote,
|
||||
@@ -159,6 +160,15 @@ enum Resend {
|
||||
No,
|
||||
}
|
||||
|
||||
/// Data stored on a per-peer basis.
|
||||
#[derive(Debug)]
|
||||
struct PeerData {
|
||||
/// The peer's view.
|
||||
view: View,
|
||||
/// The peer's protocol version.
|
||||
version: ValidationVersion,
|
||||
}
|
||||
|
||||
/// The [`State`] struct is responsible for tracking the overall state of the subsystem.
|
||||
///
|
||||
/// It tracks metadata about our view of the unfinalized chain,
|
||||
@@ -179,7 +189,7 @@ struct State {
|
||||
pending_known: HashMap<Hash, Vec<(PeerId, PendingMessage)>>,
|
||||
|
||||
/// Peer data is partially stored here, and partially inline within the [`BlockEntry`]s
|
||||
peer_views: HashMap<PeerId, View>,
|
||||
peer_data: HashMap<PeerId, PeerData>,
|
||||
|
||||
/// Keeps a topology for various different sessions.
|
||||
topologies: SessionGridTopologies,
|
||||
@@ -349,14 +359,30 @@ impl State {
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
match event {
|
||||
NetworkBridgeEvent::PeerConnected(peer_id, role, _, _) => {
|
||||
NetworkBridgeEvent::PeerConnected(peer_id, role, version, _) => {
|
||||
// insert a blank view if none already present
|
||||
gum::trace!(target: LOG_TARGET, ?peer_id, ?role, "Peer connected");
|
||||
self.peer_views.entry(peer_id).or_default();
|
||||
let version = match ValidationVersion::try_from(version).ok() {
|
||||
Some(v) => v,
|
||||
None => {
|
||||
// sanity: network bridge is supposed to detect this already.
|
||||
gum::error!(
|
||||
target: LOG_TARGET,
|
||||
?peer_id,
|
||||
?version,
|
||||
"Unsupported protocol version"
|
||||
);
|
||||
return
|
||||
},
|
||||
};
|
||||
|
||||
self.peer_data
|
||||
.entry(peer_id)
|
||||
.or_insert_with(|| PeerData { version, view: Default::default() });
|
||||
},
|
||||
NetworkBridgeEvent::PeerDisconnected(peer_id) => {
|
||||
gum::trace!(target: LOG_TARGET, ?peer_id, "Peer disconnected");
|
||||
self.peer_views.remove(&peer_id);
|
||||
self.peer_data.remove(&peer_id);
|
||||
self.blocks.iter_mut().for_each(|(_hash, entry)| {
|
||||
entry.known_by.remove(&peer_id);
|
||||
})
|
||||
@@ -393,12 +419,12 @@ impl State {
|
||||
live
|
||||
});
|
||||
},
|
||||
NetworkBridgeEvent::PeerMessage(peer_id, msg) => {
|
||||
self.process_incoming_peer_message(ctx, metrics, peer_id, msg, rng).await;
|
||||
},
|
||||
NetworkBridgeEvent::UpdatedAuthorityIds { .. } => {
|
||||
// The approval-distribution subsystem doesn't deal with `AuthorityDiscoveryId`s.
|
||||
},
|
||||
NetworkBridgeEvent::PeerMessage(peer_id, Versioned::V1(msg)) => {
|
||||
self.process_incoming_peer_message(ctx, metrics, peer_id, msg, rng).await;
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -455,16 +481,18 @@ impl State {
|
||||
|
||||
{
|
||||
let sender = ctx.sender();
|
||||
for (peer_id, view) in self.peer_views.iter() {
|
||||
let intersection = view.iter().filter(|h| new_hashes.contains(h));
|
||||
let view_intersection = View::new(intersection.cloned(), view.finalized_number);
|
||||
for (peer_id, data) in self.peer_data.iter() {
|
||||
let intersection = data.view.iter().filter(|h| new_hashes.contains(h));
|
||||
let view_intersection =
|
||||
View::new(intersection.cloned(), data.view.finalized_number);
|
||||
Self::unify_with_peer(
|
||||
sender,
|
||||
metrics,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
self.peer_views.len(),
|
||||
self.peer_data.len(),
|
||||
*peer_id,
|
||||
data.version,
|
||||
view_intersection,
|
||||
rng,
|
||||
)
|
||||
@@ -547,6 +575,7 @@ impl State {
|
||||
|
||||
adjust_required_routing_and_propagate(
|
||||
ctx,
|
||||
&self.peer_data,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
|block_entry| block_entry.session == session,
|
||||
@@ -566,13 +595,16 @@ impl State {
|
||||
ctx: &mut Context,
|
||||
metrics: &Metrics,
|
||||
peer_id: PeerId,
|
||||
msg: protocol_v1::ApprovalDistributionMessage,
|
||||
msg: net_protocol::ApprovalDistributionMessage,
|
||||
rng: &mut R,
|
||||
) where
|
||||
R: CryptoRng + Rng,
|
||||
{
|
||||
match msg {
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments) => {
|
||||
Versioned::V1(protocol_v1::ApprovalDistributionMessage::Assignments(assignments)) |
|
||||
Versioned::VStaging(protocol_vstaging::ApprovalDistributionMessage::Assignments(
|
||||
assignments,
|
||||
)) => {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
peer_id = %peer_id,
|
||||
@@ -611,7 +643,10 @@ impl State {
|
||||
.await;
|
||||
}
|
||||
},
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals) => {
|
||||
Versioned::V1(protocol_v1::ApprovalDistributionMessage::Approvals(approvals)) |
|
||||
Versioned::VStaging(protocol_vstaging::ApprovalDistributionMessage::Approvals(
|
||||
approvals,
|
||||
)) => {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
peer_id = %peer_id,
|
||||
@@ -664,9 +699,14 @@ impl State {
|
||||
{
|
||||
gum::trace!(target: LOG_TARGET, ?view, "Peer view change");
|
||||
let finalized_number = view.finalized_number;
|
||||
let old_view =
|
||||
self.peer_views.get_mut(&peer_id).map(|d| std::mem::replace(d, view.clone()));
|
||||
let old_finalized_number = old_view.map(|v| v.finalized_number).unwrap_or(0);
|
||||
let (peer_protocol_version, old_finalized_number) = match self
|
||||
.peer_data
|
||||
.get_mut(&peer_id)
|
||||
.map(|d| (d.version, std::mem::replace(&mut d.view, view.clone())))
|
||||
{
|
||||
Some((v, view)) => (v, view.finalized_number),
|
||||
None => return, // unknown peer
|
||||
};
|
||||
|
||||
// we want to prune every block known_by peer up to (including) view.finalized_number
|
||||
let blocks = &mut self.blocks;
|
||||
@@ -691,8 +731,9 @@ impl State {
|
||||
metrics,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
self.peer_views.len(),
|
||||
self.peer_data.len(),
|
||||
peer_id,
|
||||
peer_protocol_version,
|
||||
view,
|
||||
rng,
|
||||
)
|
||||
@@ -992,7 +1033,7 @@ impl State {
|
||||
// then messages will be sent when we get it.
|
||||
|
||||
let assignments = vec![(assignment, claimed_candidate_index)];
|
||||
let n_peers_total = self.peer_views.len();
|
||||
let n_peers_total = self.peer_data.len();
|
||||
let source_peer = source.peer_id();
|
||||
|
||||
let mut peer_filter = move |peer| {
|
||||
@@ -1019,31 +1060,53 @@ impl State {
|
||||
route_random
|
||||
};
|
||||
|
||||
let peers = entry.known_by.keys().filter(|p| peer_filter(p)).cloned().collect::<Vec<_>>();
|
||||
let (v1_peers, vstaging_peers) = {
|
||||
let peer_data = &self.peer_data;
|
||||
let peers = entry
|
||||
.known_by
|
||||
.keys()
|
||||
.filter_map(|p| peer_data.get_key_value(p))
|
||||
.filter(|(p, _)| peer_filter(p))
|
||||
.map(|(p, peer_data)| (*p, peer_data.version))
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for peer in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(peer_knowledge) = entry.known_by.get_mut(peer) {
|
||||
peer_knowledge.sent.insert(message_subject.clone(), message_kind);
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for (peer, _) in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(peer_knowledge) = entry.known_by.get_mut(peer) {
|
||||
peer_knowledge.sent.insert(message_subject.clone(), message_kind);
|
||||
}
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?claimed_candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an assignment to peers",
|
||||
);
|
||||
}
|
||||
|
||||
let v1_peers = filter_peers_by_version(&peers, ValidationVersion::V1);
|
||||
let vstaging_peers = filter_peers_by_version(&peers, ValidationVersion::VStaging);
|
||||
|
||||
(v1_peers, vstaging_peers)
|
||||
};
|
||||
|
||||
if !v1_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
v1_peers,
|
||||
versioned_assignments_packet(ValidationVersion::V1, assignments.clone()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?claimed_candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an assignment to peers",
|
||||
);
|
||||
|
||||
if !vstaging_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments),
|
||||
)),
|
||||
vstaging_peers,
|
||||
versioned_assignments_packet(ValidationVersion::VStaging, assignments.clone()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
@@ -1332,38 +1395,55 @@ impl State {
|
||||
in_topology || knowledge.sent.contains(message_subject, MessageKind::Assignment)
|
||||
};
|
||||
|
||||
let peers = entry
|
||||
.known_by
|
||||
.iter()
|
||||
.filter(|(p, k)| peer_filter(p, k))
|
||||
.map(|(p, _)| p)
|
||||
.cloned()
|
||||
.collect::<Vec<_>>();
|
||||
let (v1_peers, vstaging_peers) = {
|
||||
let peer_data = &self.peer_data;
|
||||
let peers = entry
|
||||
.known_by
|
||||
.iter()
|
||||
.filter_map(|(p, k)| peer_data.get(&p).map(|pd| (p, k, pd.version)))
|
||||
.filter(|(p, k, _)| peer_filter(p, k))
|
||||
.map(|(p, _, v)| (*p, v))
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for peer in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(entry) = entry.known_by.get_mut(peer) {
|
||||
entry.sent.insert(message_subject.clone(), message_kind);
|
||||
// Add the metadata of the assignment to the knowledge of each peer.
|
||||
for (peer, _) in peers.iter() {
|
||||
// we already filtered peers above, so this should always be Some
|
||||
if let Some(peer_knowledge) = entry.known_by.get_mut(peer) {
|
||||
peer_knowledge.sent.insert(message_subject.clone(), message_kind);
|
||||
}
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an approval to peers",
|
||||
);
|
||||
}
|
||||
|
||||
let v1_peers = filter_peers_by_version(&peers, ValidationVersion::V1);
|
||||
let vstaging_peers = filter_peers_by_version(&peers, ValidationVersion::VStaging);
|
||||
|
||||
(v1_peers, vstaging_peers)
|
||||
};
|
||||
|
||||
let approvals = vec![vote];
|
||||
|
||||
if !v1_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
v1_peers,
|
||||
versioned_approvals_packet(ValidationVersion::V1, approvals.clone()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
if !peers.is_empty() {
|
||||
let approvals = vec![vote];
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
?block_hash,
|
||||
?candidate_index,
|
||||
local = source.peer_id().is_none(),
|
||||
num_peers = peers.len(),
|
||||
"Sending an approval to peers",
|
||||
);
|
||||
|
||||
if !vstaging_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals),
|
||||
)),
|
||||
vstaging_peers,
|
||||
versioned_approvals_packet(ValidationVersion::VStaging, approvals),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
@@ -1427,6 +1507,7 @@ impl State {
|
||||
topologies: &SessionGridTopologies,
|
||||
total_peers: usize,
|
||||
peer_id: PeerId,
|
||||
peer_protocol_version: ValidationVersion,
|
||||
view: View,
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
@@ -1536,7 +1617,8 @@ impl State {
|
||||
"Sending assignments to unified peer",
|
||||
);
|
||||
|
||||
send_assignments_batched(sender, assignments_to_send, peer_id).await;
|
||||
send_assignments_batched(sender, assignments_to_send, peer_id, peer_protocol_version)
|
||||
.await;
|
||||
}
|
||||
|
||||
if !approvals_to_send.is_empty() {
|
||||
@@ -1547,7 +1629,7 @@ impl State {
|
||||
"Sending approvals to unified peer",
|
||||
);
|
||||
|
||||
send_approvals_batched(sender, approvals_to_send, peer_id).await;
|
||||
send_approvals_batched(sender, approvals_to_send, peer_id, peer_protocol_version).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1583,6 +1665,7 @@ impl State {
|
||||
|
||||
adjust_required_routing_and_propagate(
|
||||
ctx,
|
||||
&self.peer_data,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
|block_entry| {
|
||||
@@ -1610,6 +1693,7 @@ impl State {
|
||||
|
||||
adjust_required_routing_and_propagate(
|
||||
ctx,
|
||||
&self.peer_data,
|
||||
&mut self.blocks,
|
||||
&self.topologies,
|
||||
|block_entry| {
|
||||
@@ -1669,6 +1753,7 @@ impl State {
|
||||
#[overseer::contextbounds(ApprovalDistribution, prefix = self::overseer)]
|
||||
async fn adjust_required_routing_and_propagate<Context, BlockFilter, RoutingModifier>(
|
||||
ctx: &mut Context,
|
||||
peer_data: &HashMap<PeerId, PeerData>,
|
||||
blocks: &mut HashMap<Hash, BlockEntry>,
|
||||
topologies: &SessionGridTopologies,
|
||||
block_filter: BlockFilter,
|
||||
@@ -1758,11 +1843,22 @@ async fn adjust_required_routing_and_propagate<Context, BlockFilter, RoutingModi
|
||||
// Send messages in accumulated packets, assignments preceding approvals.
|
||||
|
||||
for (peer, assignments_packet) in peer_assignments {
|
||||
send_assignments_batched(ctx.sender(), assignments_packet, peer).await;
|
||||
let peer_protocol_version = match peer_data.get(&peer).map(|pd| pd.version) {
|
||||
None => continue,
|
||||
Some(v) => v,
|
||||
};
|
||||
|
||||
send_assignments_batched(ctx.sender(), assignments_packet, peer, peer_protocol_version)
|
||||
.await;
|
||||
}
|
||||
|
||||
for (peer, approvals_packet) in peer_approvals {
|
||||
send_approvals_batched(ctx.sender(), approvals_packet, peer).await;
|
||||
let peer_protocol_version = match peer_data.get(&peer).map(|pd| pd.version) {
|
||||
None => continue,
|
||||
Some(v) => v,
|
||||
};
|
||||
|
||||
send_approvals_batched(ctx.sender(), approvals_packet, peer, peer_protocol_version).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1912,6 +2008,49 @@ impl ApprovalDistribution {
|
||||
}
|
||||
}
|
||||
|
||||
fn versioned_approvals_packet(
|
||||
version: ValidationVersion,
|
||||
approvals: Vec<IndirectSignedApprovalVote>,
|
||||
) -> VersionedValidationProtocol {
|
||||
match version {
|
||||
ValidationVersion::V1 =>
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals),
|
||||
)),
|
||||
ValidationVersion::VStaging =>
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(approvals),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn versioned_assignments_packet(
|
||||
version: ValidationVersion,
|
||||
assignments: Vec<(IndirectAssignmentCert, CandidateIndex)>,
|
||||
) -> VersionedValidationProtocol {
|
||||
match version {
|
||||
ValidationVersion::V1 =>
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments),
|
||||
)),
|
||||
ValidationVersion::VStaging =>
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Assignments(assignments),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn filter_peers_by_version(
|
||||
peers: &[(PeerId, ValidationVersion)],
|
||||
version: ValidationVersion,
|
||||
) -> Vec<PeerId> {
|
||||
peers
|
||||
.iter()
|
||||
.filter(|(_, v)| v == &version)
|
||||
.map(|(peer_id, _)| *peer_id)
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[overseer::subsystem(ApprovalDistribution, error=SubsystemError, prefix=self::overseer)]
|
||||
impl<Context> ApprovalDistribution {
|
||||
fn start(self, ctx: Context) -> SpawnedSubsystem {
|
||||
@@ -1954,19 +2093,16 @@ pub(crate) async fn send_assignments_batched(
|
||||
sender: &mut impl overseer::ApprovalDistributionSenderTrait,
|
||||
assignments: Vec<(IndirectAssignmentCert, CandidateIndex)>,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
) {
|
||||
let mut batches = assignments.into_iter().peekable();
|
||||
|
||||
while batches.peek().is_some() {
|
||||
let batch: Vec<_> = batches.by_ref().take(MAX_ASSIGNMENT_BATCH_SIZE).collect();
|
||||
let versioned = versioned_assignments_packet(protocol_version, batch);
|
||||
|
||||
sender
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vec![peer],
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(batch),
|
||||
)),
|
||||
))
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(vec![peer], versioned))
|
||||
.await;
|
||||
}
|
||||
}
|
||||
@@ -1976,19 +2112,16 @@ pub(crate) async fn send_approvals_batched(
|
||||
sender: &mut impl overseer::ApprovalDistributionSenderTrait,
|
||||
approvals: Vec<IndirectSignedApprovalVote>,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
) {
|
||||
let mut batches = approvals.into_iter().peekable();
|
||||
|
||||
while batches.peek().is_some() {
|
||||
let batch: Vec<_> = batches.by_ref().take(MAX_APPROVAL_BATCH_SIZE).collect();
|
||||
let versioned = versioned_approvals_packet(protocol_version, batch);
|
||||
|
||||
sender
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vec![peer],
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(batch),
|
||||
)),
|
||||
))
|
||||
.send_message(NetworkBridgeTxMessage::SendValidationMessage(vec![peer], versioned))
|
||||
.await;
|
||||
}
|
||||
}
|
||||
@@ -219,6 +219,7 @@ async fn setup_gossip_topology(
|
||||
async fn setup_peer_with_view(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer_id: &PeerId,
|
||||
validation_version: ValidationVersion,
|
||||
view: View,
|
||||
) {
|
||||
overseer_send(
|
||||
@@ -226,7 +227,7 @@ async fn setup_peer_with_view(
|
||||
ApprovalDistributionMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerConnected(
|
||||
*peer_id,
|
||||
ObservedRole::Full,
|
||||
ValidationVersion::V1.into(),
|
||||
validation_version.into(),
|
||||
None,
|
||||
)),
|
||||
)
|
||||
@@ -243,13 +244,12 @@ async fn setup_peer_with_view(
|
||||
async fn send_message_from_peer(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer_id: &PeerId,
|
||||
msg: protocol_v1::ApprovalDistributionMessage,
|
||||
msg: net_protocol::ApprovalDistributionMessage,
|
||||
) {
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
ApprovalDistributionMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
*peer_id,
|
||||
Versioned::V1(msg),
|
||||
*peer_id, msg,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
@@ -331,9 +331,9 @@ fn try_import_the_same_assignment() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup peers
|
||||
setup_peer_with_view(overseer, &peer_a, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::V1, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -353,7 +353,7 @@ fn try_import_the_same_assignment() {
|
||||
let assignments = vec![(cert.clone(), 0u32)];
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, &peer_a, msg).await;
|
||||
send_message_from_peer(overseer, &peer_a, Versioned::V1(msg)).await;
|
||||
|
||||
expect_reputation_change(overseer, &peer_a, COST_UNEXPECTED_MESSAGE).await;
|
||||
|
||||
@@ -386,11 +386,11 @@ fn try_import_the_same_assignment() {
|
||||
);
|
||||
|
||||
// setup new peer
|
||||
setup_peer_with_view(overseer, &peer_d, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_d, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// send the same assignment from peer_d
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments);
|
||||
send_message_from_peer(overseer, &peer_d, msg).await;
|
||||
send_message_from_peer(overseer, &peer_d, Versioned::V1(msg)).await;
|
||||
|
||||
expect_reputation_change(overseer, &peer_d, COST_UNEXPECTED_MESSAGE).await;
|
||||
expect_reputation_change(overseer, &peer_d, BENEFIT_VALID_MESSAGE).await;
|
||||
@@ -413,7 +413,7 @@ fn delay_reputation_change() {
|
||||
let overseer = &mut virtual_overseer;
|
||||
|
||||
// Setup peers
|
||||
setup_peer_with_view(overseer, &peer, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -433,7 +433,7 @@ fn delay_reputation_change() {
|
||||
let assignments = vec![(cert.clone(), 0u32)];
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, &peer, msg).await;
|
||||
send_message_from_peer(overseer, &peer, Versioned::V1(msg)).await;
|
||||
|
||||
// send an `Accept` message from the Approval Voting subsystem
|
||||
assert_matches!(
|
||||
@@ -474,7 +474,7 @@ fn spam_attack_results_in_negative_reputation_change() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
let peer = &peer_a;
|
||||
setup_peer_with_view(overseer, peer, view![]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// new block `hash_b` with 20 candidates
|
||||
let candidates_count = 20;
|
||||
@@ -501,7 +501,7 @@ fn spam_attack_results_in_negative_reputation_change() {
|
||||
.collect();
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
for i in 0..candidates_count {
|
||||
expect_reputation_change(overseer, peer, COST_UNEXPECTED_MESSAGE).await;
|
||||
@@ -533,7 +533,7 @@ fn spam_attack_results_in_negative_reputation_change() {
|
||||
.await;
|
||||
|
||||
// send the assignments again
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
// each of them will incur `COST_UNEXPECTED_MESSAGE`, not only the first one
|
||||
for _ in 0..candidates_count {
|
||||
@@ -558,7 +558,7 @@ fn peer_sending_us_the_same_we_just_sent_them_is_ok() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
let peer = &peer_a;
|
||||
setup_peer_with_view(overseer, peer, view![]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![]).await;
|
||||
|
||||
// new block `hash` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -610,12 +610,12 @@ fn peer_sending_us_the_same_we_just_sent_them_is_ok() {
|
||||
// the peer could send us it as well
|
||||
let assignments = vec![(cert, candidate_index)];
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments);
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
assert!(overseer.recv().timeout(TIMEOUT).await.is_none(), "we should not punish the peer");
|
||||
|
||||
// send the assignments again
|
||||
send_message_from_peer(overseer, peer, msg).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg)).await;
|
||||
|
||||
// now we should
|
||||
expect_reputation_change(overseer, peer, COST_DUPLICATE_MESSAGE).await;
|
||||
@@ -634,9 +634,9 @@ fn import_approval_happy_path() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup peers
|
||||
setup_peer_with_view(overseer, &peer_a, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::V1, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -681,7 +681,7 @@ fn import_approval_happy_path() {
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -722,8 +722,8 @@ fn import_approval_bad() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup peers
|
||||
setup_peer_with_view(overseer, &peer_a, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::V1, view![]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -749,14 +749,14 @@ fn import_approval_bad() {
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
expect_reputation_change(overseer, &peer_b, COST_UNEXPECTED_MESSAGE).await;
|
||||
|
||||
// now import an assignment from peer_b
|
||||
let assignments = vec![(cert.clone(), candidate_index)];
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -775,7 +775,7 @@ fn import_approval_bad() {
|
||||
|
||||
// and try again
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_b, msg).await;
|
||||
send_message_from_peer(overseer, &peer_b, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -916,7 +916,7 @@ fn update_peer_view() {
|
||||
overseer_send(overseer, ApprovalDistributionMessage::DistributeAssignment(cert_b, 0)).await;
|
||||
|
||||
// connect a peer
|
||||
setup_peer_with_view(overseer, peer, view![hash_a]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash_a]).await;
|
||||
|
||||
// we should send relevant assignments to the peer
|
||||
assert_matches!(
|
||||
@@ -934,7 +934,7 @@ fn update_peer_view() {
|
||||
virtual_overseer
|
||||
});
|
||||
|
||||
assert_eq!(state.peer_views.get(peer).map(|v| v.finalized_number), Some(0));
|
||||
assert_eq!(state.peer_data.get(peer).map(|data| data.view.finalized_number), Some(0));
|
||||
assert_eq!(
|
||||
state
|
||||
.blocks
|
||||
@@ -986,7 +986,7 @@ fn update_peer_view() {
|
||||
virtual_overseer
|
||||
});
|
||||
|
||||
assert_eq!(state.peer_views.get(peer).map(|v| v.finalized_number), Some(2));
|
||||
assert_eq!(state.peer_data.get(peer).map(|data| data.view.finalized_number), Some(2));
|
||||
assert_eq!(
|
||||
state
|
||||
.blocks
|
||||
@@ -1016,7 +1016,10 @@ fn update_peer_view() {
|
||||
virtual_overseer
|
||||
});
|
||||
|
||||
assert_eq!(state.peer_views.get(peer).map(|v| v.finalized_number), Some(finalized_number));
|
||||
assert_eq!(
|
||||
state.peer_data.get(peer).map(|data| data.view.finalized_number),
|
||||
Some(finalized_number)
|
||||
);
|
||||
assert!(state.blocks.get(&hash_c).unwrap().known_by.get(peer).is_none());
|
||||
}
|
||||
|
||||
@@ -1031,7 +1034,7 @@ fn import_remotely_then_locally() {
|
||||
let _ = test_harness(state_without_reputation_delay(), |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// setup the peer
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
@@ -1051,7 +1054,7 @@ fn import_remotely_then_locally() {
|
||||
let cert = fake_assignment_cert(hash, validator_index);
|
||||
let assignments = vec![(cert.clone(), candidate_index)];
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, peer, msg).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg)).await;
|
||||
|
||||
// send an `Accept` message from the Approval Voting subsystem
|
||||
assert_matches!(
|
||||
@@ -1086,7 +1089,7 @@ fn import_remotely_then_locally() {
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, peer, msg).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -1152,7 +1155,7 @@ fn sends_assignments_even_when_state_is_approved() {
|
||||
.await;
|
||||
|
||||
// connect the peer.
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
let assignments = vec![(cert.clone(), candidate_index)];
|
||||
let approvals = vec![approval.clone()];
|
||||
@@ -1216,7 +1219,7 @@ fn race_condition_in_local_vs_remote_view_update() {
|
||||
};
|
||||
|
||||
// This will send a peer view that is ahead of our view
|
||||
setup_peer_with_view(overseer, peer, view![hash_b]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash_b]).await;
|
||||
|
||||
// Send our view update to include a new head
|
||||
overseer_send(
|
||||
@@ -1237,7 +1240,7 @@ fn race_condition_in_local_vs_remote_view_update() {
|
||||
.collect();
|
||||
|
||||
let msg = protocol_v1::ApprovalDistributionMessage::Assignments(assignments.clone());
|
||||
send_message_from_peer(overseer, peer, msg.clone()).await;
|
||||
send_message_from_peer(overseer, peer, Versioned::V1(msg.clone())).await;
|
||||
|
||||
// This will handle pending messages being processed
|
||||
let msg = ApprovalDistributionMessage::NewBlocks(vec![meta]);
|
||||
@@ -1280,7 +1283,7 @@ fn propagates_locally_generated_assignment_to_both_dimensions() {
|
||||
|
||||
// Connect all peers.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// Set up a gossip topology.
|
||||
@@ -1385,7 +1388,7 @@ fn propagates_assignments_along_unshared_dimension() {
|
||||
|
||||
// Connect all peers.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// Set up a gossip topology.
|
||||
@@ -1421,7 +1424,7 @@ fn propagates_assignments_along_unshared_dimension() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -1470,7 +1473,7 @@ fn propagates_assignments_along_unshared_dimension() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -1527,7 +1530,7 @@ fn propagates_to_required_after_connect() {
|
||||
// Connect all peers except omitted.
|
||||
for (i, (peer, _)) in peers.iter().enumerate() {
|
||||
if !omitted.contains(&i) {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1616,7 +1619,7 @@ fn propagates_to_required_after_connect() {
|
||||
);
|
||||
|
||||
for i in omitted.iter().copied() {
|
||||
setup_peer_with_view(overseer, &peers[i].0, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peers[i].0, ValidationVersion::V1, view![hash]).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
@@ -1665,7 +1668,7 @@ fn sends_to_more_peers_after_getting_topology() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -1817,7 +1820,7 @@ fn originator_aggression_l1() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -1976,7 +1979,7 @@ fn non_originator_aggression_l1() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -2010,7 +2013,7 @@ fn non_originator_aggression_l1() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -2081,7 +2084,7 @@ fn non_originator_aggression_l2() {
|
||||
|
||||
// Connect all peers except omitted.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
@@ -2115,7 +2118,7 @@ fn non_originator_aggression_l2() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -2246,7 +2249,7 @@ fn resends_messages_periodically() {
|
||||
|
||||
// Connect all peers.
|
||||
for (peer, _) in &peers {
|
||||
setup_peer_with_view(overseer, peer, view![hash]).await;
|
||||
setup_peer_with_view(overseer, peer, ValidationVersion::V1, view![hash]).await;
|
||||
}
|
||||
|
||||
// Set up a gossip topology.
|
||||
@@ -2281,7 +2284,7 @@ fn resends_messages_periodically() {
|
||||
|
||||
// Issuer of the message is important, not the peer we receive from.
|
||||
// 99 deliberately chosen because it's not in X or Y.
|
||||
send_message_from_peer(overseer, &peers[99].0, msg).await;
|
||||
send_message_from_peer(overseer, &peers[99].0, Versioned::V1(msg)).await;
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
|
||||
@@ -2372,6 +2375,126 @@ fn resends_messages_periodically() {
|
||||
});
|
||||
}
|
||||
|
||||
/// Tests that peers correctly receive versioned messages.
|
||||
#[test]
|
||||
fn import_versioned_approval() {
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
let parent_hash = Hash::repeat_byte(0xFF);
|
||||
let hash = Hash::repeat_byte(0xAA);
|
||||
|
||||
let state = state_without_reputation_delay();
|
||||
let _ = test_harness(state, |mut virtual_overseer| async move {
|
||||
let overseer = &mut virtual_overseer;
|
||||
// All peers are aware of relay parent.
|
||||
setup_peer_with_view(overseer, &peer_a, ValidationVersion::VStaging, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_b, ValidationVersion::V1, view![hash]).await;
|
||||
setup_peer_with_view(overseer, &peer_c, ValidationVersion::VStaging, view![hash]).await;
|
||||
|
||||
// new block `hash_a` with 1 candidates
|
||||
let meta = BlockApprovalMeta {
|
||||
hash,
|
||||
parent_hash,
|
||||
number: 1,
|
||||
candidates: vec![Default::default(); 1],
|
||||
slot: 1.into(),
|
||||
session: 1,
|
||||
};
|
||||
let msg = ApprovalDistributionMessage::NewBlocks(vec![meta]);
|
||||
overseer_send(overseer, msg).await;
|
||||
|
||||
// import an assignment related to `hash` locally
|
||||
let validator_index = ValidatorIndex(0);
|
||||
let candidate_index = 0u32;
|
||||
let cert = fake_assignment_cert(hash, validator_index);
|
||||
overseer_send(
|
||||
overseer,
|
||||
ApprovalDistributionMessage::DistributeAssignment(cert, candidate_index),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Assignments(assignments)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_b]);
|
||||
assert_eq!(assignments.len(), 1);
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Assignments(assignments)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers.len(), 2);
|
||||
assert!(peers.contains(&peer_a));
|
||||
assert!(peers.contains(&peer_c));
|
||||
|
||||
assert_eq!(assignments.len(), 1);
|
||||
}
|
||||
);
|
||||
|
||||
// send the an approval from peer_a
|
||||
let approval = IndirectSignedApprovalVote {
|
||||
block_hash: hash,
|
||||
candidate_index,
|
||||
validator: validator_index,
|
||||
signature: dummy_signature(),
|
||||
};
|
||||
let msg = protocol_vstaging::ApprovalDistributionMessage::Approvals(vec![approval.clone()]);
|
||||
send_message_from_peer(overseer, &peer_a, Versioned::VStaging(msg)).await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportApproval(
|
||||
vote,
|
||||
tx,
|
||||
)) => {
|
||||
assert_eq!(vote, approval);
|
||||
tx.send(ApprovalCheckResult::Accepted).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
expect_reputation_change(overseer, &peer_a, BENEFIT_VALID_MESSAGE_FIRST).await;
|
||||
|
||||
// Peers b and c receive versioned approval messages.
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::V1(protocol_v1::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_v1::ApprovalDistributionMessage::Approvals(approvals)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_b]);
|
||||
assert_eq!(approvals.len(), 1);
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
overseer_recv(overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
peers,
|
||||
Versioned::VStaging(protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(approvals)
|
||||
))
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_c]);
|
||||
assert_eq!(approvals.len(), 1);
|
||||
}
|
||||
);
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
fn batch_test_round(message_count: usize) {
|
||||
use polkadot_node_subsystem::SubsystemContext;
|
||||
let pool = sp_core::testing::TaskExecutor::new();
|
||||
@@ -2402,8 +2525,9 @@ fn batch_test_round(message_count: usize) {
|
||||
.collect();
|
||||
|
||||
let peer = PeerId::random();
|
||||
send_assignments_batched(&mut sender, assignments.clone(), peer).await;
|
||||
send_approvals_batched(&mut sender, approvals.clone(), peer).await;
|
||||
send_assignments_batched(&mut sender, assignments.clone(), peer, ValidationVersion::V1)
|
||||
.await;
|
||||
send_approvals_batched(&mut sender, approvals.clone(), peer, ValidationVersion::V1).await;
|
||||
|
||||
// Check expected assignments batches.
|
||||
for assignment_index in (0..assignments.len()).step_by(super::MAX_ASSIGNMENT_BATCH_SIZE) {
|
||||
|
||||
@@ -6,6 +6,7 @@ edition.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
[dependencies]
|
||||
always-assert = "0.1"
|
||||
futures = "0.3.21"
|
||||
futures-timer = "3.0.2"
|
||||
gum = { package = "tracing-gum", path = "../../gum" }
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
|
||||
#![deny(unused_crate_dependencies)]
|
||||
|
||||
use always_assert::never;
|
||||
use futures::{channel::oneshot, FutureExt};
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
@@ -29,7 +30,9 @@ use polkadot_node_network_protocol::{
|
||||
grid_topology::{
|
||||
GridNeighbors, RandomRouting, RequiredRouting, SessionBoundGridTopologyStorage,
|
||||
},
|
||||
v1 as protocol_v1, OurView, PeerId, UnifiedReputationChange as Rep, Versioned, View,
|
||||
peer_set::{ProtocolVersion, ValidationVersion},
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, OurView, PeerId,
|
||||
UnifiedReputationChange as Rep, Versioned, View,
|
||||
};
|
||||
use polkadot_node_subsystem::{
|
||||
jaeger, messages::*, overseer, ActiveLeavesUpdate, FromOrchestra, OverseerSignal, PerLeafSpan,
|
||||
@@ -76,25 +79,63 @@ struct BitfieldGossipMessage {
|
||||
}
|
||||
|
||||
impl BitfieldGossipMessage {
|
||||
fn into_validation_protocol(self) -> net_protocol::VersionedValidationProtocol {
|
||||
self.into_network_message().into()
|
||||
fn into_validation_protocol(
|
||||
self,
|
||||
recipient_version: ProtocolVersion,
|
||||
) -> net_protocol::VersionedValidationProtocol {
|
||||
self.into_network_message(recipient_version).into()
|
||||
}
|
||||
|
||||
fn into_network_message(self) -> net_protocol::BitfieldDistributionMessage {
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
))
|
||||
fn into_network_message(
|
||||
self,
|
||||
recipient_version: ProtocolVersion,
|
||||
) -> net_protocol::BitfieldDistributionMessage {
|
||||
match ValidationVersion::try_from(recipient_version).ok() {
|
||||
Some(ValidationVersion::V1) =>
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
)),
|
||||
Some(ValidationVersion::VStaging) =>
|
||||
Versioned::VStaging(protocol_vstaging::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
)),
|
||||
None => {
|
||||
never!("Peers should only have supported protocol versions.");
|
||||
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?recipient_version,
|
||||
"Unknown protocol version provided for message recipient"
|
||||
);
|
||||
|
||||
// fall back to v1 to avoid
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
self.relay_parent,
|
||||
self.signed_availability.into(),
|
||||
))
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Data stored on a per-peer basis.
|
||||
#[derive(Debug)]
|
||||
pub struct PeerData {
|
||||
/// The peer's view.
|
||||
view: View,
|
||||
/// The peer's protocol version.
|
||||
version: ProtocolVersion,
|
||||
}
|
||||
|
||||
/// Data used to track information of peers and relay parents the
|
||||
/// overseer ordered us to work on.
|
||||
#[derive(Default, Debug)]
|
||||
struct ProtocolState {
|
||||
/// Track all active peers and their views
|
||||
/// to determine what is relevant to them.
|
||||
peer_views: HashMap<PeerId, View>,
|
||||
peer_data: HashMap<PeerId, PeerData>,
|
||||
|
||||
/// The current and previous gossip topologies
|
||||
topologies: SessionBoundGridTopologyStorage,
|
||||
@@ -357,7 +398,7 @@ async fn handle_bitfield_distribution<Context>(
|
||||
ctx,
|
||||
job_data,
|
||||
topology,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator,
|
||||
msg,
|
||||
required_routing,
|
||||
@@ -376,7 +417,7 @@ async fn relay_message<Context>(
|
||||
ctx: &mut Context,
|
||||
job_data: &mut PerRelayParentData,
|
||||
topology_neighbors: &GridNeighbors,
|
||||
peer_views: &mut HashMap<PeerId, View>,
|
||||
peers: &mut HashMap<PeerId, PeerData>,
|
||||
validator: ValidatorId,
|
||||
message: BitfieldGossipMessage,
|
||||
required_routing: RequiredRouting,
|
||||
@@ -394,16 +435,16 @@ async fn relay_message<Context>(
|
||||
.await;
|
||||
|
||||
drop(_span);
|
||||
let total_peers = peer_views.len();
|
||||
let total_peers = peers.len();
|
||||
let mut random_routing: RandomRouting = Default::default();
|
||||
|
||||
let _span = span.child("interested-peers");
|
||||
// pass on the bitfield distribution to all interested peers
|
||||
let interested_peers = peer_views
|
||||
let interested_peers = peers
|
||||
.iter()
|
||||
.filter_map(|(peer, view)| {
|
||||
.filter_map(|(peer, data)| {
|
||||
// check interest in the peer in this message's relay parent
|
||||
if view.contains(&message.relay_parent) {
|
||||
if data.view.contains(&message.relay_parent) {
|
||||
let message_needed =
|
||||
job_data.message_from_validator_needed_by_peer(&peer, &validator);
|
||||
if message_needed {
|
||||
@@ -418,7 +459,7 @@ async fn relay_message<Context>(
|
||||
};
|
||||
|
||||
if need_routing {
|
||||
Some(*peer)
|
||||
Some((*peer, data.version))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
@@ -429,9 +470,9 @@ async fn relay_message<Context>(
|
||||
None
|
||||
}
|
||||
})
|
||||
.collect::<Vec<PeerId>>();
|
||||
.collect::<Vec<(PeerId, ProtocolVersion)>>();
|
||||
|
||||
interested_peers.iter().for_each(|peer| {
|
||||
interested_peers.iter().for_each(|(peer, _)| {
|
||||
// track the message as sent for this peer
|
||||
job_data
|
||||
.message_sent_to_peer
|
||||
@@ -450,11 +491,35 @@ async fn relay_message<Context>(
|
||||
);
|
||||
} else {
|
||||
let _span = span.child("gossip");
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
interested_peers,
|
||||
message.into_validation_protocol(),
|
||||
))
|
||||
.await;
|
||||
|
||||
let filter_by_version = |peers: &[(PeerId, ProtocolVersion)],
|
||||
version: ValidationVersion| {
|
||||
peers
|
||||
.iter()
|
||||
.filter(|(_, v)| v == &version.into())
|
||||
.map(|(peer_id, _)| *peer_id)
|
||||
.collect::<Vec<_>>()
|
||||
};
|
||||
|
||||
let v1_interested_peers = filter_by_version(&interested_peers, ValidationVersion::V1);
|
||||
let vstaging_interested_peers =
|
||||
filter_by_version(&interested_peers, ValidationVersion::VStaging);
|
||||
|
||||
if !v1_interested_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
v1_interested_peers,
|
||||
message.clone().into_validation_protocol(ValidationVersion::V1.into()),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
if !vstaging_interested_peers.is_empty() {
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vstaging_interested_peers,
|
||||
message.into_validation_protocol(ValidationVersion::VStaging.into()),
|
||||
))
|
||||
.await
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -465,10 +530,20 @@ async fn process_incoming_peer_message<Context>(
|
||||
state: &mut ProtocolState,
|
||||
metrics: &Metrics,
|
||||
origin: PeerId,
|
||||
message: protocol_v1::BitfieldDistributionMessage,
|
||||
message: net_protocol::BitfieldDistributionMessage,
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
let protocol_v1::BitfieldDistributionMessage::Bitfield(relay_parent, bitfield) = message;
|
||||
let (relay_parent, bitfield) = match message {
|
||||
Versioned::V1(protocol_v1::BitfieldDistributionMessage::Bitfield(
|
||||
relay_parent,
|
||||
bitfield,
|
||||
)) => (relay_parent, bitfield),
|
||||
Versioned::VStaging(protocol_vstaging::BitfieldDistributionMessage::Bitfield(
|
||||
relay_parent,
|
||||
bitfield,
|
||||
)) => (relay_parent, bitfield),
|
||||
};
|
||||
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
peer = %origin,
|
||||
@@ -616,7 +691,7 @@ async fn process_incoming_peer_message<Context>(
|
||||
ctx,
|
||||
job_data,
|
||||
topology,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator,
|
||||
message,
|
||||
required_routing,
|
||||
@@ -647,15 +722,18 @@ async fn handle_network_msg<Context>(
|
||||
let _timer = metrics.time_handle_network_msg();
|
||||
|
||||
match bridge_message {
|
||||
NetworkBridgeEvent::PeerConnected(peer, role, _, _) => {
|
||||
NetworkBridgeEvent::PeerConnected(peer, role, version, _) => {
|
||||
gum::trace!(target: LOG_TARGET, ?peer, ?role, "Peer connected");
|
||||
// insert if none already present
|
||||
state.peer_views.entry(peer).or_default();
|
||||
state
|
||||
.peer_data
|
||||
.entry(peer)
|
||||
.or_insert_with(|| PeerData { view: View::default(), version });
|
||||
},
|
||||
NetworkBridgeEvent::PeerDisconnected(peer) => {
|
||||
gum::trace!(target: LOG_TARGET, ?peer, "Peer disconnected");
|
||||
// get rid of superfluous data
|
||||
state.peer_views.remove(&peer);
|
||||
state.peer_data.remove(&peer);
|
||||
},
|
||||
NetworkBridgeEvent::NewGossipTopology(gossip_topology) => {
|
||||
let session_index = gossip_topology.session;
|
||||
@@ -680,12 +758,21 @@ async fn handle_network_msg<Context>(
|
||||
);
|
||||
|
||||
for new_peer in newly_added {
|
||||
// in case we already knew that peer in the past
|
||||
// it might have had an existing view, we use to initialize
|
||||
// and minimize the delta on `PeerViewChange` to be sent
|
||||
if let Some(old_view) = state.peer_views.remove(&new_peer) {
|
||||
handle_peer_view_change(ctx, state, new_peer, old_view, rng).await;
|
||||
}
|
||||
let old_view = match state.peer_data.get_mut(&new_peer) {
|
||||
Some(d) => {
|
||||
// in case we already knew that peer in the past
|
||||
// it might have had an existing view, we use to initialize
|
||||
// and minimize the delta on `PeerViewChange` to be sent
|
||||
std::mem::replace(&mut d.view, Default::default())
|
||||
},
|
||||
None => {
|
||||
// For peers which are currently unknown, we'll send topology-related
|
||||
// messages to them when they connect and send their first view update.
|
||||
continue
|
||||
},
|
||||
};
|
||||
|
||||
handle_peer_view_change(ctx, state, new_peer, old_view, rng).await;
|
||||
}
|
||||
},
|
||||
NetworkBridgeEvent::PeerViewChange(peerid, new_view) => {
|
||||
@@ -696,7 +783,7 @@ async fn handle_network_msg<Context>(
|
||||
gum::trace!(target: LOG_TARGET, ?new_view, "Our view change");
|
||||
handle_our_view_change(state, new_view);
|
||||
},
|
||||
NetworkBridgeEvent::PeerMessage(remote, Versioned::V1(message)) =>
|
||||
NetworkBridgeEvent::PeerMessage(remote, message) =>
|
||||
process_incoming_peer_message(ctx, state, metrics, remote, message, rng).await,
|
||||
NetworkBridgeEvent::UpdatedAuthorityIds { .. } => {
|
||||
// The bitfield-distribution subsystem doesn't deal with `AuthorityDiscoveryId`s.
|
||||
@@ -728,6 +815,9 @@ fn handle_our_view_change(state: &mut ProtocolState, view: OurView) {
|
||||
|
||||
// Send the difference between two views which were not sent
|
||||
// to that particular peer.
|
||||
//
|
||||
// This requires that there is an entry in the `peer_data` field for the
|
||||
// peer.
|
||||
#[overseer::contextbounds(BitfieldDistribution, prefix=self::overseer)]
|
||||
async fn handle_peer_view_change<Context>(
|
||||
ctx: &mut Context,
|
||||
@@ -736,13 +826,20 @@ async fn handle_peer_view_change<Context>(
|
||||
view: View,
|
||||
rng: &mut (impl CryptoRng + Rng),
|
||||
) {
|
||||
let added = state
|
||||
.peer_views
|
||||
.entry(origin)
|
||||
.or_default()
|
||||
.replace_difference(view)
|
||||
.cloned()
|
||||
.collect::<Vec<_>>();
|
||||
let peer_data = match state.peer_data.get_mut(&origin) {
|
||||
None => {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
peer = ?origin,
|
||||
"Attempted to update peer view for unknown peer."
|
||||
);
|
||||
|
||||
return
|
||||
},
|
||||
Some(pd) => pd,
|
||||
};
|
||||
|
||||
let added = peer_data.view.replace_difference(view).cloned().collect::<Vec<_>>();
|
||||
|
||||
let topology = state.topologies.get_current_topology().local_grid_neighbors();
|
||||
let is_gossip_peer = topology.route_to_peer(RequiredRouting::GridXY, &origin);
|
||||
@@ -808,11 +905,14 @@ async fn send_tracked_gossip_message<Context>(
|
||||
"Sending gossip message"
|
||||
);
|
||||
|
||||
let version =
|
||||
if let Some(peer_data) = state.peer_data.get(&dest) { peer_data.version } else { return };
|
||||
|
||||
job_data.message_sent_to_peer.entry(dest).or_default().insert(validator.clone());
|
||||
|
||||
ctx.send_message(NetworkBridgeTxMessage::SendValidationMessage(
|
||||
vec![dest],
|
||||
message.into_validation_protocol(),
|
||||
message.into_validation_protocol(version),
|
||||
))
|
||||
.await;
|
||||
}
|
||||
|
||||
@@ -56,6 +56,10 @@ fn dummy_rng() -> ChaCha12Rng {
|
||||
rand_chacha::ChaCha12Rng::seed_from_u64(12345)
|
||||
}
|
||||
|
||||
fn peer_data_v1(view: View) -> PeerData {
|
||||
PeerData { view, version: ValidationVersion::V1.into() }
|
||||
}
|
||||
|
||||
/// A very limited state, only interested in the relay parent of the
|
||||
/// given message, which must be signed by `validator` and a set of peers
|
||||
/// which are also only interested in that relay parent.
|
||||
@@ -85,7 +89,11 @@ fn prewarmed_state(
|
||||
span: PerLeafSpan::new(Arc::new(jaeger::Span::Disabled), "test"),
|
||||
},
|
||||
},
|
||||
peer_views: peers.iter().cloned().map(|peer| (peer, view!(relay_parent))).collect(),
|
||||
peer_data: peers
|
||||
.iter()
|
||||
.cloned()
|
||||
.map(|peer| (peer, peer_data_v1(view![relay_parent])))
|
||||
.collect(),
|
||||
topologies,
|
||||
view: our_view!(relay_parent),
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -211,7 +219,10 @@ fn receive_invalid_signature() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, invalid_msg.into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
invalid_msg.into_network_message(ValidationVersion::V1.into())
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -222,7 +233,10 @@ fn receive_invalid_signature() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, invalid_msg_2.into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
invalid_msg_2.into_network_message(ValidationVersion::V1.into())
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
// reputation change due to invalid signature
|
||||
@@ -256,7 +270,7 @@ fn receive_invalid_validator_index() {
|
||||
let (mut state, signing_context, keystore, validator) =
|
||||
state_with_view(our_view![hash_a, hash_b], hash_a, ReputationAggregator::new(|_| true));
|
||||
|
||||
state.peer_views.insert(peer_b, view![hash_a]);
|
||||
state.peer_data.insert(peer_b, peer_data_v1(view![hash_a]));
|
||||
|
||||
let payload = AvailabilityBitfield(bitvec![u8, bitvec::order::Lsb0; 1u8; 32]);
|
||||
let signed = Signed::<AvailabilityBitfield>::sign(
|
||||
@@ -281,7 +295,10 @@ fn receive_invalid_validator_index() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.into_network_message(ValidationVersion::V1.into())
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -344,7 +361,10 @@ fn receive_duplicate_messages() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -377,7 +397,10 @@ fn receive_duplicate_messages() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_a, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_a,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -396,7 +419,10 @@ fn receive_duplicate_messages() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -463,7 +489,10 @@ fn delay_reputation_change() {
|
||||
handle
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: BitfieldDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(peer, msg.clone().into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
@@ -486,7 +515,10 @@ fn delay_reputation_change() {
|
||||
handle
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: BitfieldDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(peer, msg.clone().into_network_message()),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
@@ -546,8 +578,8 @@ fn do_not_relay_message_twice() {
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
|
||||
state.peer_views.insert(peer_b, view![hash]);
|
||||
state.peer_views.insert(peer_a, view![hash]);
|
||||
state.peer_data.insert(peer_b, peer_data_v1(view![hash]));
|
||||
state.peer_data.insert(peer_a, peer_data_v1(view![hash]));
|
||||
|
||||
let msg =
|
||||
BitfieldGossipMessage { relay_parent: hash, signed_availability: signed_bitfield.clone() };
|
||||
@@ -564,7 +596,7 @@ fn do_not_relay_message_twice() {
|
||||
&mut ctx,
|
||||
state.per_relay_parent.get_mut(&hash).unwrap(),
|
||||
&gossip_peers,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator.clone(),
|
||||
msg.clone(),
|
||||
RequiredRouting::GridXY,
|
||||
@@ -591,7 +623,7 @@ fn do_not_relay_message_twice() {
|
||||
assert_eq!(2, peers.len());
|
||||
assert!(peers.contains(&peer_a));
|
||||
assert!(peers.contains(&peer_b));
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol());
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
@@ -600,7 +632,7 @@ fn do_not_relay_message_twice() {
|
||||
&mut ctx,
|
||||
state.per_relay_parent.get_mut(&hash).unwrap(),
|
||||
&gossip_peers,
|
||||
&mut state.peer_views,
|
||||
&mut state.peer_data,
|
||||
validator.clone(),
|
||||
msg.clone(),
|
||||
RequiredRouting::GridXY,
|
||||
@@ -687,14 +719,17 @@ fn changing_view() {
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
assert!(state.peer_views.contains_key(&peer_b));
|
||||
assert!(state.peer_data.contains_key(&peer_b));
|
||||
|
||||
// recv a first message from the network
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -729,8 +764,11 @@ fn changing_view() {
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
assert!(state.peer_views.contains_key(&peer_b));
|
||||
assert_eq!(state.peer_views.get(&peer_b).expect("Must contain value for peer B"), &view![]);
|
||||
assert!(state.peer_data.contains_key(&peer_b));
|
||||
assert_eq!(
|
||||
&state.peer_data.get(&peer_b).expect("Must contain value for peer B").view,
|
||||
&view![]
|
||||
);
|
||||
|
||||
// on rx of the same message, since we are not interested,
|
||||
// should give penalty
|
||||
@@ -738,7 +776,10 @@ fn changing_view() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -770,7 +811,10 @@ fn changing_view() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_a, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_a,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -817,8 +861,8 @@ fn do_not_send_message_back_to_origin() {
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
|
||||
state.peer_views.insert(peer_b, view![hash]);
|
||||
state.peer_views.insert(peer_a, view![hash]);
|
||||
state.peer_data.insert(peer_b, peer_data_v1(view![hash]));
|
||||
state.peer_data.insert(peer_a, peer_data_v1(view![hash]));
|
||||
|
||||
let msg =
|
||||
BitfieldGossipMessage { relay_parent: hash, signed_availability: signed_bitfield.clone() };
|
||||
@@ -833,7 +877,10 @@ fn do_not_send_message_back_to_origin() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peer_b, msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_b,
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -855,7 +902,7 @@ fn do_not_send_message_back_to_origin() {
|
||||
) => {
|
||||
assert_eq!(1, peers.len());
|
||||
assert!(peers.contains(&peer_a));
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol());
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
@@ -932,7 +979,7 @@ fn topology_test() {
|
||||
.expect("should be signed");
|
||||
|
||||
peers_x.iter().chain(peers_y.iter()).for_each(|peer| {
|
||||
state.peer_views.insert(*peer, view![hash]);
|
||||
state.peer_data.insert(*peer, peer_data_v1(view![hash]));
|
||||
});
|
||||
|
||||
let msg =
|
||||
@@ -948,7 +995,10 @@ fn topology_test() {
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(peers_x[0], msg.clone().into_network_message(),),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peers_x[0],
|
||||
msg.clone().into_network_message(ValidationVersion::V1.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
@@ -975,7 +1025,7 @@ fn topology_test() {
|
||||
assert!(topology.peers_x.iter().filter(|peer| peers.contains(&peer)).count() == 4);
|
||||
// Must never include originator
|
||||
assert!(!peers.contains(&peers_x[0]));
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol());
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
@@ -1050,3 +1100,127 @@ fn need_message_works() {
|
||||
// also not ok for Bob
|
||||
assert!(!pretend_send(&mut state, peer_b, &validator_set[1]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning() {
|
||||
let hash_a: Hash = [0; 32].into();
|
||||
let hash_b: Hash = [1; 32].into();
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
|
||||
let peers = [
|
||||
(peer_a, ValidationVersion::VStaging),
|
||||
(peer_b, ValidationVersion::V1),
|
||||
(peer_c, ValidationVersion::VStaging),
|
||||
];
|
||||
|
||||
// validator 0 key pair
|
||||
let (mut state, signing_context, keystore, validator) =
|
||||
state_with_view(our_view![hash_a, hash_b], hash_a, ReputationAggregator::new(|_| true));
|
||||
|
||||
let pool = sp_core::testing::TaskExecutor::new();
|
||||
let (mut ctx, mut handle) = make_subsystem_context::<BitfieldDistributionMessage, _>(pool);
|
||||
let mut rng = dummy_rng();
|
||||
|
||||
executor::block_on(async move {
|
||||
// create a signed message by validator 0
|
||||
let payload = AvailabilityBitfield(bitvec![u8, bitvec::order::Lsb0; 1u8; 32]);
|
||||
let signed_bitfield = Signed::<AvailabilityBitfield>::sign(
|
||||
&keystore,
|
||||
payload,
|
||||
&signing_context,
|
||||
ValidatorIndex(0),
|
||||
&validator,
|
||||
)
|
||||
.ok()
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
let msg = BitfieldGossipMessage {
|
||||
relay_parent: hash_a,
|
||||
signed_availability: signed_bitfield.clone(),
|
||||
};
|
||||
|
||||
for (peer, protocol_version) in peers {
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Full,
|
||||
protocol_version.into(),
|
||||
None
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerViewChange(peer, view![hash_a, hash_b]),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
assert!(state.peer_data.contains_key(&peer));
|
||||
}
|
||||
|
||||
launch!(handle_network_msg(
|
||||
&mut ctx,
|
||||
&mut state,
|
||||
&Default::default(),
|
||||
NetworkBridgeEvent::PeerMessage(
|
||||
peer_a,
|
||||
msg.clone().into_network_message(ValidationVersion::VStaging.into()),
|
||||
),
|
||||
&mut rng,
|
||||
));
|
||||
|
||||
// gossip to the overseer
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::Provisioner(ProvisionerMessage::ProvisionableData(
|
||||
_,
|
||||
ProvisionableData::Bitfield(hash, signed)
|
||||
)) => {
|
||||
assert_eq!(hash, hash_a);
|
||||
assert_eq!(signed, signed_bitfield)
|
||||
}
|
||||
);
|
||||
|
||||
// v1 gossip
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendValidationMessage(peers, send_msg),
|
||||
) => {
|
||||
assert_eq!(peers, vec![peer_b]);
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::V1.into()));
|
||||
}
|
||||
);
|
||||
|
||||
// vstaging gossip
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendValidationMessage(peers, send_msg),
|
||||
) => {
|
||||
assert_eq!(peers, vec![peer_c]);
|
||||
assert_eq!(send_msg, msg.clone().into_validation_protocol(ValidationVersion::VStaging.into()));
|
||||
}
|
||||
);
|
||||
|
||||
// reputation change
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer, rep))
|
||||
) => {
|
||||
assert_eq!(peer, peer_a);
|
||||
assert_eq!(rep, BENEFIT_VALID_MESSAGE_FIRST.into())
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
@@ -55,6 +55,9 @@ pub(crate) fn send_message<M>(
|
||||
) where
|
||||
M: Encode + Clone,
|
||||
{
|
||||
if peers.is_empty() {
|
||||
return
|
||||
}
|
||||
let message = {
|
||||
let encoded = message.encode();
|
||||
metrics.on_notification_sent(peer_set, version, encoded.len(), peers.len());
|
||||
@@ -65,8 +68,11 @@ pub(crate) fn send_message<M>(
|
||||
// list. The message payload can be quite large. If the underlying
|
||||
// network used `Bytes` this would not be necessary.
|
||||
let last_peer = peers.pop();
|
||||
// optimization: generate the protocol name once.
|
||||
let protocol_name = protocol_names.get_name(peer_set, version);
|
||||
|
||||
// We always send messages on the "main" name even when a negotiated
|
||||
// fallback is used. The libp2p implementation handles the fallback
|
||||
// under the hood.
|
||||
let protocol_name = protocol_names.get_main_name(peer_set);
|
||||
peers.into_iter().for_each(|peer| {
|
||||
net.write_notification(peer, protocol_name.clone(), message.clone());
|
||||
});
|
||||
|
||||
@@ -33,7 +33,8 @@ use polkadot_node_network_protocol::{
|
||||
CollationVersion, PeerSet, PeerSetProtocolNames, PerPeerSet, ProtocolVersion,
|
||||
ValidationVersion,
|
||||
},
|
||||
v1 as protocol_v1, ObservedRole, OurView, PeerId, UnifiedReputationChange as Rep, View,
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, ObservedRole, OurView, PeerId,
|
||||
UnifiedReputationChange as Rep, View,
|
||||
};
|
||||
|
||||
use polkadot_node_subsystem::{
|
||||
@@ -246,15 +247,32 @@ where
|
||||
)
|
||||
.await;
|
||||
|
||||
send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Validation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::ValidationProtocol>::ViewUpdate(local_view),
|
||||
&metrics,
|
||||
);
|
||||
match ValidationVersion::try_from(version)
|
||||
.expect("try_get_protocol has already checked version is known; qed")
|
||||
{
|
||||
ValidationVersion::V1 => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Validation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::ValidationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
ValidationVersion::VStaging => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Validation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_vstaging::ValidationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
PeerSet::Collation => {
|
||||
dispatch_collation_events_to_all(
|
||||
@@ -271,15 +289,32 @@ where
|
||||
)
|
||||
.await;
|
||||
|
||||
send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Collation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::CollationProtocol>::ViewUpdate(local_view),
|
||||
&metrics,
|
||||
);
|
||||
match CollationVersion::try_from(version)
|
||||
.expect("try_get_protocol has already checked version is known; qed")
|
||||
{
|
||||
CollationVersion::V1 => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Collation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_v1::CollationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
CollationVersion::VStaging => send_message(
|
||||
&mut network_service,
|
||||
vec![peer],
|
||||
PeerSet::Collation,
|
||||
version,
|
||||
&peerset_protocol_names,
|
||||
WireMessage::<protocol_vstaging::CollationProtocol>::ViewUpdate(
|
||||
local_view,
|
||||
),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
}
|
||||
},
|
||||
@@ -417,30 +452,39 @@ where
|
||||
);
|
||||
|
||||
if !v_messages.is_empty() {
|
||||
let (events, reports) =
|
||||
if expected_versions[PeerSet::Validation] ==
|
||||
Some(ValidationVersion::V1.into())
|
||||
{
|
||||
handle_v1_peer_messages::<protocol_v1::ValidationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Validation,
|
||||
&mut shared.0.lock().validation_peers,
|
||||
v_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Validation],
|
||||
"Major logic bug. Peer somehow has unsupported validation protocol version."
|
||||
);
|
||||
let (events, reports) = if expected_versions[PeerSet::Validation] ==
|
||||
Some(ValidationVersion::V1.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_v1::ValidationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Validation,
|
||||
&mut shared.0.lock().validation_peers,
|
||||
v_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else if expected_versions[PeerSet::Validation] ==
|
||||
Some(ValidationVersion::VStaging.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_vstaging::ValidationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Validation,
|
||||
&mut shared.0.lock().validation_peers,
|
||||
v_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Validation],
|
||||
"Major logic bug. Peer somehow has unsupported validation protocol version."
|
||||
);
|
||||
|
||||
never!("Only version 1 is supported; peer set connection checked above; qed");
|
||||
never!("Only versions 1 and 2 are supported; peer set connection checked above; qed");
|
||||
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
|
||||
for report in reports {
|
||||
network_service.report_peer(remote, report.into());
|
||||
@@ -450,30 +494,39 @@ where
|
||||
}
|
||||
|
||||
if !c_messages.is_empty() {
|
||||
let (events, reports) =
|
||||
if expected_versions[PeerSet::Collation] ==
|
||||
Some(CollationVersion::V1.into())
|
||||
{
|
||||
handle_v1_peer_messages::<protocol_v1::CollationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Collation,
|
||||
&mut shared.0.lock().collation_peers,
|
||||
c_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Collation],
|
||||
"Major logic bug. Peer somehow has unsupported collation protocol version."
|
||||
);
|
||||
let (events, reports) = if expected_versions[PeerSet::Collation] ==
|
||||
Some(CollationVersion::V1.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_v1::CollationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Collation,
|
||||
&mut shared.0.lock().collation_peers,
|
||||
c_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else if expected_versions[PeerSet::Collation] ==
|
||||
Some(CollationVersion::VStaging.into())
|
||||
{
|
||||
handle_peer_messages::<protocol_vstaging::CollationProtocol, _>(
|
||||
remote,
|
||||
PeerSet::Collation,
|
||||
&mut shared.0.lock().collation_peers,
|
||||
c_messages,
|
||||
&metrics,
|
||||
)
|
||||
} else {
|
||||
gum::warn!(
|
||||
target: LOG_TARGET,
|
||||
version = ?expected_versions[PeerSet::Collation],
|
||||
"Major logic bug. Peer somehow has unsupported collation protocol version."
|
||||
);
|
||||
|
||||
never!("Only version 1 is supported; peer set connection checked above; qed");
|
||||
never!("Only versions 1 and 2 are supported; peer set connection checked above; qed");
|
||||
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
// If a peer somehow triggers this, we'll disconnect them
|
||||
// eventually.
|
||||
(Vec::new(), vec![UNCONNECTED_PEERSET_COST])
|
||||
};
|
||||
|
||||
for report in reports {
|
||||
network_service.report_peer(remote, report.into());
|
||||
@@ -738,14 +791,34 @@ fn update_our_view<Net, Context>(
|
||||
}
|
||||
|
||||
(
|
||||
shared.validation_peers.keys().cloned().collect::<Vec<_>>(),
|
||||
shared.collation_peers.keys().cloned().collect::<Vec<_>>(),
|
||||
shared
|
||||
.validation_peers
|
||||
.iter()
|
||||
.map(|(peer_id, data)| (*peer_id, data.version))
|
||||
.collect::<Vec<_>>(),
|
||||
shared
|
||||
.collation_peers
|
||||
.iter()
|
||||
.map(|(peer_id, data)| (*peer_id, data.version))
|
||||
.collect::<Vec<_>>(),
|
||||
)
|
||||
};
|
||||
|
||||
let filter_by_version = |peers: &[(PeerId, ProtocolVersion)], version| {
|
||||
peers.iter().filter(|(_, v)| v == &version).map(|(p, _)| *p).collect::<Vec<_>>()
|
||||
};
|
||||
|
||||
let v1_validation_peers = filter_by_version(&validation_peers, ValidationVersion::V1.into());
|
||||
let v1_collation_peers = filter_by_version(&collation_peers, CollationVersion::V1.into());
|
||||
|
||||
let vstaging_validation_peers =
|
||||
filter_by_version(&validation_peers, ValidationVersion::VStaging.into());
|
||||
let vstaging_collation_peers =
|
||||
filter_by_version(&collation_peers, ValidationVersion::VStaging.into());
|
||||
|
||||
send_validation_message_v1(
|
||||
net,
|
||||
validation_peers,
|
||||
v1_validation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view.clone()),
|
||||
metrics,
|
||||
@@ -753,7 +826,23 @@ fn update_our_view<Net, Context>(
|
||||
|
||||
send_collation_message_v1(
|
||||
net,
|
||||
collation_peers,
|
||||
v1_collation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view.clone()),
|
||||
metrics,
|
||||
);
|
||||
|
||||
send_validation_message_vstaging(
|
||||
net,
|
||||
vstaging_validation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view.clone()),
|
||||
metrics,
|
||||
);
|
||||
|
||||
send_collation_message_vstaging(
|
||||
net,
|
||||
vstaging_collation_peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ViewUpdate(new_view),
|
||||
metrics,
|
||||
@@ -777,7 +866,7 @@ fn update_our_view<Net, Context>(
|
||||
|
||||
// Handle messages on a specific v1 peer-set. The peer is expected to be connected on that
|
||||
// peer-set.
|
||||
fn handle_v1_peer_messages<RawMessage: Decode, OutMessage: From<RawMessage>>(
|
||||
fn handle_peer_messages<RawMessage: Decode, OutMessage: From<RawMessage>>(
|
||||
peer: PeerId,
|
||||
peer_set: PeerSet,
|
||||
peers: &mut HashMap<PeerId, PeerData>,
|
||||
@@ -864,6 +953,42 @@ fn send_collation_message_v1(
|
||||
);
|
||||
}
|
||||
|
||||
fn send_validation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::ValidationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Validation,
|
||||
ValidationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
fn send_collation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::CollationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Collation,
|
||||
CollationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
async fn dispatch_validation_event_to_all(
|
||||
event: NetworkBridgeEvent<net_protocol::VersionedValidationProtocol>,
|
||||
ctx: &mut impl overseer::NetworkBridgeRxSenderTrait,
|
||||
|
||||
@@ -25,6 +25,7 @@ use parking_lot::Mutex;
|
||||
use std::{
|
||||
collections::HashSet,
|
||||
sync::atomic::{AtomicBool, Ordering},
|
||||
task::Poll,
|
||||
};
|
||||
|
||||
use sc_network::{Event as NetworkEvent, IfDisconnected, ProtocolName, ReputationChange};
|
||||
@@ -46,7 +47,7 @@ use polkadot_node_subsystem_test_helpers::{
|
||||
SingleItemSink, SingleItemStream, TestSubsystemContextHandle,
|
||||
};
|
||||
use polkadot_node_subsystem_util::metered;
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, Hash};
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, CandidateHash, Hash};
|
||||
|
||||
use sc_network::Multiaddr;
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
@@ -142,8 +143,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn disconnect_peer(&self, who: PeerId, protocol: ProtocolName) {
|
||||
let (peer_set, version) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -152,8 +152,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn write_notification(&self, who: PeerId, protocol: ProtocolName, message: Vec<u8>) {
|
||||
let (peer_set, version) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -195,10 +194,17 @@ impl TestNetworkHandle {
|
||||
v
|
||||
}
|
||||
|
||||
async fn connect_peer(&mut self, peer: PeerId, peer_set: PeerSet, role: ObservedRole) {
|
||||
async fn connect_peer(
|
||||
&mut self,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
peer_set: PeerSet,
|
||||
role: ObservedRole,
|
||||
) {
|
||||
let protocol_version = ProtocolVersion::from(protocol_version);
|
||||
self.send_network_event(NetworkEvent::NotificationStreamOpened {
|
||||
remote: peer,
|
||||
protocol: self.protocol_names.get_main_name(peer_set),
|
||||
protocol: self.protocol_names.get_name(peer_set, protocol_version),
|
||||
negotiated_fallback: None,
|
||||
role: role.into(),
|
||||
received_handshake: vec![],
|
||||
@@ -432,8 +438,12 @@ fn send_our_view_upon_connection() {
|
||||
|
||||
handle.await_mode_switch().await;
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle.connect_peer(peer, PeerSet::Collation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
|
||||
@@ -478,10 +488,10 @@ fn sends_view_updates_to_peers() {
|
||||
handle.await_mode_switch().await;
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
@@ -541,10 +551,10 @@ fn do_not_send_view_update_until_synced() {
|
||||
assert_ne!(peer_a, peer_b);
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
@@ -636,10 +646,10 @@ fn do_not_send_view_update_when_only_finalized_block_changed() {
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 2, 0).await;
|
||||
@@ -696,7 +706,9 @@ fn peer_view_updates_sent_via_overseer() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
|
||||
@@ -746,7 +758,9 @@ fn peer_messages_sent_via_overseer() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
|
||||
@@ -819,8 +833,12 @@ fn peer_disconnect_from_just_one_peerset() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle.connect_peer(peer, PeerSet::Collation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
|
||||
@@ -907,10 +925,10 @@ fn relays_collation_protocol_messages() {
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer_b, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer_b, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
@@ -1011,8 +1029,12 @@ fn different_views_on_different_peer_sets() {
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle.connect_peer(peer, PeerSet::Validation, ObservedRole::Full).await;
|
||||
network_handle.connect_peer(peer, PeerSet::Collation, ObservedRole::Full).await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
network_handle
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 1).await;
|
||||
|
||||
@@ -1097,7 +1119,7 @@ fn sent_views_include_finalized_number_update() {
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
@@ -1140,7 +1162,7 @@ fn view_finalized_number_can_not_go_down() {
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer_a, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer_a, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.await;
|
||||
|
||||
await_peer_connections(&shared, 1, 0).await;
|
||||
@@ -1225,3 +1247,164 @@ fn our_view_updates_decreasing_order_and_limited_to_max() {
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning_view_update() {
|
||||
let (oracle, handle) = make_sync_oracle(false);
|
||||
test_harness(Box::new(oracle), |test_harness| async move {
|
||||
let TestHarness { mut network_handle, mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let peer_ids: Vec<_> = (0..4).map(|_| PeerId::random()).collect();
|
||||
let peers = [
|
||||
(peer_ids[0], PeerSet::Validation, ValidationVersion::VStaging),
|
||||
(peer_ids[1], PeerSet::Collation, ValidationVersion::V1),
|
||||
(peer_ids[2], PeerSet::Validation, ValidationVersion::V1),
|
||||
(peer_ids[3], PeerSet::Collation, ValidationVersion::VStaging),
|
||||
];
|
||||
|
||||
let head = Hash::repeat_byte(1);
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Signal(OverseerSignal::ActiveLeaves(
|
||||
ActiveLeavesUpdate::start_work(ActivatedLeaf {
|
||||
hash: head,
|
||||
number: 1,
|
||||
status: LeafStatus::Fresh,
|
||||
span: Arc::new(jaeger::Span::Disabled),
|
||||
}),
|
||||
)))
|
||||
.await;
|
||||
|
||||
handle.await_mode_switch().await;
|
||||
|
||||
for &(peer_id, peer_set, version) in &peers {
|
||||
network_handle
|
||||
.connect_peer(peer_id, version, peer_set, ObservedRole::Full)
|
||||
.await;
|
||||
}
|
||||
|
||||
let view = view![head];
|
||||
let actions = network_handle.next_network_actions(4).await;
|
||||
|
||||
for &(peer_id, peer_set, version) in &peers {
|
||||
let wire_msg = match version {
|
||||
ValidationVersion::V1 =>
|
||||
WireMessage::<protocol_v1::ValidationProtocol>::ViewUpdate(view.clone())
|
||||
.encode(),
|
||||
ValidationVersion::VStaging =>
|
||||
WireMessage::<protocol_vstaging::ValidationProtocol>::ViewUpdate(view.clone())
|
||||
.encode(),
|
||||
};
|
||||
assert_network_actions_contains(
|
||||
&actions,
|
||||
&NetworkAction::WriteNotification(peer_id, peer_set, wire_msg),
|
||||
);
|
||||
}
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning_subsystem_msg() {
|
||||
let (oracle, _handle) = make_sync_oracle(false);
|
||||
test_harness(Box::new(oracle), |test_harness| async move {
|
||||
let TestHarness { mut network_handle, mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(
|
||||
peer,
|
||||
ValidationVersion::VStaging,
|
||||
PeerSet::Validation,
|
||||
ObservedRole::Full,
|
||||
)
|
||||
.await;
|
||||
|
||||
// bridge will inform about all connected peers.
|
||||
{
|
||||
assert_sends_validation_event_to_all(
|
||||
NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Full,
|
||||
ValidationVersion::VStaging.into(),
|
||||
None,
|
||||
),
|
||||
&mut virtual_overseer,
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_sends_validation_event_to_all(
|
||||
NetworkBridgeEvent::PeerViewChange(peer, View::default()),
|
||||
&mut virtual_overseer,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let approval_distribution_message =
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(Vec::new());
|
||||
|
||||
let msg = protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
approval_distribution_message.clone(),
|
||||
);
|
||||
|
||||
network_handle
|
||||
.peer_message(
|
||||
peer,
|
||||
PeerSet::Validation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ApprovalDistribution(
|
||||
ApprovalDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(p, Versioned::VStaging(m))
|
||||
)
|
||||
) => {
|
||||
assert_eq!(p, peer);
|
||||
assert_eq!(m, approval_distribution_message);
|
||||
}
|
||||
);
|
||||
|
||||
let metadata = protocol_v1::StatementMetadata {
|
||||
relay_parent: Hash::zero(),
|
||||
candidate_hash: CandidateHash::default(),
|
||||
signed_by: ValidatorIndex(0),
|
||||
signature: sp_core::crypto::UncheckedFrom::unchecked_from([1u8; 64]),
|
||||
};
|
||||
let statement_distribution_message =
|
||||
protocol_vstaging::StatementDistributionMessage::V1Compatibility(
|
||||
protocol_v1::StatementDistributionMessage::LargeStatement(metadata),
|
||||
);
|
||||
let msg = protocol_vstaging::ValidationProtocol::StatementDistribution(
|
||||
statement_distribution_message.clone(),
|
||||
);
|
||||
|
||||
network_handle
|
||||
.peer_message(
|
||||
peer,
|
||||
PeerSet::Validation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::StatementDistribution(
|
||||
StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(p, Versioned::VStaging(m))
|
||||
)
|
||||
) => {
|
||||
assert_eq!(p, peer);
|
||||
assert_eq!(m, statement_distribution_message);
|
||||
}
|
||||
);
|
||||
|
||||
// No more messages.
|
||||
assert_matches!(futures::poll!(virtual_overseer.recv().boxed()), Poll::Pending);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
@@ -20,7 +20,7 @@ use super::*;
|
||||
use polkadot_node_network_protocol::{
|
||||
peer_set::{CollationVersion, PeerSet, PeerSetProtocolNames, ValidationVersion},
|
||||
request_response::ReqProtocolNames,
|
||||
v1 as protocol_v1, PeerId, Versioned,
|
||||
v1 as protocol_v1, vstaging as protocol_vstaging, PeerId, Versioned,
|
||||
};
|
||||
|
||||
use polkadot_node_subsystem::{
|
||||
@@ -197,6 +197,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_validation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
NetworkBridgeTxMessage::SendValidationMessages(msgs) => {
|
||||
@@ -215,6 +222,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_validation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -233,6 +247,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_collation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
},
|
||||
NetworkBridgeTxMessage::SendCollationMessages(msgs) => {
|
||||
@@ -251,6 +272,13 @@ where
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
Versioned::VStaging(msg) => send_collation_message_vstaging(
|
||||
&mut network_service,
|
||||
peers,
|
||||
peerset_protocol_names,
|
||||
WireMessage::ProtocolMessage(msg),
|
||||
&metrics,
|
||||
),
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -381,3 +409,39 @@ fn send_collation_message_v1(
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
fn send_validation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::ValidationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Validation,
|
||||
ValidationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
|
||||
fn send_collation_message_vstaging(
|
||||
net: &mut impl Network,
|
||||
peers: Vec<PeerId>,
|
||||
protocol_names: &PeerSetProtocolNames,
|
||||
message: WireMessage<protocol_vstaging::CollationProtocol>,
|
||||
metrics: &Metrics,
|
||||
) {
|
||||
send_message(
|
||||
net,
|
||||
peers,
|
||||
PeerSet::Collation,
|
||||
CollationVersion::VStaging.into(),
|
||||
protocol_names,
|
||||
message,
|
||||
metrics,
|
||||
);
|
||||
}
|
||||
@@ -130,8 +130,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn disconnect_peer(&self, who: PeerId, protocol: ProtocolName) {
|
||||
let (peer_set, version) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -140,8 +139,7 @@ impl Network for TestNetwork {
|
||||
}
|
||||
|
||||
fn write_notification(&self, who: PeerId, protocol: ProtocolName, message: Vec<u8>) {
|
||||
let (peer_set, version) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
assert_eq!(version, peer_set.get_main_version());
|
||||
let (peer_set, _) = self.peerset_protocol_names.try_get_protocol(&protocol).unwrap();
|
||||
|
||||
self.action_tx
|
||||
.lock()
|
||||
@@ -173,10 +171,17 @@ impl TestNetworkHandle {
|
||||
self.action_rx.next().await.expect("subsystem concluded early")
|
||||
}
|
||||
|
||||
async fn connect_peer(&mut self, peer: PeerId, peer_set: PeerSet, role: ObservedRole) {
|
||||
async fn connect_peer(
|
||||
&mut self,
|
||||
peer: PeerId,
|
||||
protocol_version: ValidationVersion,
|
||||
peer_set: PeerSet,
|
||||
role: ObservedRole,
|
||||
) {
|
||||
let protocol_version = ProtocolVersion::from(protocol_version);
|
||||
self.send_network_event(NetworkEvent::NotificationStreamOpened {
|
||||
remote: peer,
|
||||
protocol: self.peerset_protocol_names.get_main_name(peer_set),
|
||||
protocol: self.peerset_protocol_names.get_name(peer_set, protocol_version),
|
||||
negotiated_fallback: None,
|
||||
role: role.into(),
|
||||
received_handshake: vec![],
|
||||
@@ -242,7 +247,7 @@ fn send_messages_to_peers() {
|
||||
let peer = PeerId::random();
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer, PeerSet::Validation, ObservedRole::Full)
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Validation, ObservedRole::Full)
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
@@ -251,7 +256,7 @@ fn send_messages_to_peers() {
|
||||
// so the single item sink has to be free explicitly
|
||||
|
||||
network_handle
|
||||
.connect_peer(peer, PeerSet::Collation, ObservedRole::Full)
|
||||
.connect_peer(peer, ValidationVersion::V1, PeerSet::Collation, ObservedRole::Full)
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
@@ -328,3 +333,107 @@ fn send_messages_to_peers() {
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn network_protocol_versioning_send() {
|
||||
test_harness(|test_harness| async move {
|
||||
let TestHarness { mut network_handle, mut virtual_overseer } = test_harness;
|
||||
|
||||
let peer_ids: Vec<_> = (0..4).map(|_| PeerId::random()).collect();
|
||||
let peers = [
|
||||
(peer_ids[0], PeerSet::Validation, ValidationVersion::VStaging),
|
||||
(peer_ids[1], PeerSet::Collation, ValidationVersion::V1),
|
||||
(peer_ids[2], PeerSet::Validation, ValidationVersion::V1),
|
||||
(peer_ids[3], PeerSet::Collation, ValidationVersion::VStaging),
|
||||
];
|
||||
|
||||
for &(peer_id, peer_set, version) in &peers {
|
||||
network_handle
|
||||
.connect_peer(peer_id, version, peer_set, ObservedRole::Full)
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
}
|
||||
|
||||
// send a validation protocol message.
|
||||
|
||||
{
|
||||
let approval_distribution_message =
|
||||
protocol_vstaging::ApprovalDistributionMessage::Approvals(Vec::new());
|
||||
|
||||
let msg = protocol_vstaging::ValidationProtocol::ApprovalDistribution(
|
||||
approval_distribution_message.clone(),
|
||||
);
|
||||
|
||||
// Note that bridge doesn't ensure neither peer's protocol version
|
||||
// or peer set match the message.
|
||||
let receivers = vec![peer_ids[0], peer_ids[3]];
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: NetworkBridgeTxMessage::SendValidationMessage(
|
||||
receivers.clone(),
|
||||
Versioned::VStaging(msg.clone()),
|
||||
),
|
||||
})
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur");
|
||||
|
||||
for peer in &receivers {
|
||||
assert_eq!(
|
||||
network_handle
|
||||
.next_network_action()
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur"),
|
||||
NetworkAction::WriteNotification(
|
||||
*peer,
|
||||
PeerSet::Validation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// send a collation protocol message.
|
||||
|
||||
{
|
||||
let collator_protocol_message = protocol_vstaging::CollatorProtocolMessage::Declare(
|
||||
Sr25519Keyring::Alice.public().into(),
|
||||
0_u32.into(),
|
||||
dummy_collator_signature(),
|
||||
);
|
||||
|
||||
let msg = protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
collator_protocol_message.clone(),
|
||||
);
|
||||
|
||||
let receivers = vec![peer_ids[1], peer_ids[2]];
|
||||
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: NetworkBridgeTxMessage::SendCollationMessages(vec![(
|
||||
receivers.clone(),
|
||||
Versioned::VStaging(msg.clone()),
|
||||
)]),
|
||||
})
|
||||
.await;
|
||||
|
||||
for peer in &receivers {
|
||||
assert_eq!(
|
||||
network_handle
|
||||
.next_network_action()
|
||||
.timeout(TIMEOUT)
|
||||
.await
|
||||
.expect("Timeout does not occur"),
|
||||
NetworkAction::WriteNotification(
|
||||
*peer,
|
||||
PeerSet::Collation,
|
||||
WireMessage::ProtocolMessage(msg.clone()).encode(),
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
@@ -6,7 +6,6 @@ edition.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
[dependencies]
|
||||
always-assert = "0.1.2"
|
||||
bitvec = { version = "1.0.1", default-features = false, features = ["alloc"] }
|
||||
futures = "0.3.21"
|
||||
futures-timer = "3"
|
||||
@@ -23,6 +22,7 @@ polkadot-node-subsystem-util = { path = "../../subsystem-util" }
|
||||
polkadot-node-subsystem = {path = "../../subsystem" }
|
||||
fatality = "0.0.6"
|
||||
thiserror = "1.0.31"
|
||||
tokio-util = "0.7.1"
|
||||
|
||||
[dev-dependencies]
|
||||
log = "0.4.17"
|
||||
@@ -31,6 +31,7 @@ assert_matches = "1.4.0"
|
||||
|
||||
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master", features = ["std"] }
|
||||
sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sc-keystore = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sc-network = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
parity-scale-codec = { version = "3.6.1", features = ["std"] }
|
||||
|
||||
|
||||
@@ -0,0 +1,162 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Primitives for tracking collations-related data.
|
||||
|
||||
use std::collections::{HashSet, VecDeque};
|
||||
|
||||
use futures::{future::BoxFuture, stream::FuturesUnordered};
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
request_response::{
|
||||
incoming::OutgoingResponse, v1 as protocol_v1, vstaging as protocol_vstaging,
|
||||
IncomingRequest,
|
||||
},
|
||||
PeerId,
|
||||
};
|
||||
use polkadot_node_primitives::PoV;
|
||||
use polkadot_primitives::{CandidateHash, CandidateReceipt, Hash, Id as ParaId};
|
||||
|
||||
/// The status of a collation as seen from the collator.
|
||||
pub enum CollationStatus {
|
||||
/// The collation was created, but we did not advertise it to any validator.
|
||||
Created,
|
||||
/// The collation was advertised to at least one validator.
|
||||
Advertised,
|
||||
/// The collation was requested by at least one validator.
|
||||
Requested,
|
||||
}
|
||||
|
||||
impl CollationStatus {
|
||||
/// Advance to the [`Self::Advertised`] status.
|
||||
///
|
||||
/// This ensures that `self` isn't already [`Self::Requested`].
|
||||
pub fn advance_to_advertised(&mut self) {
|
||||
if !matches!(self, Self::Requested) {
|
||||
*self = Self::Advertised;
|
||||
}
|
||||
}
|
||||
|
||||
/// Advance to the [`Self::Requested`] status.
|
||||
pub fn advance_to_requested(&mut self) {
|
||||
*self = Self::Requested;
|
||||
}
|
||||
}
|
||||
|
||||
/// A collation built by the collator.
|
||||
pub struct Collation {
|
||||
/// Candidate receipt.
|
||||
pub receipt: CandidateReceipt,
|
||||
/// Parent head-data hash.
|
||||
pub parent_head_data_hash: Hash,
|
||||
/// Proof to verify the state transition of the parachain.
|
||||
pub pov: PoV,
|
||||
/// Collation status.
|
||||
pub status: CollationStatus,
|
||||
}
|
||||
|
||||
/// Stores the state for waiting collation fetches per relay parent.
|
||||
#[derive(Default)]
|
||||
pub struct WaitingCollationFetches {
|
||||
/// A flag indicating that we have an ongoing request.
|
||||
/// This limits the number of collations being sent at any moment
|
||||
/// of time to 1 for each relay parent.
|
||||
///
|
||||
/// If set to `true`, any new request will be queued.
|
||||
pub collation_fetch_active: bool,
|
||||
/// The collation fetches waiting to be fulfilled.
|
||||
pub req_queue: VecDeque<VersionedCollationRequest>,
|
||||
/// All peers that are waiting or actively uploading.
|
||||
///
|
||||
/// We will not accept multiple requests from the same peer, otherwise our DoS protection of
|
||||
/// moving on to the next peer after `MAX_UNSHARED_UPLOAD_TIME` would be pointless.
|
||||
pub waiting_peers: HashSet<(PeerId, CandidateHash)>,
|
||||
}
|
||||
|
||||
/// Backwards-compatible wrapper for incoming collations requests.
|
||||
pub enum VersionedCollationRequest {
|
||||
V1(IncomingRequest<protocol_v1::CollationFetchingRequest>),
|
||||
VStaging(IncomingRequest<protocol_vstaging::CollationFetchingRequest>),
|
||||
}
|
||||
|
||||
impl From<IncomingRequest<protocol_v1::CollationFetchingRequest>> for VersionedCollationRequest {
|
||||
fn from(req: IncomingRequest<protocol_v1::CollationFetchingRequest>) -> Self {
|
||||
Self::V1(req)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<IncomingRequest<protocol_vstaging::CollationFetchingRequest>>
|
||||
for VersionedCollationRequest
|
||||
{
|
||||
fn from(req: IncomingRequest<protocol_vstaging::CollationFetchingRequest>) -> Self {
|
||||
Self::VStaging(req)
|
||||
}
|
||||
}
|
||||
|
||||
impl VersionedCollationRequest {
|
||||
/// Returns parachain id from the request payload.
|
||||
pub fn para_id(&self) -> ParaId {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.payload.para_id,
|
||||
VersionedCollationRequest::VStaging(req) => req.payload.para_id,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns relay parent from the request payload.
|
||||
pub fn relay_parent(&self) -> Hash {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.payload.relay_parent,
|
||||
VersionedCollationRequest::VStaging(req) => req.payload.relay_parent,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns id of the peer the request was received from.
|
||||
pub fn peer_id(&self) -> PeerId {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.peer,
|
||||
VersionedCollationRequest::VStaging(req) => req.peer,
|
||||
}
|
||||
}
|
||||
|
||||
/// Sends the response back to requester.
|
||||
pub fn send_outgoing_response(
|
||||
self,
|
||||
response: OutgoingResponse<protocol_v1::CollationFetchingResponse>,
|
||||
) -> Result<(), ()> {
|
||||
match self {
|
||||
VersionedCollationRequest::V1(req) => req.send_outgoing_response(response),
|
||||
VersionedCollationRequest::VStaging(req) => req.send_outgoing_response(response),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Result of the finished background send-collation task.
|
||||
///
|
||||
/// Note that if the timeout was hit the request doesn't get
|
||||
/// aborted, it only indicates that we should start processing
|
||||
/// the next one from the queue.
|
||||
pub struct CollationSendResult {
|
||||
/// Candidate's relay parent.
|
||||
pub relay_parent: Hash,
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Peer id.
|
||||
pub peer_id: PeerId,
|
||||
/// Whether the max unshared timeout was hit.
|
||||
pub timed_out: bool,
|
||||
}
|
||||
|
||||
pub type ActiveCollationFetches = FuturesUnordered<BoxFuture<'static, CollationSendResult>>;
|
||||
@@ -20,7 +20,7 @@ use polkadot_node_subsystem_util::metrics::{self, prometheus};
|
||||
pub struct Metrics(Option<MetricsInner>);
|
||||
|
||||
impl Metrics {
|
||||
pub fn on_advertisment_made(&self) {
|
||||
pub fn on_advertisement_made(&self) {
|
||||
if let Some(metrics) = &self.0 {
|
||||
metrics.advertisements_made.inc();
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+366
-113
@@ -37,6 +37,7 @@ use polkadot_node_network_protocol::{
|
||||
};
|
||||
use polkadot_node_primitives::BlockData;
|
||||
use polkadot_node_subsystem::{
|
||||
errors::RuntimeApiError,
|
||||
jaeger,
|
||||
messages::{AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest},
|
||||
ActivatedLeaf, ActiveLeavesUpdate, LeafStatus,
|
||||
@@ -49,8 +50,13 @@ use polkadot_primitives::{
|
||||
};
|
||||
use polkadot_primitives_test_helpers::TestCandidateBuilder;
|
||||
|
||||
mod prospective_parachains;
|
||||
|
||||
const REPUTATION_CHANGE_TEST_INTERVAL: Duration = Duration::from_millis(10);
|
||||
|
||||
const ASYNC_BACKING_DISABLED_ERROR: RuntimeApiError =
|
||||
RuntimeApiError::NotSupported { runtime_api_name: "test-runtime" };
|
||||
|
||||
#[derive(Clone)]
|
||||
struct TestState {
|
||||
para_id: ParaId,
|
||||
@@ -186,6 +192,17 @@ impl TestState {
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, self.relay_parent);
|
||||
tx.send(Err(ASYNC_BACKING_DISABLED_ERROR)).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -193,7 +210,8 @@ type VirtualOverseer = test_helpers::TestSubsystemContextHandle<CollatorProtocol
|
||||
|
||||
struct TestHarness {
|
||||
virtual_overseer: VirtualOverseer,
|
||||
req_cfg: sc_network::config::RequestResponseConfig,
|
||||
req_v1_cfg: sc_network::config::RequestResponseConfig,
|
||||
req_vstaging_cfg: sc_network::config::RequestResponseConfig,
|
||||
}
|
||||
|
||||
fn test_harness<T: Future<Output = TestHarness>>(
|
||||
@@ -215,7 +233,9 @@ fn test_harness<T: Future<Output = TestHarness>>(
|
||||
let genesis_hash = Hash::repeat_byte(0xff);
|
||||
let req_protocol_names = ReqProtocolNames::new(&genesis_hash, None);
|
||||
|
||||
let (collation_req_receiver, req_cfg) =
|
||||
let (collation_req_receiver, req_v1_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (collation_req_vstaging_receiver, req_vstaging_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let subsystem = async {
|
||||
run_inner(
|
||||
@@ -223,6 +243,7 @@ fn test_harness<T: Future<Output = TestHarness>>(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
collation_req_receiver,
|
||||
collation_req_vstaging_receiver,
|
||||
Default::default(),
|
||||
reputation,
|
||||
REPUTATION_CHANGE_TEST_INTERVAL,
|
||||
@@ -231,7 +252,7 @@ fn test_harness<T: Future<Output = TestHarness>>(
|
||||
.unwrap();
|
||||
};
|
||||
|
||||
let test_fut = test(TestHarness { virtual_overseer, req_cfg });
|
||||
let test_fut = test(TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg });
|
||||
|
||||
futures::pin_mut!(test_fut);
|
||||
futures::pin_mut!(subsystem);
|
||||
@@ -305,6 +326,17 @@ async fn setup_system(virtual_overseer: &mut VirtualOverseer, test_state: &TestS
|
||||
])),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
tx.send(Err(ASYNC_BACKING_DISABLED_ERROR)).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/// Result of [`distribute_collation`]
|
||||
@@ -313,29 +345,23 @@ struct DistributeCollation {
|
||||
pov_block: PoV,
|
||||
}
|
||||
|
||||
/// Create some PoV and distribute it.
|
||||
async fn distribute_collation(
|
||||
async fn distribute_collation_with_receipt(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
// whether or not we expect a connection request or not.
|
||||
relay_parent: Hash,
|
||||
should_connect: bool,
|
||||
candidate: CandidateReceipt,
|
||||
pov: PoV,
|
||||
parent_head_data_hash: Hash,
|
||||
) -> DistributeCollation {
|
||||
// Now we want to distribute a `PoVBlock`
|
||||
let pov_block = PoV { block_data: BlockData(vec![42, 43, 44]) };
|
||||
|
||||
let pov_hash = pov_block.hash();
|
||||
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: test_state.relay_parent,
|
||||
pov_hash,
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::DistributeCollation(candidate.clone(), pov_block.clone(), None),
|
||||
CollatorProtocolMessage::DistributeCollation(
|
||||
candidate.clone(),
|
||||
parent_head_data_hash,
|
||||
pov.clone(),
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -343,10 +369,10 @@ async fn distribute_collation(
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
_relay_parent,
|
||||
RuntimeApiRequest::AvailabilityCores(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(relay_parent, _relay_parent);
|
||||
tx.send(Ok(test_state.availability_cores.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
@@ -358,7 +384,7 @@ async fn distribute_collation(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::SessionIndexForChild(tx),
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(relay_parent, relay_parent);
|
||||
tx.send(Ok(test_state.current_session_index())).unwrap();
|
||||
},
|
||||
|
||||
@@ -366,17 +392,17 @@ async fn distribute_collation(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::SessionInfo(index, tx),
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(relay_parent, relay_parent);
|
||||
assert_eq!(index, test_state.current_session_index());
|
||||
|
||||
tx.send(Ok(Some(test_state.session_info.clone()))).unwrap();
|
||||
},
|
||||
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
_relay_parent,
|
||||
RuntimeApiRequest::ValidatorGroups(tx),
|
||||
)) => {
|
||||
assert_eq!(relay_parent, test_state.relay_parent);
|
||||
assert_eq!(_relay_parent, relay_parent);
|
||||
tx.send(Ok((
|
||||
test_state.session_info.validator_groups.to_vec(),
|
||||
test_state.group_rotation_info.clone(),
|
||||
@@ -400,13 +426,48 @@ async fn distribute_collation(
|
||||
);
|
||||
}
|
||||
|
||||
DistributeCollation { candidate, pov_block }
|
||||
DistributeCollation { candidate, pov_block: pov }
|
||||
}
|
||||
|
||||
/// Create some PoV and distribute it.
|
||||
async fn distribute_collation(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
relay_parent: Hash,
|
||||
// whether or not we expect a connection request or not.
|
||||
should_connect: bool,
|
||||
) -> DistributeCollation {
|
||||
// Now we want to distribute a `PoVBlock`
|
||||
let pov_block = PoV { block_data: BlockData(vec![42, 43, 44]) };
|
||||
|
||||
let pov_hash = pov_block.hash();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent,
|
||||
pov_hash,
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
test_state,
|
||||
relay_parent,
|
||||
should_connect,
|
||||
candidate,
|
||||
pov_block,
|
||||
parent_head_data_hash,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// Connect a peer
|
||||
async fn connect_peer(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
version: CollationVersion,
|
||||
authority_id: Option<AuthorityDiscoveryId>,
|
||||
) {
|
||||
overseer_send(
|
||||
@@ -414,7 +475,7 @@ async fn connect_peer(
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
polkadot_node_network_protocol::ObservedRole::Authority,
|
||||
CollationVersion::V1.into(),
|
||||
version.into(),
|
||||
authority_id.map(|v| HashSet::from([v])),
|
||||
)),
|
||||
)
|
||||
@@ -474,30 +535,65 @@ async fn expect_declare_msg(
|
||||
}
|
||||
|
||||
/// Check that the next received message is a collation advertisement message.
|
||||
///
|
||||
/// Expects vstaging message if `expected_candidate_hashes` is `Some`, v1 otherwise.
|
||||
async fn expect_advertise_collation_msg(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: &PeerId,
|
||||
expected_relay_parent: Hash,
|
||||
expected_candidate_hashes: Option<Vec<CandidateHash>>,
|
||||
) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendCollationMessage(
|
||||
to,
|
||||
Versioned::V1(protocol_v1::CollationProtocol::CollatorProtocol(wire_message)),
|
||||
)
|
||||
) => {
|
||||
assert_eq!(to[0], *peer);
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_v1::CollatorProtocolMessage::AdvertiseCollation(
|
||||
relay_parent,
|
||||
) => {
|
||||
assert_eq!(relay_parent, expected_relay_parent);
|
||||
let mut candidate_hashes: Option<HashSet<_>> =
|
||||
expected_candidate_hashes.map(|hashes| hashes.into_iter().collect());
|
||||
let iter_num = candidate_hashes.as_ref().map(|hashes| hashes.len()).unwrap_or(1);
|
||||
|
||||
for _ in 0..iter_num {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::SendCollationMessage(
|
||||
to,
|
||||
wire_message,
|
||||
)
|
||||
) => {
|
||||
assert_eq!(to[0], *peer);
|
||||
match (candidate_hashes.as_mut(), wire_message) {
|
||||
(None, Versioned::V1(protocol_v1::CollationProtocol::CollatorProtocol(wire_message))) => {
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_v1::CollatorProtocolMessage::AdvertiseCollation(
|
||||
relay_parent,
|
||||
) => {
|
||||
assert_eq!(relay_parent, expected_relay_parent);
|
||||
}
|
||||
);
|
||||
},
|
||||
(
|
||||
Some(candidate_hashes),
|
||||
Versioned::VStaging(protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
wire_message,
|
||||
)),
|
||||
) => {
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_vstaging::CollatorProtocolMessage::AdvertiseCollation {
|
||||
relay_parent,
|
||||
candidate_hash,
|
||||
..
|
||||
} => {
|
||||
assert_eq!(relay_parent, expected_relay_parent);
|
||||
assert!(candidate_hashes.contains(&candidate_hash));
|
||||
|
||||
// Drop the hash we've already seen.
|
||||
candidate_hashes.remove(&candidate_hash);
|
||||
}
|
||||
);
|
||||
},
|
||||
_ => panic!("Invalid advertisement"),
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Send a message that the given peer's view changed.
|
||||
@@ -528,19 +624,26 @@ fn advertise_and_send_collation() {
|
||||
ReputationAggregator::new(|_| true),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let mut req_cfg = test_harness.req_cfg;
|
||||
let mut req_v1_cfg = test_harness.req_v1_cfg;
|
||||
let req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
setup_system(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let DistributeCollation { candidate, pov_block } =
|
||||
distribute_collation(&mut virtual_overseer, &test_state, true).await;
|
||||
let DistributeCollation { candidate, pov_block } = distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
true,
|
||||
)
|
||||
.await;
|
||||
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(test_state.current_group_validator_peer_ids())
|
||||
{
|
||||
connect_peer(&mut virtual_overseer, peer, Some(val.clone())).await;
|
||||
connect_peer(&mut virtual_overseer, peer, CollationVersion::V1, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
// We declare to the connected validators that we are a collator.
|
||||
@@ -558,18 +661,23 @@ fn advertise_and_send_collation() {
|
||||
|
||||
// The peer is interested in a leaf that we have a collation for;
|
||||
// advertise it.
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
&mut virtual_overseer,
|
||||
&peer,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Request a collation.
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -582,13 +690,13 @@ fn advertise_and_send_collation() {
|
||||
{
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -613,8 +721,8 @@ fn advertise_and_send_collation() {
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(receipt, pov): CollationFetchingResponse
|
||||
= CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
@@ -632,13 +740,13 @@ fn advertise_and_send_collation() {
|
||||
// Re-request a collation.
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: old_relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -652,7 +760,8 @@ fn advertise_and_send_collation() {
|
||||
|
||||
assert!(overseer_recv_with_timeout(&mut virtual_overseer, TIMEOUT).await.is_none());
|
||||
|
||||
distribute_collation(&mut virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(&mut virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
// Send info about peer's view.
|
||||
overseer_send(
|
||||
@@ -664,9 +773,14 @@ fn advertise_and_send_collation() {
|
||||
)
|
||||
.await;
|
||||
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, test_state.relay_parent)
|
||||
.await;
|
||||
TestHarness { virtual_overseer, req_cfg }
|
||||
expect_advertise_collation_msg(
|
||||
&mut virtual_overseer,
|
||||
&peer,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg }
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -683,18 +797,26 @@ fn delay_reputation_change() {
|
||||
ReputationAggregator::new(|_| false),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let mut req_cfg = test_harness.req_cfg;
|
||||
let mut req_v1_cfg = test_harness.req_v1_cfg;
|
||||
let req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
setup_system(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let _ = distribute_collation(&mut virtual_overseer, &test_state, true).await;
|
||||
let _ = distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
true,
|
||||
)
|
||||
.await;
|
||||
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(test_state.current_group_validator_peer_ids())
|
||||
{
|
||||
connect_peer(&mut virtual_overseer, peer, Some(val.clone())).await;
|
||||
connect_peer(&mut virtual_overseer, peer, CollationVersion::V1, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
// We declare to the connected validators that we are a collator.
|
||||
@@ -712,18 +834,23 @@ fn delay_reputation_change() {
|
||||
|
||||
// The peer is interested in a leaf that we have a collation for;
|
||||
// advertise it.
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
&mut virtual_overseer,
|
||||
&peer,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Request a collation.
|
||||
let (pending_response, _rx) = oneshot::channel();
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -736,13 +863,13 @@ fn delay_reputation_change() {
|
||||
{
|
||||
let (pending_response, _rx) = oneshot::channel();
|
||||
|
||||
req_cfg
|
||||
req_v1_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -767,7 +894,90 @@ fn delay_reputation_change() {
|
||||
);
|
||||
}
|
||||
|
||||
TestHarness { virtual_overseer, req_cfg }
|
||||
TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg }
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/// Tests that collator side works with vstaging network protocol
|
||||
/// before async backing is enabled.
|
||||
#[test]
|
||||
fn advertise_collation_vstaging_protocol() {
|
||||
let test_state = TestState::default();
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
let DistributeCollation { candidate, .. } =
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
let validators = test_state.current_group_validator_authority_ids();
|
||||
assert!(validators.len() >= 2);
|
||||
let peer_ids = test_state.current_group_validator_peer_ids();
|
||||
|
||||
// Connect first peer with v1.
|
||||
connect_peer(
|
||||
virtual_overseer,
|
||||
peer_ids[0],
|
||||
CollationVersion::V1,
|
||||
Some(validators[0].clone()),
|
||||
)
|
||||
.await;
|
||||
// The rest with vstaging.
|
||||
for (val, peer) in validators.iter().zip(peer_ids.iter()).skip(1) {
|
||||
connect_peer(
|
||||
virtual_overseer,
|
||||
*peer,
|
||||
CollationVersion::VStaging,
|
||||
Some(val.clone()),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
// Declare messages.
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer_ids[0]).await;
|
||||
for peer_id in peer_ids.iter().skip(1) {
|
||||
prospective_parachains::expect_declare_msg_vstaging(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
&peer_id,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
// Send info about peers view.
|
||||
for peer in peer_ids.iter() {
|
||||
send_peer_view_change(virtual_overseer, peer, vec![test_state.relay_parent]).await;
|
||||
}
|
||||
|
||||
// Versioned advertisements work.
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&peer_ids[0],
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
for peer_id in peer_ids.iter().skip(1) {
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
peer_id,
|
||||
test_state.relay_parent,
|
||||
Some(vec![candidate.hash()]), // This is `Some`, advertisement is vstaging.
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
test_harness
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -814,7 +1024,13 @@ fn collators_declare_to_connected_peers() {
|
||||
setup_system(&mut test_harness.virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(&mut test_harness.virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(
|
||||
&mut test_harness.virtual_overseer,
|
||||
peer,
|
||||
CollationVersion::V1,
|
||||
Some(validator_id),
|
||||
)
|
||||
.await;
|
||||
expect_declare_msg(&mut test_harness.virtual_overseer, &test_state, &peer).await;
|
||||
test_harness
|
||||
},
|
||||
@@ -843,10 +1059,10 @@ fn collations_are_only_advertised_to_validators_with_correct_view() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
|
||||
// Connect the second validator
|
||||
connect_peer(virtual_overseer, peer2, Some(validator_id2)).await;
|
||||
connect_peer(virtual_overseer, peer2, CollationVersion::V1, Some(validator_id2)).await;
|
||||
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer2).await;
|
||||
@@ -854,15 +1070,18 @@ fn collations_are_only_advertised_to_validators_with_correct_view() {
|
||||
// And let it tell us that it is has the same view.
|
||||
send_peer_view_change(virtual_overseer, &peer2, vec![test_state.relay_parent]).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent, None)
|
||||
.await;
|
||||
|
||||
// The other validator announces that it changed its view.
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![test_state.relay_parent]).await;
|
||||
|
||||
// After changing the view we should receive the advertisement
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent, None)
|
||||
.await;
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
@@ -890,15 +1109,16 @@ fn collate_on_two_different_relay_chain_blocks() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
|
||||
// Connect the second validator
|
||||
connect_peer(virtual_overseer, peer2, Some(validator_id2)).await;
|
||||
connect_peer(virtual_overseer, peer2, CollationVersion::V1, Some(validator_id2)).await;
|
||||
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer2).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
let old_relay_parent = test_state.relay_parent;
|
||||
|
||||
@@ -906,14 +1126,16 @@ fn collate_on_two_different_relay_chain_blocks() {
|
||||
// parent are active.
|
||||
test_state.advance_to_new_round(virtual_overseer, true).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![old_relay_parent]).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, old_relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, old_relay_parent, None).await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer2, vec![test_state.relay_parent]).await;
|
||||
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer2, test_state.relay_parent, None)
|
||||
.await;
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
@@ -938,17 +1160,20 @@ fn validator_reconnect_does_not_advertise_a_second_time() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id.clone())).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id.clone()))
|
||||
.await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![test_state.relay_parent]).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent).await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &peer, test_state.relay_parent, None)
|
||||
.await;
|
||||
|
||||
// Disconnect and reconnect directly
|
||||
disconnect_peer(virtual_overseer, peer).await;
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
send_peer_view_change(virtual_overseer, &peer, vec![test_state.relay_parent]).await;
|
||||
@@ -979,7 +1204,7 @@ fn collators_reject_declare_messages() {
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
// A validator connected to us
|
||||
connect_peer(virtual_overseer, peer, Some(validator_id)).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(validator_id)).await;
|
||||
expect_declare_msg(virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
overseer_send(
|
||||
@@ -1031,19 +1256,20 @@ where
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
let req_cfg = &mut test_harness.req_cfg;
|
||||
let req_cfg = &mut test_harness.req_v1_cfg;
|
||||
|
||||
setup_system(virtual_overseer, &test_state).await;
|
||||
|
||||
let DistributeCollation { candidate, pov_block } =
|
||||
distribute_collation(virtual_overseer, &test_state, true).await;
|
||||
distribute_collation(virtual_overseer, &test_state, test_state.relay_parent, true)
|
||||
.await;
|
||||
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(test_state.current_group_validator_peer_ids())
|
||||
{
|
||||
connect_peer(virtual_overseer, peer, Some(val.clone())).await;
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::V1, Some(val.clone())).await;
|
||||
}
|
||||
|
||||
// We declare to the connected validators that we are a collator.
|
||||
@@ -1064,10 +1290,20 @@ where
|
||||
|
||||
// The peer is interested in a leaf that we have a collation for;
|
||||
// advertise it.
|
||||
expect_advertise_collation_msg(virtual_overseer, &validator_0, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(virtual_overseer, &validator_1, test_state.relay_parent)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&validator_0,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&validator_1,
|
||||
test_state.relay_parent,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Request a collation.
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
@@ -1077,7 +1313,7 @@ where
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer: validator_0,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -1092,8 +1328,8 @@ where
|
||||
let feedback_tx = assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(receipt, pov): CollationFetchingResponse
|
||||
= CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
@@ -1113,7 +1349,7 @@ where
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer: validator_1,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: test_state.relay_parent,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -1129,8 +1365,8 @@ where
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(receipt, pov): CollationFetchingResponse
|
||||
= CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
@@ -1159,7 +1395,8 @@ fn connect_to_buffered_groups() {
|
||||
ReputationAggregator::new(|_| true),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let mut req_cfg = test_harness.req_cfg;
|
||||
let mut req_cfg = test_harness.req_v1_cfg;
|
||||
let req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
setup_system(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
@@ -1167,7 +1404,13 @@ fn connect_to_buffered_groups() {
|
||||
let peers_a = test_state.current_group_validator_peer_ids();
|
||||
assert!(group_a.len() > 1);
|
||||
|
||||
distribute_collation(&mut virtual_overseer, &test_state, false).await;
|
||||
distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
false,
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
@@ -1181,7 +1424,8 @@ fn connect_to_buffered_groups() {
|
||||
let head_a = test_state.relay_parent;
|
||||
|
||||
for (val, peer) in group_a.iter().zip(&peers_a) {
|
||||
connect_peer(&mut virtual_overseer, *peer, Some(val.clone())).await;
|
||||
connect_peer(&mut virtual_overseer, *peer, CollationVersion::V1, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
for peer_id in &peers_a {
|
||||
@@ -1191,7 +1435,7 @@ fn connect_to_buffered_groups() {
|
||||
// Update views.
|
||||
for peed_id in &peers_a {
|
||||
send_peer_view_change(&mut virtual_overseer, peed_id, vec![head_a]).await;
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, peed_id, head_a).await;
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, peed_id, head_a, None).await;
|
||||
}
|
||||
|
||||
let peer = peers_a[0];
|
||||
@@ -1203,7 +1447,7 @@ fn connect_to_buffered_groups() {
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: CollationFetchingRequest {
|
||||
payload: request_v1::CollationFetchingRequest {
|
||||
relay_parent: head_a,
|
||||
para_id: test_state.para_id,
|
||||
}
|
||||
@@ -1215,14 +1459,17 @@ fn connect_to_buffered_groups() {
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
let CollationFetchingResponse::Collation(..): CollationFetchingResponse =
|
||||
CollationFetchingResponse::decode(
|
||||
let request_v1::CollationFetchingResponse::Collation(..) =
|
||||
request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result.expect("We should have a proper answer").as_ref(),
|
||||
)
|
||||
.expect("Decoding should work");
|
||||
}
|
||||
);
|
||||
|
||||
// Let the subsystem process process the collation event.
|
||||
test_helpers::Yield::new().await;
|
||||
|
||||
test_state.advance_to_new_round(&mut virtual_overseer, true).await;
|
||||
test_state.group_rotation_info = test_state.group_rotation_info.bump_rotation();
|
||||
|
||||
@@ -1231,7 +1478,13 @@ fn connect_to_buffered_groups() {
|
||||
assert_ne!(head_a, head_b);
|
||||
assert_ne!(group_a, group_b);
|
||||
|
||||
distribute_collation(&mut virtual_overseer, &test_state, false).await;
|
||||
distribute_collation(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
test_state.relay_parent,
|
||||
false,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Should be connected to both groups except for the validator that fetched advertised
|
||||
// collation.
|
||||
@@ -1248,7 +1501,7 @@ fn connect_to_buffered_groups() {
|
||||
}
|
||||
);
|
||||
|
||||
TestHarness { virtual_overseer, req_cfg }
|
||||
TestHarness { virtual_overseer, req_v1_cfg: req_cfg, req_vstaging_cfg }
|
||||
},
|
||||
);
|
||||
}
|
||||
+575
@@ -0,0 +1,575 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Tests for the collator side with enabled prospective parachains.
|
||||
|
||||
use super::*;
|
||||
|
||||
use polkadot_node_subsystem::messages::{ChainApiMessage, ProspectiveParachainsMessage};
|
||||
use polkadot_primitives::{vstaging as vstaging_primitives, Header, OccupiedCore};
|
||||
|
||||
const ASYNC_BACKING_PARAMETERS: vstaging_primitives::AsyncBackingParams =
|
||||
vstaging_primitives::AsyncBackingParams { max_candidate_depth: 4, allowed_ancestry_len: 3 };
|
||||
|
||||
fn get_parent_hash(hash: Hash) -> Hash {
|
||||
Hash::from_low_u64_be(hash.to_low_u64_be() + 1)
|
||||
}
|
||||
|
||||
/// Handle a view update.
|
||||
async fn update_view(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
new_view: Vec<(Hash, u32)>, // Hash and block number.
|
||||
activated: u8, // How many new heads does this update contain?
|
||||
) {
|
||||
let new_view: HashMap<Hash, u32> = HashMap::from_iter(new_view);
|
||||
|
||||
let our_view =
|
||||
OurView::new(new_view.keys().map(|hash| (*hash, Arc::new(jaeger::Span::Disabled))), 0);
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(our_view)),
|
||||
)
|
||||
.await;
|
||||
|
||||
let mut next_overseer_message = None;
|
||||
for _ in 0..activated {
|
||||
let (leaf_hash, leaf_number) = assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx),
|
||||
)) => {
|
||||
tx.send(Ok(ASYNC_BACKING_PARAMETERS)).unwrap();
|
||||
(parent, new_view.get(&parent).copied().expect("Unknown parent requested"))
|
||||
}
|
||||
);
|
||||
|
||||
let min_number = leaf_number.saturating_sub(ASYNC_BACKING_PARAMETERS.allowed_ancestry_len);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetMinimumRelayParents(parent, tx),
|
||||
) if parent == leaf_hash => {
|
||||
tx.send(vec![(test_state.para_id, min_number)]).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let ancestry_len = leaf_number + 1 - min_number;
|
||||
let ancestry_hashes = std::iter::successors(Some(leaf_hash), |h| Some(get_parent_hash(*h)))
|
||||
.take(ancestry_len as usize);
|
||||
let ancestry_numbers = (min_number..=leaf_number).rev();
|
||||
let mut ancestry_iter = ancestry_hashes.clone().zip(ancestry_numbers).peekable();
|
||||
|
||||
while let Some((hash, number)) = ancestry_iter.next() {
|
||||
// May be `None` for the last element.
|
||||
let parent_hash =
|
||||
ancestry_iter.peek().map(|(h, _)| *h).unwrap_or_else(|| get_parent_hash(hash));
|
||||
|
||||
let msg = match next_overseer_message.take() {
|
||||
Some(msg) => Some(msg),
|
||||
None =>
|
||||
overseer_recv_with_timeout(virtual_overseer, Duration::from_millis(50)).await,
|
||||
};
|
||||
|
||||
let msg = match msg {
|
||||
Some(msg) => msg,
|
||||
None => {
|
||||
// We're done.
|
||||
return
|
||||
},
|
||||
};
|
||||
|
||||
if !matches!(
|
||||
&msg,
|
||||
AllMessages::ChainApi(ChainApiMessage::BlockHeader(_hash, ..))
|
||||
if *_hash == hash
|
||||
) {
|
||||
// Ancestry has already been cached for this leaf.
|
||||
next_overseer_message.replace(msg);
|
||||
break
|
||||
}
|
||||
|
||||
assert_matches!(
|
||||
msg,
|
||||
AllMessages::ChainApi(ChainApiMessage::BlockHeader(.., tx)) => {
|
||||
let header = Header {
|
||||
parent_hash,
|
||||
number,
|
||||
state_root: Hash::zero(),
|
||||
extrinsics_root: Hash::zero(),
|
||||
digest: Default::default(),
|
||||
};
|
||||
|
||||
tx.send(Ok(Some(header))).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Check that the next received message is a `Declare` message.
|
||||
pub(super) async fn expect_declare_msg_vstaging(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
peer: &PeerId,
|
||||
) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendCollationMessage(
|
||||
to,
|
||||
Versioned::VStaging(protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
wire_message,
|
||||
)),
|
||||
)) => {
|
||||
assert_eq!(to[0], *peer);
|
||||
assert_matches!(
|
||||
wire_message,
|
||||
protocol_vstaging::CollatorProtocolMessage::Declare(
|
||||
collator_id,
|
||||
para_id,
|
||||
signature,
|
||||
) => {
|
||||
assert!(signature.verify(
|
||||
&*protocol_vstaging::declare_signature_payload(&test_state.local_peer_id),
|
||||
&collator_id),
|
||||
);
|
||||
assert_eq!(collator_id, test_state.collator_pair.public());
|
||||
assert_eq!(para_id, test_state.para_id);
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/// Test that a collator distributes a collation from the allowed ancestry
|
||||
/// to correct validators group.
|
||||
#[test]
|
||||
fn distribute_collation_from_implicit_view() {
|
||||
let head_a = Hash::from_low_u64_be(126);
|
||||
let head_a_num: u32 = 66;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 64;
|
||||
|
||||
// Grandparent of head `b`.
|
||||
let head_c = Hash::from_low_u64_be(130);
|
||||
let head_c_num = 62;
|
||||
|
||||
let group_rotation_info = GroupRotationInfo {
|
||||
session_start_block: head_c_num - 2,
|
||||
group_rotation_frequency: 3,
|
||||
now: head_c_num,
|
||||
};
|
||||
|
||||
let mut test_state = TestState::default();
|
||||
test_state.group_rotation_info = group_rotation_info;
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(virtual_overseer, CollatorProtocolMessage::CollateOn(test_state.para_id))
|
||||
.await;
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let validator_peer_ids = test_state.current_group_validator_peer_ids();
|
||||
for (val, peer) in test_state
|
||||
.current_group_validator_authority_ids()
|
||||
.into_iter()
|
||||
.zip(validator_peer_ids.clone())
|
||||
{
|
||||
connect_peer(virtual_overseer, peer, CollationVersion::VStaging, Some(val.clone()))
|
||||
.await;
|
||||
}
|
||||
|
||||
// Collator declared itself to each peer.
|
||||
for peer_id in &validator_peer_ids {
|
||||
expect_declare_msg_vstaging(virtual_overseer, &test_state, peer_id).await;
|
||||
}
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1, 2, 3]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xAA);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_c,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
let DistributeCollation { candidate, pov_block: _ } =
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
head_c,
|
||||
false, // Check the group manually.
|
||||
candidate,
|
||||
pov,
|
||||
parent_head_data_hash,
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ConnectToValidators { validator_ids, .. }
|
||||
) => {
|
||||
let expected_validators = test_state.current_group_validator_authority_ids();
|
||||
|
||||
assert_eq!(expected_validators, validator_ids);
|
||||
}
|
||||
);
|
||||
|
||||
let candidate_hash = candidate.hash();
|
||||
|
||||
// Update peer views.
|
||||
for peed_id in &validator_peer_ids {
|
||||
send_peer_view_change(virtual_overseer, peed_id, vec![head_b]).await;
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
peed_id,
|
||||
head_c,
|
||||
Some(vec![candidate_hash]),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
// Head `c` goes out of view.
|
||||
// Build a different candidate for this relay parent and attempt to distribute it.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![4, 5, 6]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xBB);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_c,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::DistributeCollation(
|
||||
candidate.clone(),
|
||||
parent_head_data_hash,
|
||||
pov.clone(),
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Parent out of view, nothing happens.
|
||||
assert!(overseer_recv_with_timeout(virtual_overseer, Duration::from_millis(100))
|
||||
.await
|
||||
.is_none());
|
||||
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Tests that collator can distribute up to `MAX_CANDIDATE_DEPTH + 1` candidates
|
||||
/// per relay parent.
|
||||
#[test]
|
||||
fn distribute_collation_up_to_limit() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
let head_a = Hash::from_low_u64_be(128);
|
||||
let head_a_num: u32 = 64;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(130);
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(virtual_overseer, CollatorProtocolMessage::CollateOn(test_state.para_id))
|
||||
.await;
|
||||
// Activated leaf is `a`, but the collation will be based on `b`.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
for i in 0..(ASYNC_BACKING_PARAMETERS.max_candidate_depth + 1) {
|
||||
let pov = PoV { block_data: BlockData(vec![i as u8]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xAA);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
head_b,
|
||||
true,
|
||||
candidate,
|
||||
pov,
|
||||
parent_head_data_hash,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![10, 12, 6]) };
|
||||
let parent_head_data_hash = Hash::repeat_byte(0xBB);
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::DistributeCollation(
|
||||
candidate.clone(),
|
||||
parent_head_data_hash,
|
||||
pov.clone(),
|
||||
None,
|
||||
),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Limit has been reached.
|
||||
assert!(overseer_recv_with_timeout(virtual_overseer, Duration::from_millis(100))
|
||||
.await
|
||||
.is_none());
|
||||
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Tests that collator correctly handles peer V2 requests.
|
||||
#[test]
|
||||
fn advertise_and_send_collation_by_hash() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|test_harness| async move {
|
||||
let mut virtual_overseer = test_harness.virtual_overseer;
|
||||
let req_v1_cfg = test_harness.req_v1_cfg;
|
||||
let mut req_vstaging_cfg = test_harness.req_vstaging_cfg;
|
||||
|
||||
let head_a = Hash::from_low_u64_be(128);
|
||||
let head_a_num: u32 = 64;
|
||||
|
||||
// Parent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(129);
|
||||
let head_b_num: u32 = 63;
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::CollateOn(test_state.para_id),
|
||||
)
|
||||
.await;
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
let candidates: Vec<_> = (0..2)
|
||||
.map(|i| {
|
||||
let pov = PoV { block_data: BlockData(vec![i as u8]) };
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
(candidate, pov)
|
||||
})
|
||||
.collect();
|
||||
for (candidate, pov) in &candidates {
|
||||
distribute_collation_with_receipt(
|
||||
&mut virtual_overseer,
|
||||
&test_state,
|
||||
head_b,
|
||||
true,
|
||||
candidate.clone(),
|
||||
pov.clone(),
|
||||
Hash::zero(),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let peer = test_state.validator_peer_id[0];
|
||||
let validator_id = test_state.current_group_validator_authority_ids()[0].clone();
|
||||
connect_peer(
|
||||
&mut virtual_overseer,
|
||||
peer,
|
||||
CollationVersion::VStaging,
|
||||
Some(validator_id.clone()),
|
||||
)
|
||||
.await;
|
||||
expect_declare_msg_vstaging(&mut virtual_overseer, &test_state, &peer).await;
|
||||
|
||||
// Head `b` is not a leaf, but both advertisements are still relevant.
|
||||
send_peer_view_change(&mut virtual_overseer, &peer, vec![head_b]).await;
|
||||
let hashes: Vec<_> = candidates.iter().map(|(candidate, _)| candidate.hash()).collect();
|
||||
expect_advertise_collation_msg(&mut virtual_overseer, &peer, head_b, Some(hashes))
|
||||
.await;
|
||||
|
||||
for (candidate, pov_block) in candidates {
|
||||
let (pending_response, rx) = oneshot::channel();
|
||||
req_vstaging_cfg
|
||||
.inbound_queue
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.send(RawIncomingRequest {
|
||||
peer,
|
||||
payload: request_vstaging::CollationFetchingRequest {
|
||||
relay_parent: head_b,
|
||||
para_id: test_state.para_id,
|
||||
candidate_hash: candidate.hash(),
|
||||
}
|
||||
.encode(),
|
||||
pending_response,
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert_matches!(
|
||||
rx.await,
|
||||
Ok(full_response) => {
|
||||
// Response is the same for vstaging.
|
||||
let request_v1::CollationFetchingResponse::Collation(receipt, pov): request_v1::CollationFetchingResponse
|
||||
= request_v1::CollationFetchingResponse::decode(
|
||||
&mut full_response.result
|
||||
.expect("We should have a proper answer").as_ref()
|
||||
)
|
||||
.expect("Decoding should work");
|
||||
assert_eq!(receipt, candidate);
|
||||
assert_eq!(pov, pov_block);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
TestHarness { virtual_overseer, req_v1_cfg, req_vstaging_cfg }
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Tests that collator distributes collation built on top of occupied core.
|
||||
#[test]
|
||||
fn advertise_core_occupied() {
|
||||
let mut test_state = TestState::default();
|
||||
let candidate =
|
||||
TestCandidateBuilder { para_id: test_state.para_id, ..Default::default() }.build();
|
||||
test_state.availability_cores[0] = CoreState::Occupied(OccupiedCore {
|
||||
next_up_on_available: None,
|
||||
occupied_since: 0,
|
||||
time_out_at: 0,
|
||||
next_up_on_time_out: None,
|
||||
availability: BitVec::default(),
|
||||
group_responsible: GroupIndex(0),
|
||||
candidate_hash: candidate.hash(),
|
||||
candidate_descriptor: candidate.descriptor,
|
||||
});
|
||||
|
||||
let local_peer_id = test_state.local_peer_id;
|
||||
let collator_pair = test_state.collator_pair.clone();
|
||||
|
||||
test_harness(
|
||||
local_peer_id,
|
||||
collator_pair,
|
||||
ReputationAggregator::new(|_| true),
|
||||
|mut test_harness| async move {
|
||||
let virtual_overseer = &mut test_harness.virtual_overseer;
|
||||
|
||||
let head_a = Hash::from_low_u64_be(128);
|
||||
let head_a_num: u32 = 64;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(130);
|
||||
|
||||
// Set collating para id.
|
||||
overseer_send(virtual_overseer, CollatorProtocolMessage::CollateOn(test_state.para_id))
|
||||
.await;
|
||||
// Activated leaf is `a`, but the collation will be based on `b`.
|
||||
update_view(virtual_overseer, &test_state, vec![(head_a, head_a_num)], 1).await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1, 2, 3]) };
|
||||
let candidate = TestCandidateBuilder {
|
||||
para_id: test_state.para_id,
|
||||
relay_parent: head_b,
|
||||
pov_hash: pov.hash(),
|
||||
..Default::default()
|
||||
}
|
||||
.build();
|
||||
let candidate_hash = candidate.hash();
|
||||
distribute_collation_with_receipt(
|
||||
virtual_overseer,
|
||||
&test_state,
|
||||
head_b,
|
||||
true,
|
||||
candidate,
|
||||
pov,
|
||||
Hash::zero(),
|
||||
)
|
||||
.await;
|
||||
|
||||
let validators = test_state.current_group_validator_authority_ids();
|
||||
let peer_ids = test_state.current_group_validator_peer_ids();
|
||||
|
||||
connect_peer(
|
||||
virtual_overseer,
|
||||
peer_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
Some(validators[0].clone()),
|
||||
)
|
||||
.await;
|
||||
expect_declare_msg_vstaging(virtual_overseer, &test_state, &peer_ids[0]).await;
|
||||
// Peer is aware of the leaf.
|
||||
send_peer_view_change(virtual_overseer, &peer_ids[0], vec![head_a]).await;
|
||||
|
||||
// Collation is advertised.
|
||||
expect_advertise_collation_msg(
|
||||
virtual_overseer,
|
||||
&peer_ids[0],
|
||||
head_b,
|
||||
Some(vec![candidate_hash]),
|
||||
)
|
||||
.await;
|
||||
|
||||
test_harness
|
||||
},
|
||||
)
|
||||
}
|
||||
@@ -31,13 +31,19 @@
|
||||
|
||||
use std::{
|
||||
collections::{HashMap, VecDeque},
|
||||
future::Future,
|
||||
num::NonZeroUsize,
|
||||
ops::Range,
|
||||
pin::Pin,
|
||||
task::{Context, Poll},
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
use bitvec::{bitvec, vec::BitVec};
|
||||
use futures::FutureExt;
|
||||
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, GroupIndex, Hash, SessionIndex};
|
||||
use polkadot_node_network_protocol::PeerId;
|
||||
use polkadot_primitives::{AuthorityDiscoveryId, CandidateHash, GroupIndex, SessionIndex};
|
||||
|
||||
/// The ring buffer stores at most this many unique validator groups.
|
||||
///
|
||||
@@ -66,9 +72,9 @@ pub struct ValidatorGroupsBuffer {
|
||||
group_infos: VecDeque<ValidatorsGroupInfo>,
|
||||
/// Continuous buffer of validators discovery keys.
|
||||
validators: VecDeque<AuthorityDiscoveryId>,
|
||||
/// Mapping from relay-parent to bit-vectors with bits for all `validators`.
|
||||
/// Mapping from candidate hashes to bit-vectors with bits for all `validators`.
|
||||
/// Invariants kept: All bit-vectors are guaranteed to have the same size.
|
||||
should_be_connected: HashMap<Hash, BitVec>,
|
||||
should_be_connected: HashMap<CandidateHash, BitVec>,
|
||||
/// Buffer capacity, limits the number of **groups** tracked.
|
||||
cap: NonZeroUsize,
|
||||
}
|
||||
@@ -107,7 +113,7 @@ impl ValidatorGroupsBuffer {
|
||||
/// of the buffer.
|
||||
pub fn note_collation_advertised(
|
||||
&mut self,
|
||||
relay_parent: Hash,
|
||||
candidate_hash: CandidateHash,
|
||||
session_index: SessionIndex,
|
||||
group_index: GroupIndex,
|
||||
validators: &[AuthorityDiscoveryId],
|
||||
@@ -121,19 +127,19 @@ impl ValidatorGroupsBuffer {
|
||||
}) {
|
||||
Some((idx, group)) => {
|
||||
let group_start_idx = self.group_lengths_iter().take(idx).sum();
|
||||
self.set_bits(relay_parent, group_start_idx..(group_start_idx + group.len));
|
||||
self.set_bits(candidate_hash, group_start_idx..(group_start_idx + group.len));
|
||||
},
|
||||
None => self.push(relay_parent, session_index, group_index, validators),
|
||||
None => self.push(candidate_hash, session_index, group_index, validators),
|
||||
}
|
||||
}
|
||||
|
||||
/// Note that a validator is no longer interested in a given relay parent.
|
||||
pub fn reset_validator_interest(
|
||||
&mut self,
|
||||
relay_parent: Hash,
|
||||
candidate_hash: CandidateHash,
|
||||
authority_id: &AuthorityDiscoveryId,
|
||||
) {
|
||||
let bits = match self.should_be_connected.get_mut(&relay_parent) {
|
||||
let bits = match self.should_be_connected.get_mut(&candidate_hash) {
|
||||
Some(bits) => bits,
|
||||
None => return,
|
||||
};
|
||||
@@ -145,17 +151,12 @@ impl ValidatorGroupsBuffer {
|
||||
}
|
||||
}
|
||||
|
||||
/// Remove relay parent from the buffer.
|
||||
/// Remove advertised candidate from the buffer.
|
||||
///
|
||||
/// The buffer will no longer track which validators are interested in a corresponding
|
||||
/// advertisement.
|
||||
pub fn remove_relay_parent(&mut self, relay_parent: &Hash) {
|
||||
self.should_be_connected.remove(relay_parent);
|
||||
}
|
||||
|
||||
/// Removes all advertisements from the buffer.
|
||||
pub fn clear_advertisements(&mut self) {
|
||||
self.should_be_connected.clear();
|
||||
pub fn remove_candidate(&mut self, candidate_hash: &CandidateHash) {
|
||||
self.should_be_connected.remove(candidate_hash);
|
||||
}
|
||||
|
||||
/// Pushes a new group to the buffer along with advertisement, setting all validators
|
||||
@@ -164,7 +165,7 @@ impl ValidatorGroupsBuffer {
|
||||
/// If the buffer is full, drops group from the tail.
|
||||
fn push(
|
||||
&mut self,
|
||||
relay_parent: Hash,
|
||||
candidate_hash: CandidateHash,
|
||||
session_index: SessionIndex,
|
||||
group_index: GroupIndex,
|
||||
validators: &[AuthorityDiscoveryId],
|
||||
@@ -193,17 +194,17 @@ impl ValidatorGroupsBuffer {
|
||||
self.should_be_connected
|
||||
.values_mut()
|
||||
.for_each(|bits| bits.resize(new_len, false));
|
||||
self.set_bits(relay_parent, group_start_idx..(group_start_idx + validators.len()));
|
||||
self.set_bits(candidate_hash, group_start_idx..(group_start_idx + validators.len()));
|
||||
}
|
||||
|
||||
/// Sets advertisement bits to 1 in a given range (usually corresponding to some group).
|
||||
/// If the relay parent is unknown, inserts 0-initialized bitvec first.
|
||||
///
|
||||
/// The range must be ensured to be within bounds.
|
||||
fn set_bits(&mut self, relay_parent: Hash, range: Range<usize>) {
|
||||
fn set_bits(&mut self, candidate_hash: CandidateHash, range: Range<usize>) {
|
||||
let bits = self
|
||||
.should_be_connected
|
||||
.entry(relay_parent)
|
||||
.entry(candidate_hash)
|
||||
.or_insert_with(|| bitvec![0; self.validators.len()]);
|
||||
|
||||
bits[range].fill(true);
|
||||
@@ -217,9 +218,40 @@ impl ValidatorGroupsBuffer {
|
||||
}
|
||||
}
|
||||
|
||||
/// A timeout for resetting validators' interests in collations.
|
||||
pub const RESET_INTEREST_TIMEOUT: Duration = Duration::from_secs(6);
|
||||
|
||||
/// A future that returns a candidate hash along with validator discovery
|
||||
/// keys once a timeout hit.
|
||||
///
|
||||
/// If a validator doesn't manage to fetch a collation within this timeout
|
||||
/// we should reset its interest in this advertisement in a buffer. For example,
|
||||
/// when the PoV was already requested from another peer.
|
||||
pub struct ResetInterestTimeout {
|
||||
fut: futures_timer::Delay,
|
||||
candidate_hash: CandidateHash,
|
||||
peer_id: PeerId,
|
||||
}
|
||||
|
||||
impl ResetInterestTimeout {
|
||||
/// Returns new `ResetInterestTimeout` that resolves after given timeout.
|
||||
pub fn new(candidate_hash: CandidateHash, peer_id: PeerId, delay: Duration) -> Self {
|
||||
Self { fut: futures_timer::Delay::new(delay), candidate_hash, peer_id }
|
||||
}
|
||||
}
|
||||
|
||||
impl Future for ResetInterestTimeout {
|
||||
type Output = (CandidateHash, PeerId);
|
||||
|
||||
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
self.fut.poll_unpin(cx).map(|_| (self.candidate_hash, self.peer_id))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use polkadot_primitives::Hash;
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
|
||||
#[test]
|
||||
@@ -227,8 +259,8 @@ mod tests {
|
||||
let cap = NonZeroUsize::new(1).unwrap();
|
||||
let mut buf = ValidatorGroupsBuffer::with_capacity(cap);
|
||||
|
||||
let hash_a = Hash::repeat_byte(0x1);
|
||||
let hash_b = Hash::repeat_byte(0x2);
|
||||
let hash_a = CandidateHash(Hash::repeat_byte(0x1));
|
||||
let hash_b = CandidateHash(Hash::repeat_byte(0x2));
|
||||
|
||||
let validators: Vec<_> = [
|
||||
Sr25519Keyring::Alice,
|
||||
@@ -263,7 +295,7 @@ mod tests {
|
||||
let cap = NonZeroUsize::new(3).unwrap();
|
||||
let mut buf = ValidatorGroupsBuffer::with_capacity(cap);
|
||||
|
||||
let hashes: Vec<_> = (0..5).map(Hash::repeat_byte).collect();
|
||||
let hashes: Vec<_> = (0..5).map(|i| CandidateHash(Hash::repeat_byte(i))).collect();
|
||||
|
||||
let validators: Vec<_> = [
|
||||
Sr25519Keyring::Alice,
|
||||
|
||||
@@ -17,10 +17,12 @@
|
||||
|
||||
//! Error handling related code and Error/Result definitions.
|
||||
|
||||
use futures::channel::oneshot;
|
||||
|
||||
use polkadot_node_network_protocol::request_response::incoming;
|
||||
use polkadot_node_primitives::UncheckedSignedFullStatement;
|
||||
use polkadot_node_subsystem::errors::SubsystemError;
|
||||
use polkadot_node_subsystem_util::runtime;
|
||||
use polkadot_node_subsystem::{errors::SubsystemError, RuntimeApiError};
|
||||
use polkadot_node_subsystem_util::{backing_implicit_view, runtime};
|
||||
|
||||
use crate::LOG_TARGET;
|
||||
|
||||
@@ -44,10 +46,78 @@ pub enum Error {
|
||||
#[error("Error while accessing runtime information")]
|
||||
Runtime(#[from] runtime::Error),
|
||||
|
||||
#[error("Error while accessing Runtime API")]
|
||||
RuntimeApi(#[from] RuntimeApiError),
|
||||
|
||||
#[error(transparent)]
|
||||
ImplicitViewFetchError(backing_implicit_view::FetchError),
|
||||
|
||||
#[error("Response receiver for active validators request cancelled")]
|
||||
CancelledActiveValidators(oneshot::Canceled),
|
||||
|
||||
#[error("Response receiver for validator groups request cancelled")]
|
||||
CancelledValidatorGroups(oneshot::Canceled),
|
||||
|
||||
#[error("Response receiver for availability cores request cancelled")]
|
||||
CancelledAvailabilityCores(oneshot::Canceled),
|
||||
|
||||
#[error("CollationSeconded contained statement with invalid signature")]
|
||||
InvalidStatementSignature(UncheckedSignedFullStatement),
|
||||
}
|
||||
|
||||
/// An error happened on the validator side of the protocol when attempting
|
||||
/// to start seconding a candidate.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum SecondingError {
|
||||
#[error("Error while accessing Runtime API")]
|
||||
RuntimeApi(#[from] RuntimeApiError),
|
||||
|
||||
#[error("Response receiver for persisted validation data request cancelled")]
|
||||
CancelledRuntimePersistedValidationData(oneshot::Canceled),
|
||||
|
||||
#[error("Response receiver for prospective validation data request cancelled")]
|
||||
CancelledProspectiveValidationData(oneshot::Canceled),
|
||||
|
||||
#[error("Persisted validation data is not available")]
|
||||
PersistedValidationDataNotFound,
|
||||
|
||||
#[error("Persisted validation data hash doesn't match one in the candidate receipt.")]
|
||||
PersistedValidationDataMismatch,
|
||||
|
||||
#[error("Candidate hash doesn't match the advertisement")]
|
||||
CandidateHashMismatch,
|
||||
|
||||
#[error("Received duplicate collation from the peer")]
|
||||
Duplicate,
|
||||
}
|
||||
|
||||
impl SecondingError {
|
||||
/// Returns true if an error indicates that a peer is malicious.
|
||||
pub fn is_malicious(&self) -> bool {
|
||||
use SecondingError::*;
|
||||
matches!(self, PersistedValidationDataMismatch | CandidateHashMismatch | Duplicate)
|
||||
}
|
||||
}
|
||||
|
||||
/// A validator failed to request a collation due to an error.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum FetchError {
|
||||
#[error("Collation was not previously advertised")]
|
||||
NotAdvertised,
|
||||
|
||||
#[error("Peer is unknown")]
|
||||
UnknownPeer,
|
||||
|
||||
#[error("Collation was already requested")]
|
||||
AlreadyRequested,
|
||||
|
||||
#[error("Relay parent went out of view")]
|
||||
RelayParentOutOfView,
|
||||
|
||||
#[error("Peer's protocol doesn't match the advertisement")]
|
||||
ProtocolMismatch,
|
||||
}
|
||||
|
||||
/// Utility for eating top level errors and log them.
|
||||
///
|
||||
/// We basically always want to try and continue on error. This utility function is meant to
|
||||
|
||||
@@ -32,7 +32,7 @@ use polkadot_node_subsystem_util::reputation::ReputationAggregator;
|
||||
use sp_keystore::KeystorePtr;
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
request_response::{v1 as request_v1, IncomingRequestReceiver},
|
||||
request_response::{v1 as request_v1, vstaging as protocol_vstaging, IncomingRequestReceiver},
|
||||
PeerId, UnifiedReputationChange as Rep,
|
||||
};
|
||||
use polkadot_primitives::CollatorPair;
|
||||
@@ -76,12 +76,19 @@ pub enum ProtocolSide {
|
||||
metrics: validator_side::Metrics,
|
||||
},
|
||||
/// Collators operate on a parachain.
|
||||
Collator(
|
||||
PeerId,
|
||||
CollatorPair,
|
||||
IncomingRequestReceiver<request_v1::CollationFetchingRequest>,
|
||||
collator_side::Metrics,
|
||||
),
|
||||
Collator {
|
||||
/// Local peer id.
|
||||
peer_id: PeerId,
|
||||
/// Parachain collator pair.
|
||||
collator_pair: CollatorPair,
|
||||
/// Receiver for v1 collation fetching requests.
|
||||
request_receiver_v1: IncomingRequestReceiver<request_v1::CollationFetchingRequest>,
|
||||
/// Receiver for vstaging collation fetching requests.
|
||||
request_receiver_vstaging:
|
||||
IncomingRequestReceiver<protocol_vstaging::CollationFetchingRequest>,
|
||||
/// Metrics.
|
||||
metrics: collator_side::Metrics,
|
||||
},
|
||||
/// No protocol side, just disable it.
|
||||
None,
|
||||
}
|
||||
@@ -110,10 +117,22 @@ impl<Context> CollatorProtocolSubsystem {
|
||||
validator_side::run(ctx, keystore, eviction_policy, metrics)
|
||||
.map_err(|e| SubsystemError::with_origin("collator-protocol", e))
|
||||
.boxed(),
|
||||
ProtocolSide::Collator(local_peer_id, collator_pair, req_receiver, metrics) =>
|
||||
collator_side::run(ctx, local_peer_id, collator_pair, req_receiver, metrics)
|
||||
.map_err(|e| SubsystemError::with_origin("collator-protocol", e))
|
||||
.boxed(),
|
||||
ProtocolSide::Collator {
|
||||
peer_id,
|
||||
collator_pair,
|
||||
request_receiver_v1,
|
||||
request_receiver_vstaging,
|
||||
metrics,
|
||||
} => collator_side::run(
|
||||
ctx,
|
||||
peer_id,
|
||||
collator_pair,
|
||||
request_receiver_v1,
|
||||
request_receiver_vstaging,
|
||||
metrics,
|
||||
)
|
||||
.map_err(|e| SubsystemError::with_origin("collator-protocol", e))
|
||||
.boxed(),
|
||||
ProtocolSide::None => return DummySubsystem.start(ctx),
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,366 @@
|
||||
// Copyright 2017-2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Primitives for tracking collations-related data.
|
||||
//!
|
||||
//! Usually a path of collations is as follows:
|
||||
//! 1. First, collation must be advertised by collator.
|
||||
//! 2. If the advertisement was accepted, it's queued for fetch (per relay parent).
|
||||
//! 3. Once it's requested, the collation is said to be Pending.
|
||||
//! 4. Pending collation becomes Fetched once received, we send it to backing for validation.
|
||||
//! 5. If it turns to be invalid or async backing allows seconding another candidate, carry on
|
||||
//! with the next advertisement, otherwise we're done with this relay parent.
|
||||
//!
|
||||
//! ┌──────────────────────────────────────────┐
|
||||
//! └─▶Advertised ─▶ Pending ─▶ Fetched ─▶ Validated
|
||||
|
||||
use std::{collections::VecDeque, future::Future, pin::Pin, task::Poll};
|
||||
|
||||
use futures::{future::BoxFuture, FutureExt};
|
||||
use polkadot_node_network_protocol::{
|
||||
request_response::{outgoing::RequestError, v1 as request_v1, OutgoingResult},
|
||||
PeerId,
|
||||
};
|
||||
use polkadot_node_primitives::PoV;
|
||||
use polkadot_node_subsystem::jaeger;
|
||||
use polkadot_node_subsystem_util::{
|
||||
metrics::prometheus::prometheus::HistogramTimer, runtime::ProspectiveParachainsMode,
|
||||
};
|
||||
use polkadot_primitives::{
|
||||
CandidateHash, CandidateReceipt, CollatorId, Hash, Id as ParaId, PersistedValidationData,
|
||||
};
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
use crate::{error::SecondingError, LOG_TARGET};
|
||||
|
||||
/// Candidate supplied with a para head it's built on top of.
|
||||
#[derive(Debug, Copy, Clone, Hash, Eq, PartialEq)]
|
||||
pub struct ProspectiveCandidate {
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Parent head-data hash as supplied in advertisement.
|
||||
pub parent_head_data_hash: Hash,
|
||||
}
|
||||
|
||||
impl ProspectiveCandidate {
|
||||
pub fn candidate_hash(&self) -> CandidateHash {
|
||||
self.candidate_hash
|
||||
}
|
||||
}
|
||||
|
||||
/// Identifier of a fetched collation.
|
||||
#[derive(Debug, Clone, Hash, Eq, PartialEq)]
|
||||
pub struct FetchedCollation {
|
||||
/// Candidate's relay parent.
|
||||
pub relay_parent: Hash,
|
||||
/// Parachain id.
|
||||
pub para_id: ParaId,
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Id of the collator the collation was fetched from.
|
||||
pub collator_id: CollatorId,
|
||||
}
|
||||
|
||||
impl From<&CandidateReceipt<Hash>> for FetchedCollation {
|
||||
fn from(receipt: &CandidateReceipt<Hash>) -> Self {
|
||||
let descriptor = receipt.descriptor();
|
||||
Self {
|
||||
relay_parent: descriptor.relay_parent,
|
||||
para_id: descriptor.para_id,
|
||||
candidate_hash: receipt.hash(),
|
||||
collator_id: descriptor.collator.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Identifier of a collation being requested.
|
||||
#[derive(Debug, Copy, Clone, Hash, Eq, PartialEq)]
|
||||
pub struct PendingCollation {
|
||||
/// Candidate's relay parent.
|
||||
pub relay_parent: Hash,
|
||||
/// Parachain id.
|
||||
pub para_id: ParaId,
|
||||
/// Peer that advertised this collation.
|
||||
pub peer_id: PeerId,
|
||||
/// Optional candidate hash and parent head-data hash if were
|
||||
/// supplied in advertisement.
|
||||
pub prospective_candidate: Option<ProspectiveCandidate>,
|
||||
/// Hash of the candidate's commitments.
|
||||
pub commitments_hash: Option<Hash>,
|
||||
}
|
||||
|
||||
impl PendingCollation {
|
||||
pub fn new(
|
||||
relay_parent: Hash,
|
||||
para_id: ParaId,
|
||||
peer_id: &PeerId,
|
||||
prospective_candidate: Option<ProspectiveCandidate>,
|
||||
) -> Self {
|
||||
Self {
|
||||
relay_parent,
|
||||
para_id,
|
||||
peer_id: *peer_id,
|
||||
prospective_candidate,
|
||||
commitments_hash: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// vstaging advertisement that was rejected by the backing
|
||||
/// subsystem. Validator may fetch it later if its fragment
|
||||
/// membership gets recognized before relay parent goes out of view.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct BlockedAdvertisement {
|
||||
/// Peer that advertised the collation.
|
||||
pub peer_id: PeerId,
|
||||
/// Collator id.
|
||||
pub collator_id: CollatorId,
|
||||
/// The relay-parent of the candidate.
|
||||
pub candidate_relay_parent: Hash,
|
||||
/// Hash of the candidate.
|
||||
pub candidate_hash: CandidateHash,
|
||||
}
|
||||
|
||||
/// Performs a sanity check between advertised and fetched collations.
|
||||
///
|
||||
/// Since the persisted validation data is constructed using the advertised
|
||||
/// parent head data hash, the latter doesn't require an additional check.
|
||||
pub fn fetched_collation_sanity_check(
|
||||
advertised: &PendingCollation,
|
||||
fetched: &CandidateReceipt,
|
||||
persisted_validation_data: &PersistedValidationData,
|
||||
) -> Result<(), SecondingError> {
|
||||
if persisted_validation_data.hash() != fetched.descriptor().persisted_validation_data_hash {
|
||||
Err(SecondingError::PersistedValidationDataMismatch)
|
||||
} else if advertised
|
||||
.prospective_candidate
|
||||
.map_or(false, |pc| pc.candidate_hash() != fetched.hash())
|
||||
{
|
||||
Err(SecondingError::CandidateHashMismatch)
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// Identifier for a requested collation and the respective collator that advertised it.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CollationEvent {
|
||||
/// Collator id.
|
||||
pub collator_id: CollatorId,
|
||||
/// The requested collation data.
|
||||
pub pending_collation: PendingCollation,
|
||||
}
|
||||
|
||||
/// Fetched collation data.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PendingCollationFetch {
|
||||
/// Collation identifier.
|
||||
pub collation_event: CollationEvent,
|
||||
/// Candidate receipt.
|
||||
pub candidate_receipt: CandidateReceipt,
|
||||
/// Proof of validity.
|
||||
pub pov: PoV,
|
||||
}
|
||||
|
||||
/// The status of the collations in [`CollationsPerRelayParent`].
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub enum CollationStatus {
|
||||
/// We are waiting for a collation to be advertised to us.
|
||||
Waiting,
|
||||
/// We are currently fetching a collation.
|
||||
Fetching,
|
||||
/// We are waiting that a collation is being validated.
|
||||
WaitingOnValidation,
|
||||
/// We have seconded a collation.
|
||||
Seconded,
|
||||
}
|
||||
|
||||
impl Default for CollationStatus {
|
||||
fn default() -> Self {
|
||||
Self::Waiting
|
||||
}
|
||||
}
|
||||
|
||||
impl CollationStatus {
|
||||
/// Downgrades to `Waiting`, but only if `self != Seconded`.
|
||||
fn back_to_waiting(&mut self, relay_parent_mode: ProspectiveParachainsMode) {
|
||||
match self {
|
||||
Self::Seconded =>
|
||||
if relay_parent_mode.is_enabled() {
|
||||
// With async backing enabled it's allowed to
|
||||
// second more candidates.
|
||||
*self = Self::Waiting
|
||||
},
|
||||
_ => *self = Self::Waiting,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Information about collations per relay parent.
|
||||
#[derive(Default)]
|
||||
pub struct Collations {
|
||||
/// What is the current status in regards to a collation for this relay parent?
|
||||
pub status: CollationStatus,
|
||||
/// Collator we're fetching from, optionally which candidate was requested.
|
||||
///
|
||||
/// This is the currently last started fetch, which did not exceed `MAX_UNSHARED_DOWNLOAD_TIME`
|
||||
/// yet.
|
||||
pub fetching_from: Option<(CollatorId, Option<CandidateHash>)>,
|
||||
/// Collation that were advertised to us, but we did not yet fetch.
|
||||
pub waiting_queue: VecDeque<(PendingCollation, CollatorId)>,
|
||||
/// How many collations have been seconded.
|
||||
pub seconded_count: usize,
|
||||
}
|
||||
|
||||
impl Collations {
|
||||
/// Note a seconded collation for a given para.
|
||||
pub(super) fn note_seconded(&mut self) {
|
||||
self.seconded_count += 1
|
||||
}
|
||||
|
||||
/// Returns the next collation to fetch from the `waiting_queue`.
|
||||
///
|
||||
/// This will reset the status back to `Waiting` using [`CollationStatus::back_to_waiting`].
|
||||
///
|
||||
/// Returns `Some(_)` if there is any collation to fetch, the `status` is not `Seconded` and
|
||||
/// the passed in `finished_one` is the currently `waiting_collation`.
|
||||
pub(super) fn get_next_collation_to_fetch(
|
||||
&mut self,
|
||||
finished_one: &(CollatorId, Option<CandidateHash>),
|
||||
relay_parent_mode: ProspectiveParachainsMode,
|
||||
) -> Option<(PendingCollation, CollatorId)> {
|
||||
// If finished one does not match waiting_collation, then we already dequeued another fetch
|
||||
// to replace it.
|
||||
if let Some((collator_id, maybe_candidate_hash)) = self.fetching_from.as_ref() {
|
||||
// If a candidate hash was saved previously, `finished_one` must include this too.
|
||||
if collator_id != &finished_one.0 &&
|
||||
maybe_candidate_hash.map_or(true, |hash| Some(&hash) != finished_one.1.as_ref())
|
||||
{
|
||||
gum::trace!(
|
||||
target: LOG_TARGET,
|
||||
waiting_collation = ?self.fetching_from,
|
||||
?finished_one,
|
||||
"Not proceeding to the next collation - has already been done."
|
||||
);
|
||||
return None
|
||||
}
|
||||
}
|
||||
self.status.back_to_waiting(relay_parent_mode);
|
||||
|
||||
match self.status {
|
||||
// We don't need to fetch any other collation when we already have seconded one.
|
||||
CollationStatus::Seconded => None,
|
||||
CollationStatus::Waiting =>
|
||||
if !self.is_seconded_limit_reached(relay_parent_mode) {
|
||||
None
|
||||
} else {
|
||||
self.waiting_queue.pop_front()
|
||||
},
|
||||
CollationStatus::WaitingOnValidation | CollationStatus::Fetching =>
|
||||
unreachable!("We have reset the status above!"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Checks the limit of seconded candidates for a given para.
|
||||
pub(super) fn is_seconded_limit_reached(
|
||||
&self,
|
||||
relay_parent_mode: ProspectiveParachainsMode,
|
||||
) -> bool {
|
||||
let seconded_limit =
|
||||
if let ProspectiveParachainsMode::Enabled { max_candidate_depth, .. } =
|
||||
relay_parent_mode
|
||||
{
|
||||
max_candidate_depth + 1
|
||||
} else {
|
||||
1
|
||||
};
|
||||
self.seconded_count < seconded_limit
|
||||
}
|
||||
}
|
||||
|
||||
// Any error that can occur when awaiting a collation fetch response.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub(super) enum CollationFetchError {
|
||||
#[error("Future was cancelled.")]
|
||||
Cancelled,
|
||||
#[error("{0}")]
|
||||
Request(#[from] RequestError),
|
||||
}
|
||||
|
||||
/// Future that concludes when the collator has responded to our collation fetch request
|
||||
/// or the request was cancelled by the validator.
|
||||
pub(super) struct CollationFetchRequest {
|
||||
/// Info about the requested collation.
|
||||
pub pending_collation: PendingCollation,
|
||||
/// Collator id.
|
||||
pub collator_id: CollatorId,
|
||||
/// Responses from collator.
|
||||
pub from_collator: BoxFuture<'static, OutgoingResult<request_v1::CollationFetchingResponse>>,
|
||||
/// Handle used for checking if this request was cancelled.
|
||||
pub cancellation_token: CancellationToken,
|
||||
/// A jaeger span corresponding to the lifetime of the request.
|
||||
pub span: Option<jaeger::Span>,
|
||||
/// A metric histogram for the lifetime of the request
|
||||
pub _lifetime_timer: Option<HistogramTimer>,
|
||||
}
|
||||
|
||||
impl Future for CollationFetchRequest {
|
||||
type Output = (
|
||||
CollationEvent,
|
||||
std::result::Result<request_v1::CollationFetchingResponse, CollationFetchError>,
|
||||
);
|
||||
|
||||
fn poll(mut self: Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll<Self::Output> {
|
||||
// First check if this fetch request was cancelled.
|
||||
let cancelled = match std::pin::pin!(self.cancellation_token.cancelled()).poll(cx) {
|
||||
Poll::Ready(()) => true,
|
||||
Poll::Pending => false,
|
||||
};
|
||||
|
||||
if cancelled {
|
||||
self.span.as_mut().map(|s| s.add_string_tag("success", "false"));
|
||||
return Poll::Ready((
|
||||
CollationEvent {
|
||||
collator_id: self.collator_id.clone(),
|
||||
pending_collation: self.pending_collation,
|
||||
},
|
||||
Err(CollationFetchError::Cancelled),
|
||||
))
|
||||
}
|
||||
|
||||
let res = self.from_collator.poll_unpin(cx).map(|res| {
|
||||
(
|
||||
CollationEvent {
|
||||
collator_id: self.collator_id.clone(),
|
||||
pending_collation: self.pending_collation,
|
||||
},
|
||||
res.map_err(CollationFetchError::Request),
|
||||
)
|
||||
});
|
||||
|
||||
match &res {
|
||||
Poll::Ready((_, Ok(request_v1::CollationFetchingResponse::Collation(..)))) => {
|
||||
self.span.as_mut().map(|s| s.add_string_tag("success", "true"));
|
||||
},
|
||||
Poll::Ready((_, Err(_))) => {
|
||||
self.span.as_mut().map(|s| s.add_string_tag("success", "false"));
|
||||
},
|
||||
_ => {},
|
||||
};
|
||||
|
||||
res
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
// Copyright 2017-2023 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
use polkadot_node_subsystem_util::metrics::{self, prometheus};
|
||||
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Metrics(Option<MetricsInner>);
|
||||
|
||||
impl Metrics {
|
||||
pub fn on_request(&self, succeeded: std::result::Result<(), ()>) {
|
||||
if let Some(metrics) = &self.0 {
|
||||
match succeeded {
|
||||
Ok(()) => metrics.collation_requests.with_label_values(&["succeeded"]).inc(),
|
||||
Err(()) => metrics.collation_requests.with_label_values(&["failed"]).inc(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Provide a timer for `process_msg` which observes on drop.
|
||||
pub fn time_process_msg(&self) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0.as_ref().map(|metrics| metrics.process_msg.start_timer())
|
||||
}
|
||||
|
||||
/// Provide a timer for `handle_collation_request_result` which observes on drop.
|
||||
pub fn time_handle_collation_request_result(
|
||||
&self,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0
|
||||
.as_ref()
|
||||
.map(|metrics| metrics.handle_collation_request_result.start_timer())
|
||||
}
|
||||
|
||||
/// Note the current number of collator peers.
|
||||
pub fn note_collator_peer_count(&self, collator_peers: usize) {
|
||||
self.0
|
||||
.as_ref()
|
||||
.map(|metrics| metrics.collator_peer_count.set(collator_peers as u64));
|
||||
}
|
||||
|
||||
/// Provide a timer for `CollationFetchRequest` structure which observes on drop.
|
||||
pub fn time_collation_request_duration(
|
||||
&self,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0.as_ref().map(|metrics| metrics.collation_request_duration.start_timer())
|
||||
}
|
||||
|
||||
/// Provide a timer for `request_unblocked_collations` which observes on drop.
|
||||
pub fn time_request_unblocked_collations(
|
||||
&self,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0
|
||||
.as_ref()
|
||||
.map(|metrics| metrics.request_unblocked_collations.start_timer())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct MetricsInner {
|
||||
collation_requests: prometheus::CounterVec<prometheus::U64>,
|
||||
process_msg: prometheus::Histogram,
|
||||
handle_collation_request_result: prometheus::Histogram,
|
||||
collator_peer_count: prometheus::Gauge<prometheus::U64>,
|
||||
collation_request_duration: prometheus::Histogram,
|
||||
request_unblocked_collations: prometheus::Histogram,
|
||||
}
|
||||
|
||||
impl metrics::Metrics for Metrics {
|
||||
fn try_register(
|
||||
registry: &prometheus::Registry,
|
||||
) -> std::result::Result<Self, prometheus::PrometheusError> {
|
||||
let metrics = MetricsInner {
|
||||
collation_requests: prometheus::register(
|
||||
prometheus::CounterVec::new(
|
||||
prometheus::Opts::new(
|
||||
"polkadot_parachain_collation_requests_total",
|
||||
"Number of collations requested from Collators.",
|
||||
),
|
||||
&["success"],
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
process_msg: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_process_msg",
|
||||
"Time spent within `collator_protocol_validator::process_msg`",
|
||||
)
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
handle_collation_request_result: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_handle_collation_request_result",
|
||||
"Time spent within `collator_protocol_validator::handle_collation_request_result`",
|
||||
)
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
collator_peer_count: prometheus::register(
|
||||
prometheus::Gauge::new(
|
||||
"polkadot_parachain_collator_peer_count",
|
||||
"Amount of collator peers connected",
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
collation_request_duration: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_collation_request_duration",
|
||||
"Lifetime of the `CollationFetchRequest` structure",
|
||||
).buckets(vec![0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.75, 0.9, 1.0, 1.2, 1.5, 1.75]),
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
request_unblocked_collations: prometheus::register(
|
||||
prometheus::Histogram::with_opts(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_collator_protocol_validator_request_unblocked_collations",
|
||||
"Time spent within `collator_protocol_validator::request_unblocked_collations`",
|
||||
)
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
};
|
||||
|
||||
Ok(Metrics(Some(metrics)))
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
+348
-105
@@ -19,8 +19,8 @@ use assert_matches::assert_matches;
|
||||
use futures::{executor, future, Future};
|
||||
use sp_core::{crypto::Pair, Encode};
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
use sp_keystore::{testing::MemoryKeystore, Keystore};
|
||||
use std::{iter, sync::Arc, task::Poll, time::Duration};
|
||||
use sp_keystore::Keystore;
|
||||
use std::{iter, sync::Arc, time::Duration};
|
||||
|
||||
use polkadot_node_network_protocol::{
|
||||
our_view,
|
||||
@@ -28,24 +28,39 @@ use polkadot_node_network_protocol::{
|
||||
request_response::{Requests, ResponseSender},
|
||||
ObservedRole,
|
||||
};
|
||||
use polkadot_node_primitives::BlockData;
|
||||
use polkadot_node_subsystem::messages::{
|
||||
AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest,
|
||||
use polkadot_node_primitives::{BlockData, PoV};
|
||||
use polkadot_node_subsystem::{
|
||||
errors::RuntimeApiError,
|
||||
messages::{AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest},
|
||||
};
|
||||
use polkadot_node_subsystem_test_helpers as test_helpers;
|
||||
use polkadot_node_subsystem_util::{reputation::add_reputation, TimeoutExt};
|
||||
use polkadot_primitives::{
|
||||
CollatorPair, CoreState, GroupIndex, GroupRotationInfo, OccupiedCore, ScheduledCore,
|
||||
ValidatorId, ValidatorIndex,
|
||||
CandidateReceipt, CollatorPair, CoreState, GroupIndex, GroupRotationInfo, HeadData,
|
||||
OccupiedCore, PersistedValidationData, ScheduledCore, ValidatorId, ValidatorIndex,
|
||||
};
|
||||
use polkadot_primitives_test_helpers::{
|
||||
dummy_candidate_descriptor, dummy_candidate_receipt_bad_sig, dummy_hash,
|
||||
};
|
||||
|
||||
mod prospective_parachains;
|
||||
|
||||
const ACTIVITY_TIMEOUT: Duration = Duration::from_millis(500);
|
||||
const DECLARE_TIMEOUT: Duration = Duration::from_millis(25);
|
||||
const REPUTATION_CHANGE_TEST_INTERVAL: Duration = Duration::from_millis(10);
|
||||
|
||||
const ASYNC_BACKING_DISABLED_ERROR: RuntimeApiError =
|
||||
RuntimeApiError::NotSupported { runtime_api_name: "test-runtime" };
|
||||
|
||||
fn dummy_pvd() -> PersistedValidationData {
|
||||
PersistedValidationData {
|
||||
parent_head: HeadData(vec![7, 8, 9]),
|
||||
relay_parent_number: 5,
|
||||
max_pov_size: 1024,
|
||||
relay_parent_storage_root: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct TestState {
|
||||
chain_ids: Vec<ParaId>,
|
||||
@@ -120,6 +135,7 @@ type VirtualOverseer = test_helpers::TestSubsystemContextHandle<CollatorProtocol
|
||||
|
||||
struct TestHarness {
|
||||
virtual_overseer: VirtualOverseer,
|
||||
keystore: KeystorePtr,
|
||||
}
|
||||
|
||||
fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
@@ -136,17 +152,17 @@ fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
|
||||
let (context, virtual_overseer) = test_helpers::make_subsystem_context(pool.clone());
|
||||
|
||||
let keystore = MemoryKeystore::new();
|
||||
keystore
|
||||
.sr25519_generate_new(
|
||||
polkadot_primitives::PARACHAIN_KEY_TYPE_ID,
|
||||
Some(&Sr25519Keyring::Alice.to_seed()),
|
||||
)
|
||||
.unwrap();
|
||||
let keystore = Arc::new(sc_keystore::LocalKeystore::in_memory());
|
||||
Keystore::sr25519_generate_new(
|
||||
&*keystore,
|
||||
polkadot_primitives::PARACHAIN_KEY_TYPE_ID,
|
||||
Some(&Sr25519Keyring::Alice.to_seed()),
|
||||
)
|
||||
.expect("Insert key into keystore");
|
||||
|
||||
let subsystem = run_inner(
|
||||
context,
|
||||
Arc::new(keystore),
|
||||
keystore.clone(),
|
||||
crate::CollatorEvictionPolicy {
|
||||
inactive_collator: ACTIVITY_TIMEOUT,
|
||||
undeclared: DECLARE_TIMEOUT,
|
||||
@@ -156,7 +172,7 @@ fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
REPUTATION_CHANGE_TEST_INTERVAL,
|
||||
);
|
||||
|
||||
let test_fut = test(TestHarness { virtual_overseer });
|
||||
let test_fut = test(TestHarness { virtual_overseer, keystore });
|
||||
|
||||
futures::pin_mut!(test_fut);
|
||||
futures::pin_mut!(subsystem);
|
||||
@@ -253,16 +269,53 @@ async fn assert_candidate_backing_second(
|
||||
expected_relay_parent: Hash,
|
||||
expected_para_id: ParaId,
|
||||
expected_pov: &PoV,
|
||||
mode: ProspectiveParachainsMode,
|
||||
) -> CandidateReceipt {
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
// Depending on relay parent mode pvd will be either requested
|
||||
// from the Runtime API or Prospective Parachains.
|
||||
let msg = overseer_recv(virtual_overseer).await;
|
||||
match mode {
|
||||
ProspectiveParachainsMode::Disabled => assert_matches!(
|
||||
msg,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(para_id, assumption, tx),
|
||||
)) => {
|
||||
assert_eq!(expected_relay_parent, hash);
|
||||
assert_eq!(expected_para_id, para_id);
|
||||
assert_eq!(OccupiedCoreAssumption::Free, assumption);
|
||||
tx.send(Ok(Some(pvd.clone()))).unwrap();
|
||||
}
|
||||
),
|
||||
ProspectiveParachainsMode::Enabled { .. } => assert_matches!(
|
||||
msg,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetProspectiveValidationData(request, tx),
|
||||
) => {
|
||||
assert_eq!(expected_relay_parent, request.candidate_relay_parent);
|
||||
assert_eq!(expected_para_id, request.para_id);
|
||||
tx.send(Some(pvd.clone())).unwrap();
|
||||
}
|
||||
),
|
||||
}
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(CandidateBackingMessage::Second(relay_parent, candidate_receipt, incoming_pov)
|
||||
) => {
|
||||
assert_eq!(expected_relay_parent, relay_parent);
|
||||
assert_eq!(expected_para_id, candidate_receipt.descriptor.para_id);
|
||||
assert_eq!(*expected_pov, incoming_pov);
|
||||
candidate_receipt
|
||||
})
|
||||
AllMessages::CandidateBacking(CandidateBackingMessage::Second(
|
||||
relay_parent,
|
||||
candidate_receipt,
|
||||
received_pvd,
|
||||
incoming_pov,
|
||||
)) => {
|
||||
assert_eq!(expected_relay_parent, relay_parent);
|
||||
assert_eq!(expected_para_id, candidate_receipt.descriptor.para_id);
|
||||
assert_eq!(*expected_pov, incoming_pov);
|
||||
assert_eq!(pvd, received_pvd);
|
||||
candidate_receipt
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
/// Assert that a collator got disconnected.
|
||||
@@ -284,6 +337,7 @@ async fn assert_fetch_collation_request(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
relay_parent: Hash,
|
||||
para_id: ParaId,
|
||||
candidate_hash: Option<CandidateHash>,
|
||||
) -> ResponseSender {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
@@ -291,14 +345,26 @@ async fn assert_fetch_collation_request(
|
||||
) => {
|
||||
let req = reqs.into_iter().next()
|
||||
.expect("There should be exactly one request");
|
||||
match req {
|
||||
Requests::CollationFetchingV1(req) => {
|
||||
let payload = req.payload;
|
||||
assert_eq!(payload.relay_parent, relay_parent);
|
||||
assert_eq!(payload.para_id, para_id);
|
||||
req.pending_response
|
||||
}
|
||||
_ => panic!("Unexpected request"),
|
||||
match candidate_hash {
|
||||
None => assert_matches!(
|
||||
req,
|
||||
Requests::CollationFetchingV1(req) => {
|
||||
let payload = req.payload;
|
||||
assert_eq!(payload.relay_parent, relay_parent);
|
||||
assert_eq!(payload.para_id, para_id);
|
||||
req.pending_response
|
||||
}
|
||||
),
|
||||
Some(candidate_hash) => assert_matches!(
|
||||
req,
|
||||
Requests::CollationFetchingVStaging(req) => {
|
||||
let payload = req.payload;
|
||||
assert_eq!(payload.relay_parent, relay_parent);
|
||||
assert_eq!(payload.para_id, para_id);
|
||||
assert_eq!(payload.candidate_hash, candidate_hash);
|
||||
req.pending_response
|
||||
}
|
||||
),
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -309,27 +375,38 @@ async fn connect_and_declare_collator(
|
||||
peer: PeerId,
|
||||
collator: CollatorPair,
|
||||
para_id: ParaId,
|
||||
version: CollationVersion,
|
||||
) {
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Full,
|
||||
CollationVersion::V1.into(),
|
||||
version.into(),
|
||||
None,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
Versioned::V1(protocol_v1::CollatorProtocolMessage::Declare(
|
||||
let wire_message = match version {
|
||||
CollationVersion::V1 => Versioned::V1(protocol_v1::CollatorProtocolMessage::Declare(
|
||||
collator.public(),
|
||||
para_id,
|
||||
collator.sign(&protocol_v1::declare_signature_payload(&peer)),
|
||||
)),
|
||||
CollationVersion::VStaging =>
|
||||
Versioned::VStaging(protocol_vstaging::CollatorProtocolMessage::Declare(
|
||||
collator.public(),
|
||||
para_id,
|
||||
collator.sign(&protocol_v1::declare_signature_payload(&peer)),
|
||||
)),
|
||||
};
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
wire_message,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
@@ -340,24 +417,48 @@ async fn advertise_collation(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
relay_parent: Hash,
|
||||
candidate: Option<(CandidateHash, Hash)>, // Candidate hash + parent head data hash.
|
||||
) {
|
||||
let wire_message = match candidate {
|
||||
Some((candidate_hash, parent_head_data_hash)) =>
|
||||
Versioned::VStaging(protocol_vstaging::CollatorProtocolMessage::AdvertiseCollation {
|
||||
relay_parent,
|
||||
candidate_hash,
|
||||
parent_head_data_hash,
|
||||
}),
|
||||
None =>
|
||||
Versioned::V1(protocol_v1::CollatorProtocolMessage::AdvertiseCollation(relay_parent)),
|
||||
};
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::PeerMessage(
|
||||
peer,
|
||||
Versioned::V1(protocol_v1::CollatorProtocolMessage::AdvertiseCollation(relay_parent)),
|
||||
wire_message,
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn assert_async_backing_params_request(virtual_overseer: &mut VirtualOverseer, hash: Hash) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
relay_parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx)
|
||||
)) => {
|
||||
assert_eq!(relay_parent, hash);
|
||||
tx.send(Err(ASYNC_BACKING_DISABLED_ERROR)).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
// As we receive a relevant advertisement act on it and issue a collation request.
|
||||
#[test]
|
||||
fn act_on_advertisement() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
gum::trace!("activating");
|
||||
@@ -370,6 +471,7 @@ fn act_on_advertisement() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -379,15 +481,74 @@ fn act_on_advertisement() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
/// Tests that validator side works with vstaging network protocol
|
||||
/// before async backing is enabled.
|
||||
#[test]
|
||||
fn act_on_advertisement_vstaging() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
gum::trace!("activating");
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent],
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
// vstaging advertisement.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
test_state.relay_parent,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -401,7 +562,7 @@ fn collator_reporting_works() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
@@ -411,6 +572,8 @@ fn collator_reporting_works() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -421,6 +584,7 @@ fn collator_reporting_works() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -429,6 +593,7 @@ fn collator_reporting_works() {
|
||||
peer_c,
|
||||
test_state.collators[1].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -458,7 +623,7 @@ fn collator_authentication_verification_works() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
@@ -509,20 +674,24 @@ fn fetch_one_collation_at_a_time() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
let second = Hash::random();
|
||||
|
||||
let our_view = our_view![test_state.relay_parent, second];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent, second],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
// Iter over view since the order may change due to sorted invariant.
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
@@ -532,6 +701,7 @@ fn fetch_one_collation_at_a_time() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -540,16 +710,18 @@ fn fetch_one_collation_at_a_time() {
|
||||
peer_c,
|
||||
test_state.collators[1].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent, None).await;
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -563,10 +735,13 @@ fn fetch_one_collation_at_a_time() {
|
||||
dummy_candidate_receipt_bad_sig(dummy_hash(), Some(Default::default()));
|
||||
candidate_a.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate_a.descriptor.relay_parent = test_state.relay_parent;
|
||||
candidate_a.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
response_channel
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
assert_candidate_backing_second(
|
||||
@@ -574,6 +749,7 @@ fn fetch_one_collation_at_a_time() {
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Disabled,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -581,7 +757,7 @@ fn fetch_one_collation_at_a_time() {
|
||||
test_helpers::Yield::new().await;
|
||||
|
||||
// Second collation is not requested since there's already seconded one.
|
||||
assert_matches!(futures::poll!(virtual_overseer.recv().boxed()), Poll::Pending);
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
virtual_overseer
|
||||
})
|
||||
@@ -594,20 +770,24 @@ fn fetches_next_collation() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let second = Hash::random();
|
||||
|
||||
let our_view = our_view![test_state.relay_parent, second];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent, second],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
@@ -618,6 +798,7 @@ fn fetches_next_collation() {
|
||||
peer_b,
|
||||
test_state.collators[2].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -626,6 +807,7 @@ fn fetches_next_collation() {
|
||||
peer_c,
|
||||
test_state.collators[3].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -634,45 +816,64 @@ fn fetches_next_collation() {
|
||||
peer_d,
|
||||
test_state.collators[4].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, second).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, second).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_d, second).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, second, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, second, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_d, second, None).await;
|
||||
|
||||
// Dropping the response channel should lead to fetching the second collation.
|
||||
assert_fetch_collation_request(&mut virtual_overseer, second, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
let response_channel_non_exclusive =
|
||||
assert_fetch_collation_request(&mut virtual_overseer, second, test_state.chain_ids[0])
|
||||
.await;
|
||||
let response_channel_non_exclusive = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
// Third collator should receive response after that timeout:
|
||||
Delay::new(MAX_UNSHARED_DOWNLOAD_TIME + Duration::from_millis(50)).await;
|
||||
|
||||
let response_channel =
|
||||
assert_fetch_collation_request(&mut virtual_overseer, second, test_state.chain_ids[0])
|
||||
.await;
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1]) };
|
||||
let mut candidate_a =
|
||||
dummy_candidate_receipt_bad_sig(dummy_hash(), Some(Default::default()));
|
||||
candidate_a.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate_a.descriptor.relay_parent = second;
|
||||
candidate_a.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
|
||||
// First request finishes now:
|
||||
response_channel_non_exclusive
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
response_channel
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
assert_candidate_backing_second(
|
||||
@@ -680,6 +881,7 @@ fn fetches_next_collation() {
|
||||
second,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Disabled,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -692,7 +894,7 @@ fn reject_connection_to_next_group() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
@@ -702,6 +904,7 @@ fn reject_connection_to_next_group() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -711,6 +914,7 @@ fn reject_connection_to_next_group() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[1], // next, not current `para_id`
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -737,20 +941,24 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let second = Hash::random();
|
||||
|
||||
let our_view = our_view![test_state.relay_parent, second];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![test_state.relay_parent, second],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
let peer_c = PeerId::random();
|
||||
@@ -760,6 +968,7 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
peer_b,
|
||||
test_state.collators[0].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -768,16 +977,18 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
peer_c,
|
||||
test_state.collators[1].clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_c, test_state.relay_parent, None).await;
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -786,10 +997,13 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
dummy_candidate_receipt_bad_sig(dummy_hash(), Some(Default::default()));
|
||||
candidate_a.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate_a.descriptor.relay_parent = test_state.relay_parent;
|
||||
candidate_a.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
response_channel
|
||||
.send(Ok(
|
||||
CollationFetchingResponse::Collation(candidate_a.clone(), pov.clone()).encode()
|
||||
))
|
||||
.send(Ok(request_v1::CollationFetchingResponse::Collation(
|
||||
candidate_a.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
let receipt = assert_candidate_backing_second(
|
||||
@@ -797,6 +1011,7 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Disabled,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -822,6 +1037,7 @@ fn fetch_next_collation_on_invalid_collation() {
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -834,7 +1050,7 @@ fn inactive_disconnected() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -848,6 +1064,7 @@ fn inactive_disconnected() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, hash_a).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -857,14 +1074,16 @@ fn inactive_disconnected() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, test_state.relay_parent, None).await;
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
test_state.relay_parent,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -880,7 +1099,7 @@ fn activity_extends_life() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -888,18 +1107,20 @@ fn activity_extends_life() {
|
||||
let hash_b = Hash::repeat_byte(1);
|
||||
let hash_c = Hash::repeat_byte(2);
|
||||
|
||||
let our_view = our_view![hash_a, hash_b, hash_c];
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(
|
||||
our_view![hash_a, hash_b, hash_c],
|
||||
our_view.clone(),
|
||||
)),
|
||||
)
|
||||
.await;
|
||||
|
||||
// 3 heads, 3 times.
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
for hash in our_view.iter() {
|
||||
assert_async_backing_params_request(&mut virtual_overseer, *hash).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
}
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
@@ -908,29 +1129,45 @@ fn activity_extends_life() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 2 / 3).await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_a).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_a, None).await;
|
||||
|
||||
assert_fetch_collation_request(&mut virtual_overseer, hash_a, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
hash_a,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 2 / 3).await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_b).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_b, None).await;
|
||||
|
||||
assert_fetch_collation_request(&mut virtual_overseer, hash_b, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
hash_b,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 2 / 3).await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_c).await;
|
||||
advertise_collation(&mut virtual_overseer, peer_b, hash_c, None).await;
|
||||
|
||||
assert_fetch_collation_request(&mut virtual_overseer, hash_c, test_state.chain_ids[0])
|
||||
.await;
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
hash_c,
|
||||
test_state.chain_ids[0],
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
Delay::new(ACTIVITY_TIMEOUT * 3 / 2).await;
|
||||
|
||||
@@ -945,7 +1182,7 @@ fn disconnect_if_no_declare() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
@@ -955,6 +1192,7 @@ fn disconnect_if_no_declare() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -981,7 +1219,7 @@ fn disconnect_if_wrong_declare() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -993,6 +1231,7 @@ fn disconnect_if_wrong_declare() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -1042,7 +1281,7 @@ fn delay_reputation_change() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| false), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -1054,6 +1293,7 @@ fn delay_reputation_change() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -1127,7 +1367,7 @@ fn view_change_clears_old_collators() {
|
||||
let mut test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer } = test_harness;
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
@@ -1139,6 +1379,7 @@ fn view_change_clears_old_collators() {
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_async_backing_params_request(&mut virtual_overseer, test_state.relay_parent).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
let peer_b = PeerId::random();
|
||||
@@ -1148,6 +1389,7 @@ fn view_change_clears_old_collators() {
|
||||
peer_b,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -1162,6 +1404,7 @@ fn view_change_clears_old_collators() {
|
||||
.await;
|
||||
|
||||
test_state.group_rotation_info = test_state.group_rotation_info.bump_rotation();
|
||||
assert_async_backing_params_request(&mut virtual_overseer, hash_b).await;
|
||||
respond_to_core_info_queries(&mut virtual_overseer, &test_state).await;
|
||||
|
||||
assert_collator_disconnect(&mut virtual_overseer, peer_b).await;
|
||||
+988
@@ -0,0 +1,988 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Tests for the validator side with enabled prospective parachains.
|
||||
|
||||
use super::*;
|
||||
|
||||
use polkadot_node_subsystem::messages::ChainApiMessage;
|
||||
use polkadot_primitives::{
|
||||
vstaging as vstaging_primitives, BlockNumber, CandidateCommitments, CommittedCandidateReceipt,
|
||||
Header, SigningContext, ValidatorId,
|
||||
};
|
||||
|
||||
const ASYNC_BACKING_PARAMETERS: vstaging_primitives::AsyncBackingParams =
|
||||
vstaging_primitives::AsyncBackingParams { max_candidate_depth: 4, allowed_ancestry_len: 3 };
|
||||
|
||||
fn get_parent_hash(hash: Hash) -> Hash {
|
||||
Hash::from_low_u64_be(hash.to_low_u64_be() + 1)
|
||||
}
|
||||
|
||||
async fn assert_assign_incoming(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
hash: Hash,
|
||||
number: BlockNumber,
|
||||
next_msg: &mut Option<AllMessages>,
|
||||
) {
|
||||
let msg = match next_msg.take() {
|
||||
Some(msg) => msg,
|
||||
None => overseer_recv(virtual_overseer).await,
|
||||
};
|
||||
assert_matches!(
|
||||
msg,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::Validators(tx))
|
||||
) if parent == hash => {
|
||||
tx.send(Ok(test_state.validator_public.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::ValidatorGroups(tx))
|
||||
) if parent == hash => {
|
||||
let validator_groups = test_state.validator_groups.clone();
|
||||
let mut group_rotation_info = test_state.group_rotation_info.clone();
|
||||
group_rotation_info.now = number;
|
||||
tx.send(Ok((validator_groups, group_rotation_info))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::AvailabilityCores(tx))
|
||||
) if parent == hash => {
|
||||
tx.send(Ok(test_state.cores.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/// Handle a view update.
|
||||
async fn update_view(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
test_state: &TestState,
|
||||
new_view: Vec<(Hash, u32)>, // Hash and block number.
|
||||
activated: u8, // How many new heads does this update contain?
|
||||
) -> Option<AllMessages> {
|
||||
let new_view: HashMap<Hash, u32> = HashMap::from_iter(new_view);
|
||||
|
||||
let our_view =
|
||||
OurView::new(new_view.keys().map(|hash| (*hash, Arc::new(jaeger::Span::Disabled))), 0);
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::NetworkBridgeUpdate(NetworkBridgeEvent::OurViewChange(our_view)),
|
||||
)
|
||||
.await;
|
||||
|
||||
let mut next_overseer_message = None;
|
||||
for _ in 0..activated {
|
||||
let (leaf_hash, leaf_number) = assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
parent,
|
||||
RuntimeApiRequest::StagingAsyncBackingParams(tx),
|
||||
)) => {
|
||||
tx.send(Ok(ASYNC_BACKING_PARAMETERS)).unwrap();
|
||||
(parent, new_view.get(&parent).copied().expect("Unknown parent requested"))
|
||||
}
|
||||
);
|
||||
|
||||
assert_assign_incoming(
|
||||
virtual_overseer,
|
||||
test_state,
|
||||
leaf_hash,
|
||||
leaf_number,
|
||||
&mut next_overseer_message,
|
||||
)
|
||||
.await;
|
||||
|
||||
let min_number = leaf_number.saturating_sub(ASYNC_BACKING_PARAMETERS.allowed_ancestry_len);
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetMinimumRelayParents(parent, tx),
|
||||
) if parent == leaf_hash => {
|
||||
tx.send(test_state.chain_ids.iter().map(|para_id| (*para_id, min_number)).collect()).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let ancestry_len = leaf_number + 1 - min_number;
|
||||
let ancestry_hashes = std::iter::successors(Some(leaf_hash), |h| Some(get_parent_hash(*h)))
|
||||
.take(ancestry_len as usize);
|
||||
let ancestry_numbers = (min_number..=leaf_number).rev();
|
||||
let ancestry_iter = ancestry_hashes.clone().zip(ancestry_numbers).peekable();
|
||||
|
||||
// How many blocks were actually requested.
|
||||
let mut requested_len: usize = 0;
|
||||
{
|
||||
let mut ancestry_iter = ancestry_iter.clone();
|
||||
while let Some((hash, number)) = ancestry_iter.next() {
|
||||
// May be `None` for the last element.
|
||||
let parent_hash =
|
||||
ancestry_iter.peek().map(|(h, _)| *h).unwrap_or_else(|| get_parent_hash(hash));
|
||||
|
||||
let msg = match next_overseer_message.take() {
|
||||
Some(msg) => msg,
|
||||
None => overseer_recv(virtual_overseer).await,
|
||||
};
|
||||
|
||||
if !matches!(&msg, AllMessages::ChainApi(ChainApiMessage::BlockHeader(..))) {
|
||||
// Ancestry has already been cached for this leaf.
|
||||
next_overseer_message.replace(msg);
|
||||
break
|
||||
}
|
||||
|
||||
assert_matches!(
|
||||
msg,
|
||||
AllMessages::ChainApi(ChainApiMessage::BlockHeader(.., tx)) => {
|
||||
let header = Header {
|
||||
parent_hash,
|
||||
number,
|
||||
state_root: Hash::zero(),
|
||||
extrinsics_root: Hash::zero(),
|
||||
digest: Default::default(),
|
||||
};
|
||||
|
||||
tx.send(Ok(Some(header))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
requested_len += 1;
|
||||
}
|
||||
}
|
||||
|
||||
// Skip the leaf.
|
||||
for (hash, number) in ancestry_iter.skip(1).take(requested_len.saturating_sub(1)) {
|
||||
assert_assign_incoming(
|
||||
virtual_overseer,
|
||||
test_state,
|
||||
hash,
|
||||
number,
|
||||
&mut next_overseer_message,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
next_overseer_message
|
||||
}
|
||||
|
||||
async fn send_seconded_statement(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
keystore: KeystorePtr,
|
||||
candidate: &CommittedCandidateReceipt,
|
||||
) {
|
||||
let signing_context = SigningContext { session_index: 0, parent_hash: Hash::zero() };
|
||||
let stmt = SignedFullStatement::sign(
|
||||
&keystore,
|
||||
Statement::Seconded(candidate.clone()),
|
||||
&signing_context,
|
||||
ValidatorIndex(0),
|
||||
&ValidatorId::from(Sr25519Keyring::Alice.public()),
|
||||
)
|
||||
.ok()
|
||||
.flatten()
|
||||
.expect("should be signed");
|
||||
|
||||
overseer_send(
|
||||
virtual_overseer,
|
||||
CollatorProtocolMessage::Seconded(candidate.descriptor.relay_parent, stmt),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn assert_collation_seconded(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
relay_parent: Hash,
|
||||
peer_id: PeerId,
|
||||
) {
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::ReportPeer(
|
||||
ReportPeerMessage::Single(peer, rep)
|
||||
)) => {
|
||||
assert_eq!(peer_id, peer);
|
||||
assert_eq!(rep.value, BENEFIT_NOTIFY_GOOD.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
overseer_recv(virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendCollationMessage(
|
||||
peers,
|
||||
Versioned::VStaging(protocol_vstaging::CollationProtocol::CollatorProtocol(
|
||||
protocol_vstaging::CollatorProtocolMessage::CollationSeconded(
|
||||
_relay_parent,
|
||||
..,
|
||||
),
|
||||
)),
|
||||
)) => {
|
||||
assert_eq!(peers, vec![peer_id]);
|
||||
assert_eq!(relay_parent, _relay_parent);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn v1_advertisement_rejected() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 0;
|
||||
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
// Accept both collators from the implicit view.
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::V1,
|
||||
)
|
||||
.await;
|
||||
|
||||
advertise_collation(&mut virtual_overseer, peer_a, head_b, None).await;
|
||||
|
||||
// Not reported.
|
||||
test_helpers::Yield::new().await;
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accept_advertisements_from_implicit_view() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
let pair_b = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
let head_c = get_parent_hash(head_b);
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `d` we're assigned
|
||||
// to the first para.
|
||||
let head_d = get_parent_hash(head_c);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
// Accept both collators from the implicit view.
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair_b.clone(),
|
||||
test_state.chain_ids[1],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[1]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[1],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
// Advertise with different para.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_d, // Note different relay parent.
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_d,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn second_multiple_candidates_per_relay_parent() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, keystore } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `c` we're assigned
|
||||
// to the first para.
|
||||
let head_c = Hash::from_low_u64_be(130);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
for i in 0..(ASYNC_BACKING_PARAMETERS.max_candidate_depth + 1) {
|
||||
let mut candidate = dummy_candidate_receipt_bad_sig(head_c, Some(Default::default()));
|
||||
candidate.descriptor.para_id = test_state.chain_ids[0];
|
||||
candidate.descriptor.persisted_validation_data_hash = dummy_pvd().hash();
|
||||
let commitments = CandidateCommitments {
|
||||
head_data: HeadData(vec![i as u8]),
|
||||
horizontal_messages: Default::default(),
|
||||
upward_messages: Default::default(),
|
||||
new_validation_code: None,
|
||||
processed_downward_messages: 0,
|
||||
hrmp_watermark: 0,
|
||||
};
|
||||
candidate.commitments_hash = commitments.hash();
|
||||
|
||||
let candidate_hash = candidate.hash();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1]) };
|
||||
|
||||
response_channel
|
||||
.send(Ok(request_vstaging::CollationFetchingResponse::Collation(
|
||||
candidate.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
assert_candidate_backing_second(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
&pov,
|
||||
ProspectiveParachainsMode::Enabled {
|
||||
max_candidate_depth: ASYNC_BACKING_PARAMETERS.max_candidate_depth as _,
|
||||
allowed_ancestry_len: ASYNC_BACKING_PARAMETERS.allowed_ancestry_len as _,
|
||||
},
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate =
|
||||
CommittedCandidateReceipt { descriptor: candidate.descriptor, commitments };
|
||||
|
||||
send_seconded_statement(&mut virtual_overseer, keystore.clone(), &candidate).await;
|
||||
|
||||
assert_collation_seconded(&mut virtual_overseer, head_c, peer_a).await;
|
||||
}
|
||||
|
||||
// No more advertisements can be made for this relay parent.
|
||||
let candidate_hash = CandidateHash(Hash::repeat_byte(0xAA));
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, Hash::zero())),
|
||||
)
|
||||
.await;
|
||||
|
||||
// Reported because reached the limit of advertisements per relay parent.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer_id, rep)),
|
||||
) => {
|
||||
assert_eq!(peer_a, peer_id);
|
||||
assert_eq!(rep.value, COST_UNEXPECTED_MESSAGE.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
|
||||
// By different peer too (not reported).
|
||||
let pair_b = CollatorPair::generate().0;
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair_b.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash(Hash::repeat_byte(0xFF));
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
head_c,
|
||||
Some((candidate_hash, Hash::zero())),
|
||||
)
|
||||
.await;
|
||||
|
||||
test_helpers::Yield::new().await;
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fetched_collation_sanity_check() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair = CollatorPair::generate().0;
|
||||
|
||||
// Grandparent of head `a`.
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `c` we're assigned
|
||||
// to the first para.
|
||||
let head_c = Hash::from_low_u64_be(130);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let mut candidate = dummy_candidate_receipt_bad_sig(head_c, Some(Default::default()));
|
||||
candidate.descriptor.para_id = test_state.chain_ids[0];
|
||||
let commitments = CandidateCommitments {
|
||||
head_data: HeadData(vec![1, 2, 3]),
|
||||
horizontal_messages: Default::default(),
|
||||
upward_messages: Default::default(),
|
||||
new_validation_code: None,
|
||||
processed_downward_messages: 0,
|
||||
hrmp_watermark: 0,
|
||||
};
|
||||
candidate.commitments_hash = commitments.hash();
|
||||
|
||||
let candidate_hash = CandidateHash(Hash::zero());
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
let response_channel = assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
|
||||
let pov = PoV { block_data: BlockData(vec![1]) };
|
||||
|
||||
response_channel
|
||||
.send(Ok(request_vstaging::CollationFetchingResponse::Collation(
|
||||
candidate.clone(),
|
||||
pov.clone(),
|
||||
)
|
||||
.encode()))
|
||||
.expect("Sending response should succeed");
|
||||
|
||||
// PVD request.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetProspectiveValidationData(request, tx),
|
||||
) => {
|
||||
assert_eq!(head_c, request.candidate_relay_parent);
|
||||
assert_eq!(test_state.chain_ids[0], request.para_id);
|
||||
tx.send(Some(dummy_pvd())).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
// Reported malicious.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer_id, rep)),
|
||||
) => {
|
||||
assert_eq!(peer_a, peer_id);
|
||||
assert_eq!(rep.value, COST_REPORT_BAD.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn advertisement_spam_protection() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
let head_c = get_parent_hash(head_b);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[1],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[1]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
// Reject it.
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
// Send the same advertisement again.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
// Reported.
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(peer_id, rep)),
|
||||
) => {
|
||||
assert_eq!(peer_a, peer_id);
|
||||
assert_eq!(rep.value, COST_UNEXPECTED_MESSAGE.cost_or_benefit());
|
||||
}
|
||||
);
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn backed_candidate_unblocks_advertisements() {
|
||||
let test_state = TestState::default();
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let pair_a = CollatorPair::generate().0;
|
||||
let pair_b = CollatorPair::generate().0;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 2;
|
||||
|
||||
let head_c = get_parent_hash(head_b);
|
||||
// Grandparent of head `b`.
|
||||
// Group rotation frequency is 1 by default, at `d` we're assigned
|
||||
// to the first para.
|
||||
let head_d = get_parent_hash(head_c);
|
||||
|
||||
// Activated leaf is `b`, but the collation will be based on `c`.
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peer_a = PeerId::random();
|
||||
let peer_b = PeerId::random();
|
||||
|
||||
// Accept both collators from the implicit view.
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
pair_a.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
pair_b.clone(),
|
||||
test_state.chain_ids[1],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
|
||||
let candidate_hash = CandidateHash::default();
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_b,
|
||||
head_c,
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[1]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
// Reject it.
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
// Advertise with different para.
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_a,
|
||||
head_d, // Note different relay parent.
|
||||
Some((candidate_hash, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
overseer_send(
|
||||
&mut virtual_overseer,
|
||||
CollatorProtocolMessage::Backed {
|
||||
para_id: test_state.chain_ids[0],
|
||||
para_head: parent_head_data_hash,
|
||||
},
|
||||
)
|
||||
.await;
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidate_hash);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_d,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidate_hash),
|
||||
)
|
||||
.await;
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn active_leave_unblocks_advertisements() {
|
||||
let mut test_state = TestState::default();
|
||||
test_state.group_rotation_info.group_rotation_frequency = 100;
|
||||
|
||||
test_harness(ReputationAggregator::new(|_| true), |test_harness| async move {
|
||||
let TestHarness { mut virtual_overseer, .. } = test_harness;
|
||||
|
||||
let head_b = Hash::from_low_u64_be(128);
|
||||
let head_b_num: u32 = 0;
|
||||
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_b, head_b_num)], 1).await;
|
||||
|
||||
let peers: Vec<CollatorPair> = (0..3).map(|_| CollatorPair::generate().0).collect();
|
||||
let peer_ids: Vec<PeerId> = (0..3).map(|_| PeerId::random()).collect();
|
||||
let candidates: Vec<CandidateHash> =
|
||||
(0u8..3).map(|i| CandidateHash(Hash::repeat_byte(i))).collect();
|
||||
|
||||
for (peer, peer_id) in peers.iter().zip(&peer_ids) {
|
||||
connect_and_declare_collator(
|
||||
&mut virtual_overseer,
|
||||
*peer_id,
|
||||
peer.clone(),
|
||||
test_state.chain_ids[0],
|
||||
CollationVersion::VStaging,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
let parent_head_data_hash = Hash::zero();
|
||||
for (peer, candidate) in peer_ids.iter().zip(&candidates).take(2) {
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
*peer,
|
||||
head_b,
|
||||
Some((*candidate, parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, *candidate);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
// Send false.
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
let head_c = Hash::from_low_u64_be(127);
|
||||
let head_c_num: u32 = 1;
|
||||
|
||||
let next_overseer_message =
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_c, head_c_num)], 1)
|
||||
.await
|
||||
.expect("should've sent request to backing");
|
||||
|
||||
// Unblock first request.
|
||||
assert_matches!(
|
||||
next_overseer_message,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[0]);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_b,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidates[0]),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[1]);
|
||||
assert_eq!(request.candidate_para_id, test_state.chain_ids[0]);
|
||||
assert_eq!(request.parent_head_data_hash, parent_head_data_hash);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
// Collation request was discarded.
|
||||
test_helpers::Yield::new().await;
|
||||
assert_matches!(virtual_overseer.recv().now_or_never(), None);
|
||||
|
||||
advertise_collation(
|
||||
&mut virtual_overseer,
|
||||
peer_ids[2],
|
||||
head_c,
|
||||
Some((candidates[2], parent_head_data_hash)),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[2]);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
|
||||
let head_d = Hash::from_low_u64_be(126);
|
||||
let head_d_num: u32 = 2;
|
||||
|
||||
let next_overseer_message =
|
||||
update_view(&mut virtual_overseer, &test_state, vec![(head_d, head_d_num)], 1)
|
||||
.await
|
||||
.expect("should've sent request to backing");
|
||||
|
||||
// Reject 2, accept 3.
|
||||
assert_matches!(
|
||||
next_overseer_message,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[1]);
|
||||
tx.send(false).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
overseer_recv(&mut virtual_overseer).await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::CanSecond(request, tx),
|
||||
) => {
|
||||
assert_eq!(request.candidate_hash, candidates[2]);
|
||||
tx.send(true).expect("receiving side should be alive");
|
||||
}
|
||||
);
|
||||
assert_fetch_collation_request(
|
||||
&mut virtual_overseer,
|
||||
head_c,
|
||||
test_state.chain_ids[0],
|
||||
Some(candidates[2]),
|
||||
)
|
||||
.await;
|
||||
|
||||
virtual_overseer
|
||||
});
|
||||
}
|
||||
@@ -448,8 +448,12 @@ where
|
||||
NetworkBridgeEvent::OurViewChange(_) => {},
|
||||
NetworkBridgeEvent::PeerViewChange(_, _) => {},
|
||||
NetworkBridgeEvent::NewGossipTopology { .. } => {},
|
||||
NetworkBridgeEvent::PeerMessage(_, Versioned::V1(v)) => {
|
||||
match v {};
|
||||
NetworkBridgeEvent::PeerMessage(_, message) => {
|
||||
// match void -> LLVM unreachable
|
||||
match message {
|
||||
Versioned::V1(m) => match m {},
|
||||
Versioned::VStaging(m) => match m {},
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -23,6 +23,10 @@ fatality = "0.0.6"
|
||||
rand = "0.8"
|
||||
derive_more = "0.99"
|
||||
gum = { package = "tracing-gum", path = "../../gum" }
|
||||
bitvec = "1"
|
||||
|
||||
[dev-dependencies]
|
||||
rand_chacha = "0.3.1"
|
||||
|
||||
[features]
|
||||
network-protocol-staging = []
|
||||
@@ -253,22 +253,26 @@ impl View {
|
||||
|
||||
/// A protocol-versioned type.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Versioned<V1> {
|
||||
pub enum Versioned<V1, VStaging> {
|
||||
/// V1 type.
|
||||
V1(V1),
|
||||
/// VStaging type.
|
||||
VStaging(VStaging),
|
||||
}
|
||||
|
||||
impl<V1: Clone> Versioned<&'_ V1> {
|
||||
impl<V1: Clone, VStaging: Clone> Versioned<&'_ V1, &'_ VStaging> {
|
||||
/// Convert to a fully-owned version of the message.
|
||||
pub fn clone_inner(&self) -> Versioned<V1> {
|
||||
pub fn clone_inner(&self) -> Versioned<V1, VStaging> {
|
||||
match *self {
|
||||
Versioned::V1(inner) => Versioned::V1(inner.clone()),
|
||||
Versioned::VStaging(inner) => Versioned::VStaging(inner.clone()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// All supported versions of the validation protocol message.
|
||||
pub type VersionedValidationProtocol = Versioned<v1::ValidationProtocol>;
|
||||
pub type VersionedValidationProtocol =
|
||||
Versioned<v1::ValidationProtocol, vstaging::ValidationProtocol>;
|
||||
|
||||
impl From<v1::ValidationProtocol> for VersionedValidationProtocol {
|
||||
fn from(v1: v1::ValidationProtocol) -> Self {
|
||||
@@ -276,8 +280,14 @@ impl From<v1::ValidationProtocol> for VersionedValidationProtocol {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<vstaging::ValidationProtocol> for VersionedValidationProtocol {
|
||||
fn from(vstaging: vstaging::ValidationProtocol) -> Self {
|
||||
VersionedValidationProtocol::VStaging(vstaging)
|
||||
}
|
||||
}
|
||||
|
||||
/// All supported versions of the collation protocol message.
|
||||
pub type VersionedCollationProtocol = Versioned<v1::CollationProtocol>;
|
||||
pub type VersionedCollationProtocol = Versioned<v1::CollationProtocol, vstaging::CollationProtocol>;
|
||||
|
||||
impl From<v1::CollationProtocol> for VersionedCollationProtocol {
|
||||
fn from(v1: v1::CollationProtocol) -> Self {
|
||||
@@ -285,12 +295,19 @@ impl From<v1::CollationProtocol> for VersionedCollationProtocol {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<vstaging::CollationProtocol> for VersionedCollationProtocol {
|
||||
fn from(vstaging: vstaging::CollationProtocol) -> Self {
|
||||
VersionedCollationProtocol::VStaging(vstaging)
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! impl_versioned_full_protocol_from {
|
||||
($from:ty, $out:ty, $variant:ident) => {
|
||||
impl From<$from> for $out {
|
||||
fn from(versioned_from: $from) -> $out {
|
||||
match versioned_from {
|
||||
Versioned::V1(x) => Versioned::V1(x.into()),
|
||||
Versioned::VStaging(x) => Versioned::VStaging(x.into()),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -300,7 +317,12 @@ macro_rules! impl_versioned_full_protocol_from {
|
||||
/// Implement `TryFrom` for one versioned enum variant into the inner type.
|
||||
/// `$m_ty::$variant(inner) -> Ok(inner)`
|
||||
macro_rules! impl_versioned_try_from {
|
||||
($from:ty, $out:ty, $v1_pat:pat => $v1_out:expr) => {
|
||||
(
|
||||
$from:ty,
|
||||
$out:ty,
|
||||
$v1_pat:pat => $v1_out:expr,
|
||||
$vstaging_pat:pat => $vstaging_out:expr
|
||||
) => {
|
||||
impl TryFrom<$from> for $out {
|
||||
type Error = crate::WrongVariant;
|
||||
|
||||
@@ -308,6 +330,7 @@ macro_rules! impl_versioned_try_from {
|
||||
#[allow(unreachable_patterns)] // when there is only one variant
|
||||
match x {
|
||||
Versioned::V1($v1_pat) => Ok(Versioned::V1($v1_out)),
|
||||
Versioned::VStaging($vstaging_pat) => Ok(Versioned::VStaging($vstaging_out)),
|
||||
_ => Err(crate::WrongVariant),
|
||||
}
|
||||
}
|
||||
@@ -320,6 +343,8 @@ macro_rules! impl_versioned_try_from {
|
||||
#[allow(unreachable_patterns)] // when there is only one variant
|
||||
match x {
|
||||
Versioned::V1($v1_pat) => Ok(Versioned::V1($v1_out.clone())),
|
||||
Versioned::VStaging($vstaging_pat) =>
|
||||
Ok(Versioned::VStaging($vstaging_out.clone())),
|
||||
_ => Err(crate::WrongVariant),
|
||||
}
|
||||
}
|
||||
@@ -328,7 +353,8 @@ macro_rules! impl_versioned_try_from {
|
||||
}
|
||||
|
||||
/// Version-annotated messages used by the bitfield distribution subsystem.
|
||||
pub type BitfieldDistributionMessage = Versioned<v1::BitfieldDistributionMessage>;
|
||||
pub type BitfieldDistributionMessage =
|
||||
Versioned<v1::BitfieldDistributionMessage, vstaging::BitfieldDistributionMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
BitfieldDistributionMessage,
|
||||
VersionedValidationProtocol,
|
||||
@@ -337,11 +363,13 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedValidationProtocol,
|
||||
BitfieldDistributionMessage,
|
||||
v1::ValidationProtocol::BitfieldDistribution(x) => x
|
||||
v1::ValidationProtocol::BitfieldDistribution(x) => x,
|
||||
vstaging::ValidationProtocol::BitfieldDistribution(x) => x
|
||||
);
|
||||
|
||||
/// Version-annotated messages used by the statement distribution subsystem.
|
||||
pub type StatementDistributionMessage = Versioned<v1::StatementDistributionMessage>;
|
||||
pub type StatementDistributionMessage =
|
||||
Versioned<v1::StatementDistributionMessage, vstaging::StatementDistributionMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
StatementDistributionMessage,
|
||||
VersionedValidationProtocol,
|
||||
@@ -350,11 +378,13 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedValidationProtocol,
|
||||
StatementDistributionMessage,
|
||||
v1::ValidationProtocol::StatementDistribution(x) => x
|
||||
v1::ValidationProtocol::StatementDistribution(x) => x,
|
||||
vstaging::ValidationProtocol::StatementDistribution(x) => x
|
||||
);
|
||||
|
||||
/// Version-annotated messages used by the approval distribution subsystem.
|
||||
pub type ApprovalDistributionMessage = Versioned<v1::ApprovalDistributionMessage>;
|
||||
pub type ApprovalDistributionMessage =
|
||||
Versioned<v1::ApprovalDistributionMessage, vstaging::ApprovalDistributionMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
ApprovalDistributionMessage,
|
||||
VersionedValidationProtocol,
|
||||
@@ -363,11 +393,14 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedValidationProtocol,
|
||||
ApprovalDistributionMessage,
|
||||
v1::ValidationProtocol::ApprovalDistribution(x) => x
|
||||
v1::ValidationProtocol::ApprovalDistribution(x) => x,
|
||||
vstaging::ValidationProtocol::ApprovalDistribution(x) => x
|
||||
|
||||
);
|
||||
|
||||
/// Version-annotated messages used by the gossip-support subsystem (this is void).
|
||||
pub type GossipSupportNetworkMessage = Versioned<v1::GossipSupportNetworkMessage>;
|
||||
pub type GossipSupportNetworkMessage =
|
||||
Versioned<v1::GossipSupportNetworkMessage, vstaging::GossipSupportNetworkMessage>;
|
||||
// This is a void enum placeholder, so never gets sent over the wire.
|
||||
impl TryFrom<VersionedValidationProtocol> for GossipSupportNetworkMessage {
|
||||
type Error = WrongVariant;
|
||||
@@ -384,7 +417,8 @@ impl<'a> TryFrom<&'a VersionedValidationProtocol> for GossipSupportNetworkMessag
|
||||
}
|
||||
|
||||
/// Version-annotated messages used by the bitfield distribution subsystem.
|
||||
pub type CollatorProtocolMessage = Versioned<v1::CollatorProtocolMessage>;
|
||||
pub type CollatorProtocolMessage =
|
||||
Versioned<v1::CollatorProtocolMessage, vstaging::CollatorProtocolMessage>;
|
||||
impl_versioned_full_protocol_from!(
|
||||
CollatorProtocolMessage,
|
||||
VersionedCollationProtocol,
|
||||
@@ -393,7 +427,8 @@ impl_versioned_full_protocol_from!(
|
||||
impl_versioned_try_from!(
|
||||
VersionedCollationProtocol,
|
||||
CollatorProtocolMessage,
|
||||
v1::CollationProtocol::CollatorProtocol(x) => x
|
||||
v1::CollationProtocol::CollatorProtocol(x) => x,
|
||||
vstaging::CollationProtocol::CollatorProtocol(x) => x
|
||||
);
|
||||
|
||||
/// v1 notification protocol types.
|
||||
@@ -553,3 +588,256 @@ pub mod v1 {
|
||||
payload
|
||||
}
|
||||
}
|
||||
|
||||
/// vstaging network protocol types.
|
||||
pub mod vstaging {
|
||||
use bitvec::{order::Lsb0, slice::BitSlice, vec::BitVec};
|
||||
use parity_scale_codec::{Decode, Encode};
|
||||
|
||||
use polkadot_primitives::vstaging::{
|
||||
CandidateHash, CandidateIndex, CollatorId, CollatorSignature, GroupIndex, Hash,
|
||||
Id as ParaId, UncheckedSignedAvailabilityBitfield, UncheckedSignedStatement,
|
||||
};
|
||||
|
||||
use polkadot_node_primitives::{
|
||||
approval::{IndirectAssignmentCert, IndirectSignedApprovalVote},
|
||||
UncheckedSignedFullStatement,
|
||||
};
|
||||
|
||||
/// Network messages used by the bitfield distribution subsystem.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum BitfieldDistributionMessage {
|
||||
/// A signed availability bitfield for a given relay-parent hash.
|
||||
#[codec(index = 0)]
|
||||
Bitfield(Hash, UncheckedSignedAvailabilityBitfield),
|
||||
}
|
||||
|
||||
/// Bitfields indicating the statements that are known or undesired
|
||||
/// about a candidate.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub struct StatementFilter {
|
||||
/// Seconded statements. '1' is known or undesired.
|
||||
pub seconded_in_group: BitVec<u8, Lsb0>,
|
||||
/// Valid statements. '1' is known or undesired.
|
||||
pub validated_in_group: BitVec<u8, Lsb0>,
|
||||
}
|
||||
|
||||
impl StatementFilter {
|
||||
/// Create a new blank filter with the given group size.
|
||||
pub fn blank(group_size: usize) -> Self {
|
||||
StatementFilter {
|
||||
seconded_in_group: BitVec::repeat(false, group_size),
|
||||
validated_in_group: BitVec::repeat(false, group_size),
|
||||
}
|
||||
}
|
||||
|
||||
/// Create a new full filter with the given group size.
|
||||
pub fn full(group_size: usize) -> Self {
|
||||
StatementFilter {
|
||||
seconded_in_group: BitVec::repeat(true, group_size),
|
||||
validated_in_group: BitVec::repeat(true, group_size),
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the filter has a specific expected length, consistent across both
|
||||
/// bitfields.
|
||||
pub fn has_len(&self, len: usize) -> bool {
|
||||
self.seconded_in_group.len() == len && self.validated_in_group.len() == len
|
||||
}
|
||||
|
||||
/// Determine the number of backing validators in the statement filter.
|
||||
pub fn backing_validators(&self) -> usize {
|
||||
self.seconded_in_group
|
||||
.iter()
|
||||
.by_vals()
|
||||
.zip(self.validated_in_group.iter().by_vals())
|
||||
.filter(|&(s, v)| s || v) // no double-counting
|
||||
.count()
|
||||
}
|
||||
|
||||
/// Whether the statement filter has at least one seconded statement.
|
||||
pub fn has_seconded(&self) -> bool {
|
||||
self.seconded_in_group.iter().by_vals().any(|x| x)
|
||||
}
|
||||
|
||||
/// Mask out `Seconded` statements in `self` according to the provided
|
||||
/// bitvec. Bits appearing in `mask` will not appear in `self` afterwards.
|
||||
pub fn mask_seconded(&mut self, mask: &BitSlice<u8, Lsb0>) {
|
||||
for (mut x, mask) in self
|
||||
.seconded_in_group
|
||||
.iter_mut()
|
||||
.zip(mask.iter().by_vals().chain(std::iter::repeat(false)))
|
||||
{
|
||||
// (x, mask) => x
|
||||
// (true, true) => false
|
||||
// (true, false) => true
|
||||
// (false, true) => false
|
||||
// (false, false) => false
|
||||
*x = *x && !mask;
|
||||
}
|
||||
}
|
||||
|
||||
/// Mask out `Valid` statements in `self` according to the provided
|
||||
/// bitvec. Bits appearing in `mask` will not appear in `self` afterwards.
|
||||
pub fn mask_valid(&mut self, mask: &BitSlice<u8, Lsb0>) {
|
||||
for (mut x, mask) in self
|
||||
.validated_in_group
|
||||
.iter_mut()
|
||||
.zip(mask.iter().by_vals().chain(std::iter::repeat(false)))
|
||||
{
|
||||
// (x, mask) => x
|
||||
// (true, true) => false
|
||||
// (true, false) => true
|
||||
// (false, true) => false
|
||||
// (false, false) => false
|
||||
*x = *x && !mask;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A manifest of a known backed candidate, along with a description
|
||||
/// of the statements backing it.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub struct BackedCandidateManifest {
|
||||
/// The relay-parent of the candidate.
|
||||
pub relay_parent: Hash,
|
||||
/// The hash of the candidate.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// The group index backing the candidate at the relay-parent.
|
||||
pub group_index: GroupIndex,
|
||||
/// The para ID of the candidate. It is illegal for this to
|
||||
/// be a para ID which is not assigned to the group indicated
|
||||
/// in this manifest.
|
||||
pub para_id: ParaId,
|
||||
/// The head-data corresponding to the candidate.
|
||||
pub parent_head_data_hash: Hash,
|
||||
/// A statement filter which indicates which validators in the
|
||||
/// para's group at the relay-parent have validated this candidate
|
||||
/// and issued statements about it, to the advertiser's knowledge.
|
||||
///
|
||||
/// This MUST have exactly the minimum amount of bytes
|
||||
/// necessary to represent the number of validators in the assigned
|
||||
/// backing group as-of the relay-parent.
|
||||
pub statement_knowledge: StatementFilter,
|
||||
}
|
||||
|
||||
/// An acknowledgement of a backed candidate being known.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub struct BackedCandidateAcknowledgement {
|
||||
/// The hash of the candidate.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// A statement filter which indicates which validators in the
|
||||
/// para's group at the relay-parent have validated this candidate
|
||||
/// and issued statements about it, to the advertiser's knowledge.
|
||||
///
|
||||
/// This MUST have exactly the minimum amount of bytes
|
||||
/// necessary to represent the number of validators in the assigned
|
||||
/// backing group as-of the relay-parent.
|
||||
pub statement_knowledge: StatementFilter,
|
||||
}
|
||||
|
||||
/// Network messages used by the statement distribution subsystem.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum StatementDistributionMessage {
|
||||
/// A notification of a signed statement in compact form, for a given relay parent.
|
||||
#[codec(index = 0)]
|
||||
Statement(Hash, UncheckedSignedStatement),
|
||||
|
||||
/// A notification of a backed candidate being known by the
|
||||
/// sending node, for the purpose of being requested by the receiving node
|
||||
/// if needed.
|
||||
#[codec(index = 1)]
|
||||
BackedCandidateManifest(BackedCandidateManifest),
|
||||
|
||||
/// A notification of a backed candidate being known by the sending node,
|
||||
/// for the purpose of informing a receiving node which already has the candidate.
|
||||
#[codec(index = 2)]
|
||||
BackedCandidateKnown(BackedCandidateAcknowledgement),
|
||||
|
||||
/// All messages for V1 for compatibility with the statement distribution
|
||||
/// protocol, for relay-parents that don't support asynchronous backing.
|
||||
///
|
||||
/// These are illegal to send to V1 peers, and illegal to send concerning relay-parents
|
||||
/// which support asynchronous backing. This backwards compatibility should be
|
||||
/// considered immediately deprecated and can be removed once the node software
|
||||
/// is not required to support logic from before asynchronous backing anymore.
|
||||
#[codec(index = 255)]
|
||||
V1Compatibility(crate::v1::StatementDistributionMessage),
|
||||
}
|
||||
|
||||
/// Network messages used by the approval distribution subsystem.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum ApprovalDistributionMessage {
|
||||
/// Assignments for candidates in recent, unfinalized blocks.
|
||||
///
|
||||
/// Actually checking the assignment may yield a different result.
|
||||
#[codec(index = 0)]
|
||||
Assignments(Vec<(IndirectAssignmentCert, CandidateIndex)>),
|
||||
/// Approvals for candidates in some recent, unfinalized block.
|
||||
#[codec(index = 1)]
|
||||
Approvals(Vec<IndirectSignedApprovalVote>),
|
||||
}
|
||||
|
||||
/// Dummy network message type, so we will receive connect/disconnect events.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum GossipSupportNetworkMessage {}
|
||||
|
||||
/// Network messages used by the collator protocol subsystem
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq)]
|
||||
pub enum CollatorProtocolMessage {
|
||||
/// Declare the intent to advertise collations under a collator ID, attaching a
|
||||
/// signature of the `PeerId` of the node using the given collator ID key.
|
||||
#[codec(index = 0)]
|
||||
Declare(CollatorId, ParaId, CollatorSignature),
|
||||
/// Advertise a collation to a validator. Can only be sent once the peer has
|
||||
/// declared that they are a collator with given ID.
|
||||
#[codec(index = 1)]
|
||||
AdvertiseCollation {
|
||||
/// Hash of the relay parent advertised collation is based on.
|
||||
relay_parent: Hash,
|
||||
/// Candidate hash.
|
||||
candidate_hash: CandidateHash,
|
||||
/// Parachain head data hash before candidate execution.
|
||||
parent_head_data_hash: Hash,
|
||||
},
|
||||
/// A collation sent to a validator was seconded.
|
||||
#[codec(index = 4)]
|
||||
CollationSeconded(Hash, UncheckedSignedFullStatement),
|
||||
}
|
||||
|
||||
/// All network messages on the validation peer-set.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq, derive_more::From)]
|
||||
pub enum ValidationProtocol {
|
||||
/// Bitfield distribution messages
|
||||
#[codec(index = 1)]
|
||||
#[from]
|
||||
BitfieldDistribution(BitfieldDistributionMessage),
|
||||
/// Statement distribution messages
|
||||
#[codec(index = 3)]
|
||||
#[from]
|
||||
StatementDistribution(StatementDistributionMessage),
|
||||
/// Approval distribution messages
|
||||
#[codec(index = 4)]
|
||||
#[from]
|
||||
ApprovalDistribution(ApprovalDistributionMessage),
|
||||
}
|
||||
|
||||
/// All network messages on the collation peer-set.
|
||||
#[derive(Debug, Clone, Encode, Decode, PartialEq, Eq, derive_more::From)]
|
||||
pub enum CollationProtocol {
|
||||
/// Collator protocol messages
|
||||
#[codec(index = 0)]
|
||||
#[from]
|
||||
CollatorProtocol(CollatorProtocolMessage),
|
||||
}
|
||||
|
||||
/// Get the payload that should be signed and included in a `Declare` message.
|
||||
///
|
||||
/// The payload is the local peer id of the node, which serves to prove that it
|
||||
/// controls the collator key it is declaring an intention to collate under.
|
||||
pub fn declare_signature_payload(peer_id: &sc_network::PeerId) -> Vec<u8> {
|
||||
let mut payload = peer_id.to_bytes();
|
||||
payload.extend_from_slice(b"COLL");
|
||||
payload
|
||||
}
|
||||
}
|
||||
@@ -118,10 +118,17 @@ impl PeerSet {
|
||||
/// Networking layer relies on `get_main_version()` being the version
|
||||
/// of the main protocol name reported by [`PeerSetProtocolNames::get_main_name()`].
|
||||
pub fn get_main_version(self) -> ProtocolVersion {
|
||||
#[cfg(not(feature = "network-protocol-staging"))]
|
||||
match self {
|
||||
PeerSet::Validation => ValidationVersion::V1.into(),
|
||||
PeerSet::Collation => CollationVersion::V1.into(),
|
||||
}
|
||||
|
||||
#[cfg(feature = "network-protocol-staging")]
|
||||
match self {
|
||||
PeerSet::Validation => ValidationVersion::VStaging.into(),
|
||||
PeerSet::Collation => CollationVersion::VStaging.into(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Get the max notification size for this peer set.
|
||||
@@ -145,12 +152,16 @@ impl PeerSet {
|
||||
PeerSet::Validation =>
|
||||
if version == ValidationVersion::V1.into() {
|
||||
Some("validation/1")
|
||||
} else if version == ValidationVersion::VStaging.into() {
|
||||
Some("validation/2")
|
||||
} else {
|
||||
None
|
||||
},
|
||||
PeerSet::Collation =>
|
||||
if version == CollationVersion::V1.into() {
|
||||
Some("collation/1")
|
||||
} else if version == CollationVersion::VStaging.into() {
|
||||
Some("collation/2")
|
||||
} else {
|
||||
None
|
||||
},
|
||||
@@ -212,6 +223,8 @@ impl From<ProtocolVersion> for u32 {
|
||||
pub enum ValidationVersion {
|
||||
/// The first version.
|
||||
V1 = 1,
|
||||
/// The staging version.
|
||||
VStaging = 2,
|
||||
}
|
||||
|
||||
/// Supported collation protocol versions. Only versions defined here must be used in the codebase.
|
||||
@@ -219,6 +232,40 @@ pub enum ValidationVersion {
|
||||
pub enum CollationVersion {
|
||||
/// The first version.
|
||||
V1 = 1,
|
||||
/// The staging version.
|
||||
VStaging = 2,
|
||||
}
|
||||
|
||||
/// Marker indicating the version is unknown.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct UnknownVersion;
|
||||
|
||||
impl TryFrom<ProtocolVersion> for ValidationVersion {
|
||||
type Error = UnknownVersion;
|
||||
|
||||
fn try_from(p: ProtocolVersion) -> Result<Self, UnknownVersion> {
|
||||
for v in Self::iter() {
|
||||
if v as u32 == p.0 {
|
||||
return Ok(v)
|
||||
}
|
||||
}
|
||||
|
||||
Err(UnknownVersion)
|
||||
}
|
||||
}
|
||||
|
||||
impl TryFrom<ProtocolVersion> for CollationVersion {
|
||||
type Error = UnknownVersion;
|
||||
|
||||
fn try_from(p: ProtocolVersion) -> Result<Self, UnknownVersion> {
|
||||
for v in Self::iter() {
|
||||
if v as u32 == p.0 {
|
||||
return Ok(v)
|
||||
}
|
||||
}
|
||||
|
||||
Err(UnknownVersion)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<ValidationVersion> for ProtocolVersion {
|
||||
|
||||
@@ -51,9 +51,12 @@ pub use outgoing::{OutgoingRequest, OutgoingResult, Recipient, Requests, Respons
|
||||
///// Multiplexer for incoming requests.
|
||||
// pub mod multiplexer;
|
||||
|
||||
/// Actual versioned requests and responses, that are sent over the wire.
|
||||
/// Actual versioned requests and responses that are sent over the wire.
|
||||
pub mod v1;
|
||||
|
||||
/// Actual versioned requests and responses that are sent over the wire.
|
||||
pub mod vstaging;
|
||||
|
||||
/// A protocol per subsystem seems to make the most sense, this way we don't need any dispatching
|
||||
/// within protocols.
|
||||
#[derive(Copy, Clone, Debug, Hash, PartialEq, Eq, EnumIter)]
|
||||
@@ -62,6 +65,8 @@ pub enum Protocol {
|
||||
ChunkFetchingV1,
|
||||
/// Protocol for fetching collations from collators.
|
||||
CollationFetchingV1,
|
||||
/// Protocol for fetching collations from collators when async backing is enabled.
|
||||
CollationFetchingVStaging,
|
||||
/// Protocol for fetching seconded PoVs from validators of the same group.
|
||||
PoVFetchingV1,
|
||||
/// Protocol for fetching available data.
|
||||
@@ -70,6 +75,10 @@ pub enum Protocol {
|
||||
StatementFetchingV1,
|
||||
/// Sending of dispute statements with application level confirmations.
|
||||
DisputeSendingV1,
|
||||
|
||||
/// Protocol for requesting candidates with attestations in statement distribution
|
||||
/// when async backing is enabled.
|
||||
AttestedCandidateVStaging,
|
||||
}
|
||||
|
||||
/// Minimum bandwidth we expect for validators - 500Mbit/s is the recommendation, so approximately
|
||||
@@ -101,12 +110,30 @@ const POV_REQUEST_TIMEOUT_CONNECTED: Duration = Duration::from_millis(1200);
|
||||
/// fit statement distribution within a block of 6 seconds.)
|
||||
const STATEMENTS_TIMEOUT: Duration = Duration::from_secs(1);
|
||||
|
||||
/// We want attested candidate requests to time out relatively fast,
|
||||
/// because slow requests will bottleneck the backing system. Ideally, we'd have
|
||||
/// an adaptive timeout based on the candidate size, because there will be a lot of variance
|
||||
/// in candidate sizes: candidates with no code and no messages vs candidates with code
|
||||
/// and messages.
|
||||
///
|
||||
/// We supply leniency because there are often large candidates and asynchronous
|
||||
/// backing allows them to be included over a longer window of time. Exponential back-off
|
||||
/// up to a maximum of 10 seconds would be ideal, but isn't supported by the
|
||||
/// infrastructure here yet: see https://github.com/paritytech/polkadot/issues/6009
|
||||
const ATTESTED_CANDIDATE_TIMEOUT: Duration = Duration::from_millis(2500);
|
||||
|
||||
/// We don't want a slow peer to slow down all the others, at the same time we want to get out the
|
||||
/// data quickly in full to at least some peers (as this will reduce load on us as they then can
|
||||
/// start serving the data). So this value is a trade-off. 3 seems to be sensible. So we would need
|
||||
/// to have 3 slow nodes connected, to delay transfer for others by `STATEMENTS_TIMEOUT`.
|
||||
pub const MAX_PARALLEL_STATEMENT_REQUESTS: u32 = 3;
|
||||
|
||||
/// We don't want a slow peer to slow down all the others, at the same time we want to get out the
|
||||
/// data quickly in full to at least some peers (as this will reduce load on us as they then can
|
||||
/// start serving the data). So this value is a tradeoff. 5 seems to be sensible. So we would need
|
||||
/// to have 5 slow nodes connected, to delay transfer for others by `ATTESTED_CANDIDATE_TIMEOUT`.
|
||||
pub const MAX_PARALLEL_ATTESTED_CANDIDATE_REQUESTS: u32 = 5;
|
||||
|
||||
/// Response size limit for responses of POV like data.
|
||||
///
|
||||
/// This is larger than `MAX_POV_SIZE` to account for protocol overhead and for additional data in
|
||||
@@ -120,6 +147,12 @@ const POV_RESPONSE_SIZE: u64 = MAX_POV_SIZE as u64 + 10_000;
|
||||
/// This is `MAX_CODE_SIZE` plus some additional space for protocol overhead.
|
||||
const STATEMENT_RESPONSE_SIZE: u64 = MAX_CODE_SIZE as u64 + 10_000;
|
||||
|
||||
/// Maximum response sizes for `AttestedCandidateVStaging`.
|
||||
///
|
||||
/// This is `MAX_CODE_SIZE` plus some additional space for protocol overhead and
|
||||
/// additional backing statements.
|
||||
const ATTESTED_CANDIDATE_RESPONSE_SIZE: u64 = MAX_CODE_SIZE as u64 + 100_000;
|
||||
|
||||
/// We can have relative large timeouts here, there is no value of hitting a
|
||||
/// timeout as we want to get statements through to each node in any case.
|
||||
pub const DISPUTE_REQUEST_TIMEOUT: Duration = Duration::from_secs(12);
|
||||
@@ -166,15 +199,16 @@ impl Protocol {
|
||||
request_timeout: CHUNK_REQUEST_TIMEOUT,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::CollationFetchingV1 => RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
max_request_size: 1_000,
|
||||
max_response_size: POV_RESPONSE_SIZE,
|
||||
// Taken from initial implementation in collator protocol:
|
||||
request_timeout: POV_REQUEST_TIMEOUT_CONNECTED,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::CollationFetchingV1 | Protocol::CollationFetchingVStaging =>
|
||||
RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
max_request_size: 1_000,
|
||||
max_response_size: POV_RESPONSE_SIZE,
|
||||
// Taken from initial implementation in collator protocol:
|
||||
request_timeout: POV_REQUEST_TIMEOUT_CONNECTED,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::PoVFetchingV1 => RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
@@ -220,6 +254,14 @@ impl Protocol {
|
||||
request_timeout: DISPUTE_REQUEST_TIMEOUT,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
Protocol::AttestedCandidateVStaging => RequestResponseConfig {
|
||||
name,
|
||||
fallback_names,
|
||||
max_request_size: 1_000,
|
||||
max_response_size: ATTESTED_CANDIDATE_RESPONSE_SIZE,
|
||||
request_timeout: ATTESTED_CANDIDATE_TIMEOUT,
|
||||
inbound_queue: tx,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -233,7 +275,7 @@ impl Protocol {
|
||||
// as well.
|
||||
Protocol::ChunkFetchingV1 => 100,
|
||||
// 10 seems reasonable, considering group sizes of max 10 validators.
|
||||
Protocol::CollationFetchingV1 => 10,
|
||||
Protocol::CollationFetchingV1 | Protocol::CollationFetchingVStaging => 10,
|
||||
// 10 seems reasonable, considering group sizes of max 10 validators.
|
||||
Protocol::PoVFetchingV1 => 10,
|
||||
// Validators are constantly self-selecting to request available data which may lead
|
||||
@@ -264,23 +306,46 @@ impl Protocol {
|
||||
// average, so something in the ballpark of 100 should be fine. Nodes will retry on
|
||||
// failure, so having a good value here is mostly about performance tuning.
|
||||
Protocol::DisputeSendingV1 => 100,
|
||||
|
||||
Protocol::AttestedCandidateVStaging => {
|
||||
// We assume we can utilize up to 70% of the available bandwidth for statements.
|
||||
// This is just a guess/estimate, with the following considerations: If we are
|
||||
// faster than that, queue size will stay low anyway, even if not - requesters will
|
||||
// get an immediate error, but if we are slower, requesters will run in a timeout -
|
||||
// wasting precious time.
|
||||
let available_bandwidth = 7 * MIN_BANDWIDTH_BYTES / 10;
|
||||
let size = u64::saturating_sub(
|
||||
ATTESTED_CANDIDATE_TIMEOUT.as_millis() as u64 * available_bandwidth /
|
||||
(1000 * MAX_CODE_SIZE as u64),
|
||||
MAX_PARALLEL_ATTESTED_CANDIDATE_REQUESTS as u64,
|
||||
);
|
||||
debug_assert!(
|
||||
size > 0,
|
||||
"We should have a channel size greater zero, otherwise we won't accept any requests."
|
||||
);
|
||||
size as usize
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// Fallback protocol names of this protocol, as understood by substrate networking.
|
||||
fn get_fallback_names(self) -> Vec<ProtocolName> {
|
||||
std::iter::once(self.get_legacy_name().into()).collect()
|
||||
self.get_legacy_name().into_iter().map(Into::into).collect()
|
||||
}
|
||||
|
||||
/// Legacy protocol name associated with each peer set.
|
||||
const fn get_legacy_name(self) -> &'static str {
|
||||
/// Legacy protocol name associated with each peer set, if any.
|
||||
const fn get_legacy_name(self) -> Option<&'static str> {
|
||||
match self {
|
||||
Protocol::ChunkFetchingV1 => "/polkadot/req_chunk/1",
|
||||
Protocol::CollationFetchingV1 => "/polkadot/req_collation/1",
|
||||
Protocol::PoVFetchingV1 => "/polkadot/req_pov/1",
|
||||
Protocol::AvailableDataFetchingV1 => "/polkadot/req_available_data/1",
|
||||
Protocol::StatementFetchingV1 => "/polkadot/req_statement/1",
|
||||
Protocol::DisputeSendingV1 => "/polkadot/send_dispute/1",
|
||||
Protocol::ChunkFetchingV1 => Some("/polkadot/req_chunk/1"),
|
||||
Protocol::CollationFetchingV1 => Some("/polkadot/req_collation/1"),
|
||||
Protocol::PoVFetchingV1 => Some("/polkadot/req_pov/1"),
|
||||
Protocol::AvailableDataFetchingV1 => Some("/polkadot/req_available_data/1"),
|
||||
Protocol::StatementFetchingV1 => Some("/polkadot/req_statement/1"),
|
||||
Protocol::DisputeSendingV1 => Some("/polkadot/send_dispute/1"),
|
||||
|
||||
// Introduced after legacy names became legacy.
|
||||
Protocol::AttestedCandidateVStaging => None,
|
||||
Protocol::CollationFetchingVStaging => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -336,6 +401,9 @@ impl ReqProtocolNames {
|
||||
Protocol::AvailableDataFetchingV1 => "/req_available_data/1",
|
||||
Protocol::StatementFetchingV1 => "/req_statement/1",
|
||||
Protocol::DisputeSendingV1 => "/send_dispute/1",
|
||||
|
||||
Protocol::CollationFetchingVStaging => "/req_collation/2",
|
||||
Protocol::AttestedCandidateVStaging => "/req_attested_candidate/2",
|
||||
};
|
||||
|
||||
format!("{}{}", prefix, short_name).into()
|
||||
|
||||
@@ -23,7 +23,7 @@ use sc_network::PeerId;
|
||||
|
||||
use polkadot_primitives::AuthorityDiscoveryId;
|
||||
|
||||
use super::{v1, IsRequest, Protocol};
|
||||
use super::{v1, vstaging, IsRequest, Protocol};
|
||||
|
||||
/// All requests that can be sent to the network bridge via `NetworkBridgeTxMessage::SendRequest`.
|
||||
#[derive(Debug)]
|
||||
@@ -40,6 +40,12 @@ pub enum Requests {
|
||||
StatementFetchingV1(OutgoingRequest<v1::StatementFetchingRequest>),
|
||||
/// Requests for notifying about an ongoing dispute.
|
||||
DisputeSendingV1(OutgoingRequest<v1::DisputeRequest>),
|
||||
|
||||
/// Request a candidate and attestations.
|
||||
AttestedCandidateVStaging(OutgoingRequest<vstaging::AttestedCandidateRequest>),
|
||||
/// Fetch a collation from a collator which previously announced it.
|
||||
/// Compared to V1 it requires specifying which candidate is requested by its hash.
|
||||
CollationFetchingVStaging(OutgoingRequest<vstaging::CollationFetchingRequest>),
|
||||
}
|
||||
|
||||
impl Requests {
|
||||
@@ -48,10 +54,12 @@ impl Requests {
|
||||
match self {
|
||||
Self::ChunkFetchingV1(_) => Protocol::ChunkFetchingV1,
|
||||
Self::CollationFetchingV1(_) => Protocol::CollationFetchingV1,
|
||||
Self::CollationFetchingVStaging(_) => Protocol::CollationFetchingVStaging,
|
||||
Self::PoVFetchingV1(_) => Protocol::PoVFetchingV1,
|
||||
Self::AvailableDataFetchingV1(_) => Protocol::AvailableDataFetchingV1,
|
||||
Self::StatementFetchingV1(_) => Protocol::StatementFetchingV1,
|
||||
Self::DisputeSendingV1(_) => Protocol::DisputeSendingV1,
|
||||
Self::AttestedCandidateVStaging(_) => Protocol::AttestedCandidateVStaging,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,10 +74,12 @@ impl Requests {
|
||||
match self {
|
||||
Self::ChunkFetchingV1(r) => r.encode_request(),
|
||||
Self::CollationFetchingV1(r) => r.encode_request(),
|
||||
Self::CollationFetchingVStaging(r) => r.encode_request(),
|
||||
Self::PoVFetchingV1(r) => r.encode_request(),
|
||||
Self::AvailableDataFetchingV1(r) => r.encode_request(),
|
||||
Self::StatementFetchingV1(r) => r.encode_request(),
|
||||
Self::DisputeSendingV1(r) => r.encode_request(),
|
||||
Self::AttestedCandidateVStaging(r) => r.encode_request(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Requests and responses as sent over the wire for the individual protocols.
|
||||
|
||||
use parity_scale_codec::{Decode, Encode};
|
||||
|
||||
use polkadot_primitives::vstaging::{
|
||||
CandidateHash, CommittedCandidateReceipt, Hash, Id as ParaId, PersistedValidationData,
|
||||
UncheckedSignedStatement,
|
||||
};
|
||||
|
||||
use super::{IsRequest, Protocol};
|
||||
use crate::vstaging::StatementFilter;
|
||||
|
||||
/// Request a candidate with statements.
|
||||
#[derive(Debug, Clone, Encode, Decode)]
|
||||
pub struct AttestedCandidateRequest {
|
||||
/// Hash of the candidate we want to request.
|
||||
pub candidate_hash: CandidateHash,
|
||||
/// Statement filter with 'OR' semantics, indicating which validators
|
||||
/// not to send statements for.
|
||||
///
|
||||
/// The filter must have exactly the minimum size required to
|
||||
/// fit all validators from the backing group.
|
||||
///
|
||||
/// The response may not contain any statements masked out by this mask.
|
||||
pub mask: StatementFilter,
|
||||
}
|
||||
|
||||
/// Response to an `AttestedCandidateRequest`.
|
||||
#[derive(Debug, Clone, Encode, Decode)]
|
||||
pub struct AttestedCandidateResponse {
|
||||
/// The candidate receipt, with commitments.
|
||||
pub candidate_receipt: CommittedCandidateReceipt,
|
||||
/// The [`PersistedValidationData`] corresponding to the candidate.
|
||||
pub persisted_validation_data: PersistedValidationData,
|
||||
/// All known statements about the candidate, in compact form,
|
||||
/// omitting `Seconded` statements which were intended to be masked
|
||||
/// out.
|
||||
pub statements: Vec<UncheckedSignedStatement>,
|
||||
}
|
||||
|
||||
impl IsRequest for AttestedCandidateRequest {
|
||||
type Response = AttestedCandidateResponse;
|
||||
const PROTOCOL: Protocol = Protocol::AttestedCandidateVStaging;
|
||||
}
|
||||
|
||||
/// Responses as sent by collators.
|
||||
pub type CollationFetchingResponse = super::v1::CollationFetchingResponse;
|
||||
|
||||
/// Request the advertised collation at that relay-parent.
|
||||
#[derive(Debug, Clone, Encode, Decode)]
|
||||
pub struct CollationFetchingRequest {
|
||||
/// Relay parent collation is built on top of.
|
||||
pub relay_parent: Hash,
|
||||
/// The `ParaId` of the collation.
|
||||
pub para_id: ParaId,
|
||||
/// Candidate hash.
|
||||
pub candidate_hash: CandidateHash,
|
||||
}
|
||||
|
||||
impl IsRequest for CollationFetchingRequest {
|
||||
// The response is the same as for V1.
|
||||
type Response = CollationFetchingResponse;
|
||||
const PROTOCOL: Protocol = Protocol::CollationFetchingVStaging;
|
||||
}
|
||||
@@ -22,10 +22,12 @@ indexmap = "1.9.1"
|
||||
parity-scale-codec = { version = "3.6.1", default-features = false, features = ["derive"] }
|
||||
thiserror = "1.0.31"
|
||||
fatality = "0.0.6"
|
||||
bitvec = "1"
|
||||
|
||||
[dev-dependencies]
|
||||
polkadot-node-subsystem-test-helpers = { path = "../../subsystem-test-helpers" }
|
||||
async-channel = "1.8.0"
|
||||
assert_matches = "1.4.0"
|
||||
polkadot-node-subsystem-test-helpers = { path = "../../subsystem-test-helpers" }
|
||||
sp-authority-discovery = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
@@ -36,3 +38,5 @@ sc-keystore = { git = "https://github.com/paritytech/substrate", branch = "maste
|
||||
sc-network = { git = "https://github.com/paritytech/substrate", branch = "master" }
|
||||
futures-timer = "3.0.2"
|
||||
polkadot-primitives-test-helpers = { path = "../../../primitives/test-helpers" }
|
||||
rand_chacha = "0.3"
|
||||
polkadot-node-subsystem-types = { path = "../../subsystem-types" }
|
||||
@@ -18,9 +18,13 @@
|
||||
//! Error handling related code and Error/Result definitions.
|
||||
|
||||
use polkadot_node_network_protocol::PeerId;
|
||||
use polkadot_node_subsystem::SubsystemError;
|
||||
use polkadot_node_subsystem_util::runtime;
|
||||
use polkadot_primitives::{CandidateHash, Hash};
|
||||
use polkadot_node_subsystem::{RuntimeApiError, SubsystemError};
|
||||
use polkadot_node_subsystem_util::{
|
||||
backing_implicit_view::FetchError as ImplicitViewFetchError, runtime,
|
||||
};
|
||||
use polkadot_primitives::{CandidateHash, Hash, Id as ParaId};
|
||||
|
||||
use futures::channel::oneshot;
|
||||
|
||||
use crate::LOG_TARGET;
|
||||
|
||||
@@ -56,6 +60,27 @@ pub enum Error {
|
||||
#[error("Error while accessing runtime information")]
|
||||
Runtime(#[from] runtime::Error),
|
||||
|
||||
#[error("RuntimeAPISubsystem channel closed before receipt")]
|
||||
RuntimeApiUnavailable(#[source] oneshot::Canceled),
|
||||
|
||||
#[error("Fetching persisted validation data for para {0:?}, {1:?}")]
|
||||
FetchPersistedValidationData(ParaId, RuntimeApiError),
|
||||
|
||||
#[error("Fetching session index failed {0:?}")]
|
||||
FetchSessionIndex(RuntimeApiError),
|
||||
|
||||
#[error("Fetching session info failed {0:?}")]
|
||||
FetchSessionInfo(RuntimeApiError),
|
||||
|
||||
#[error("Fetching availability cores failed {0:?}")]
|
||||
FetchAvailabilityCores(RuntimeApiError),
|
||||
|
||||
#[error("Fetching validator groups failed {0:?}")]
|
||||
FetchValidatorGroups(RuntimeApiError),
|
||||
|
||||
#[error("Attempted to share statement when not a validator or not assigned")]
|
||||
InvalidShare,
|
||||
|
||||
#[error("Relay parent could not be found in active heads")]
|
||||
NoSuchHead(Hash),
|
||||
|
||||
@@ -76,6 +101,10 @@ pub enum Error {
|
||||
// Responder no longer waits for our data. (Should not happen right now.)
|
||||
#[error("Oneshot `GetData` channel closed")]
|
||||
ResponderGetDataCanceled,
|
||||
|
||||
// Failed to activate leaf due to a fetch error.
|
||||
#[error("Implicit view failure while activating leaf")]
|
||||
ActivateLeafFailure(ImplicitViewFetchError),
|
||||
}
|
||||
|
||||
/// Utility for eating top level errors and log them.
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+4
-1
@@ -32,7 +32,10 @@ use polkadot_node_subsystem::{Span, Stage};
|
||||
use polkadot_node_subsystem_util::TimeoutExt;
|
||||
use polkadot_primitives::{CandidateHash, CommittedCandidateReceipt, Hash};
|
||||
|
||||
use crate::{metrics::Metrics, COST_WRONG_HASH, LOG_TARGET};
|
||||
use crate::{
|
||||
legacy_v1::{COST_WRONG_HASH, LOG_TARGET},
|
||||
metrics::Metrics,
|
||||
};
|
||||
|
||||
// In case we failed fetching from our known peers, how long we should wait before attempting a
|
||||
// retry, even though we have not yet discovered any new peers. Or in other words how long to
|
||||
+2
-2
@@ -48,8 +48,8 @@ pub enum ResponderMessage {
|
||||
|
||||
/// A fetching task, taking care of fetching large statements via request/response.
|
||||
///
|
||||
/// A fetch task does not know about a particular `Statement` instead it just tries fetching a
|
||||
/// `CommittedCandidateReceipt` from peers, whether this can be used to re-assemble one ore
|
||||
/// A fetch task does not know about a particular `Statement`, instead it just tries fetching a
|
||||
/// `CommittedCandidateReceipt` from peers, whether this can be used to re-assemble one or
|
||||
/// many `SignedFullStatement`s needs to be verified by the caller.
|
||||
pub async fn respond(
|
||||
mut receiver: IncomingRequestReceiver<StatementFetchingRequest>,
|
||||
+213
-28
@@ -14,7 +14,11 @@
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
use super::{metrics::Metrics, *};
|
||||
#![allow(clippy::clone_on_copy)]
|
||||
|
||||
use super::*;
|
||||
use crate::{metrics::Metrics, *};
|
||||
|
||||
use assert_matches::assert_matches;
|
||||
use futures::executor;
|
||||
use futures_timer::Delay;
|
||||
@@ -26,19 +30,21 @@ use polkadot_node_network_protocol::{
|
||||
v1::{StatementFetchingRequest, StatementFetchingResponse},
|
||||
IncomingRequest, Recipient, ReqProtocolNames, Requests,
|
||||
},
|
||||
view, ObservedRole,
|
||||
view, ObservedRole, VersionedValidationProtocol,
|
||||
};
|
||||
use polkadot_node_primitives::{
|
||||
SignedFullStatementWithPVD, Statement, UncheckedSignedFullStatement,
|
||||
};
|
||||
use polkadot_node_primitives::{Statement, UncheckedSignedFullStatement};
|
||||
use polkadot_node_subsystem::{
|
||||
jaeger,
|
||||
messages::{
|
||||
network_bridge_event, AllMessages, ReportPeerMessage, RuntimeApiMessage, RuntimeApiRequest,
|
||||
},
|
||||
ActivatedLeaf, LeafStatus,
|
||||
ActivatedLeaf, LeafStatus, RuntimeApiError,
|
||||
};
|
||||
use polkadot_node_subsystem_test_helpers::mock::make_ferdie_keystore;
|
||||
use polkadot_primitives::{
|
||||
GroupIndex, Hash, Id as ParaId, IndexedVec, SessionInfo, ValidationCode, ValidatorId,
|
||||
GroupIndex, Hash, HeadData, Id as ParaId, IndexedVec, SessionInfo, ValidationCode,
|
||||
};
|
||||
use polkadot_primitives_test_helpers::{
|
||||
dummy_committed_candidate_receipt, dummy_hash, AlwaysZeroRng,
|
||||
@@ -54,6 +60,30 @@ use util::reputation::add_reputation;
|
||||
// Some deterministic genesis hash for protocol names
|
||||
const GENESIS_HASH: Hash = Hash::repeat_byte(0xff);
|
||||
|
||||
const ASYNC_BACKING_DISABLED_ERROR: RuntimeApiError =
|
||||
RuntimeApiError::NotSupported { runtime_api_name: "test-runtime" };
|
||||
|
||||
fn dummy_pvd() -> PersistedValidationData {
|
||||
PersistedValidationData {
|
||||
parent_head: HeadData(vec![7, 8, 9]),
|
||||
relay_parent_number: 5,
|
||||
max_pov_size: 1024,
|
||||
relay_parent_storage_root: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
fn extend_statement_with_pvd(
|
||||
statement: SignedFullStatement,
|
||||
pvd: PersistedValidationData,
|
||||
) -> SignedFullStatementWithPVD {
|
||||
statement
|
||||
.convert_to_superpayload_with(|statement| match statement {
|
||||
Statement::Seconded(receipt) => StatementWithPVD::Seconded(receipt, pvd),
|
||||
Statement::Valid(candidate_hash) => StatementWithPVD::Valid(candidate_hash),
|
||||
})
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn active_head_accepts_only_2_seconded_per_validator() {
|
||||
let validators = vec![
|
||||
@@ -496,6 +526,7 @@ fn peer_view_update_sends_messages() {
|
||||
|
||||
let mut peer_data = PeerData {
|
||||
view: old_view,
|
||||
protocol_version: ValidationVersion::V1,
|
||||
view_knowledge: {
|
||||
let mut k = HashMap::new();
|
||||
|
||||
@@ -554,8 +585,9 @@ fn peer_view_update_sends_messages() {
|
||||
for statement in active_head.statements_about(candidate_hash) {
|
||||
let message = handle.recv().await;
|
||||
let expected_to = vec![peer];
|
||||
let expected_payload =
|
||||
statement_message(hash_c, statement.statement.clone(), &Metrics::default());
|
||||
let expected_payload = VersionedValidationProtocol::from(Versioned::V1(
|
||||
v1_statement_message(hash_c, statement.statement.clone(), &Metrics::default()),
|
||||
));
|
||||
|
||||
assert_matches!(
|
||||
message,
|
||||
@@ -596,6 +628,7 @@ fn circulated_statement_goes_to_all_peers_with_view() {
|
||||
|
||||
let peer_data_from_view = |view: View| PeerData {
|
||||
view: view.clone(),
|
||||
protocol_version: ValidationVersion::V1,
|
||||
view_knowledge: view.iter().map(|v| (*v, Default::default())).collect(),
|
||||
maybe_authority: None,
|
||||
};
|
||||
@@ -697,7 +730,7 @@ fn circulated_statement_goes_to_all_peers_with_view() {
|
||||
|
||||
assert_eq!(
|
||||
payload,
|
||||
statement_message(hash_b, statement.statement.clone(), &Metrics::default()),
|
||||
VersionedValidationProtocol::from(Versioned::V1(v1_statement_message(hash_b, statement.statement.clone(), &Metrics::default()))),
|
||||
);
|
||||
}
|
||||
)
|
||||
@@ -706,12 +739,14 @@ fn circulated_statement_goes_to_all_peers_with_view() {
|
||||
|
||||
#[test]
|
||||
fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
const PARA_ID: ParaId = ParaId::new(1);
|
||||
let hash_a = Hash::repeat_byte(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
let candidate = {
|
||||
let mut c = dummy_committed_candidate_receipt(dummy_hash());
|
||||
c.descriptor.relay_parent = hash_a;
|
||||
c.descriptor.para_id = 1.into();
|
||||
c.descriptor.para_id = PARA_ID;
|
||||
c
|
||||
};
|
||||
|
||||
@@ -733,11 +768,13 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: Arc::new(LocalKeystore::in_memory()),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -758,6 +795,17 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -862,18 +910,32 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(para_id, assumption, tx),
|
||||
)) if para_id == PARA_ID &&
|
||||
assumption == OccupiedCoreAssumption::Free &&
|
||||
hash == hash_a =>
|
||||
{
|
||||
tx.send(Ok(Some(pvd))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
NetworkBridgeTxMessage::ReportPeer(ReportPeerMessage::Single(p, r))
|
||||
) if p == peer_a && r == BENEFIT_VALID_STATEMENT_FIRST.into() => {}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) if r == hash_a && s == statement => {}
|
||||
) if r == hash_a && s == statement_with_pvd => {}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
@@ -902,6 +964,9 @@ fn receiving_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
|
||||
#[test]
|
||||
fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing() {
|
||||
const PARA_ID: ParaId = ParaId::new(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
sp_tracing::try_init_simple();
|
||||
let hash_a = Hash::repeat_byte(1);
|
||||
let hash_b = Hash::repeat_byte(2);
|
||||
@@ -909,7 +974,7 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
let candidate = {
|
||||
let mut c = dummy_committed_candidate_receipt(dummy_hash());
|
||||
c.descriptor.relay_parent = hash_a;
|
||||
c.descriptor.para_id = 1.into();
|
||||
c.descriptor.para_id = PARA_ID;
|
||||
c.commitments.new_validation_code = Some(ValidationCode(vec![1, 2, 3]));
|
||||
c
|
||||
};
|
||||
@@ -937,11 +1002,13 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, mut req_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -962,6 +1029,17 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -1292,6 +1370,20 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
) if p == peer_c && r == BENEFIT_VALID_RESPONSE.into() => {}
|
||||
);
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(para_id, assumption, tx),
|
||||
)) if para_id == PARA_ID &&
|
||||
assumption == OccupiedCoreAssumption::Free &&
|
||||
hash == hash_a =>
|
||||
{
|
||||
tx.send(Ok(Some(pvd))).unwrap();
|
||||
}
|
||||
);
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
@@ -1303,7 +1395,7 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) if r == hash_a && s == statement => {}
|
||||
) if r == hash_a && s == statement_with_pvd => {}
|
||||
);
|
||||
|
||||
// Now messages should go out:
|
||||
@@ -1400,6 +1492,7 @@ fn receiving_large_statement_from_one_sends_to_another_and_to_candidate_backing(
|
||||
fn delay_reputation_changes() {
|
||||
sp_tracing::try_init_simple();
|
||||
let hash_a = Hash::repeat_byte(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
let candidate = {
|
||||
let mut c = dummy_committed_candidate_receipt(dummy_hash());
|
||||
@@ -1431,13 +1524,15 @@ fn delay_reputation_changes() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let reputation_interval = Duration::from_millis(100);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| false),
|
||||
@@ -1458,6 +1553,17 @@ fn delay_reputation_changes() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -1768,9 +1874,18 @@ fn delay_reputation_changes() {
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) if r == hash_a && s == statement => {}
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
hash,
|
||||
RuntimeApiRequest::PersistedValidationData(_, assumption, tx),
|
||||
)) if assumption == OccupiedCoreAssumption::Free && hash == hash_a =>
|
||||
{
|
||||
tx.send(Ok(Some(pvd))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::CandidateBacking(CandidateBackingMessage::Statement(..))
|
||||
);
|
||||
|
||||
// Now messages should go out:
|
||||
@@ -1885,11 +2000,13 @@ fn share_prioritizes_backing_group() {
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, mut req_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -1910,6 +2027,17 @@ fn share_prioritizes_backing_group() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -2069,9 +2197,17 @@ fn share_prioritizes_backing_group() {
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
SignedFullStatement::sign(
|
||||
// note: this is ignored by legacy-v1 code.
|
||||
let pvd = PersistedValidationData {
|
||||
parent_head: HeadData::from(vec![1, 2, 3]),
|
||||
relay_parent_number: 0,
|
||||
relay_parent_storage_root: Hash::repeat_byte(42),
|
||||
max_pov_size: 100,
|
||||
};
|
||||
|
||||
SignedFullStatementWithPVD::sign(
|
||||
&keystore,
|
||||
Statement::Seconded(candidate.clone()),
|
||||
Statement::Seconded(candidate.clone()).supply_pvd(pvd),
|
||||
&signing_context,
|
||||
ValidatorIndex(4),
|
||||
&ferdie_public.into(),
|
||||
@@ -2081,14 +2217,15 @@ fn share_prioritizes_backing_group() {
|
||||
.expect("should be signed")
|
||||
};
|
||||
|
||||
let metadata = derive_metadata_assuming_seconded(hash_a, statement.clone().into());
|
||||
|
||||
handle
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::Share(hash_a, statement.clone()),
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement = StatementWithPVD::drop_pvd_from_signed(statement);
|
||||
let metadata = derive_metadata_assuming_seconded(hash_a, statement.clone().into());
|
||||
|
||||
// Messages should go out:
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
@@ -2180,10 +2317,12 @@ fn peer_cant_flood_with_large_statements() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let bg = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: make_ferdie_keystore(),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -2204,6 +2343,17 @@ fn peer_cant_flood_with_large_statements() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == hash_a
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -2341,6 +2491,7 @@ fn peer_cant_flood_with_large_statements() {
|
||||
#[test]
|
||||
fn handle_multiple_seconded_statements() {
|
||||
let relay_parent_hash = Hash::repeat_byte(1);
|
||||
let pvd = dummy_pvd();
|
||||
|
||||
let candidate = dummy_committed_candidate_receipt(relay_parent_hash);
|
||||
let candidate_hash = candidate.hash();
|
||||
@@ -2384,11 +2535,13 @@ fn handle_multiple_seconded_statements() {
|
||||
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
|
||||
let virtual_overseer_fut = async move {
|
||||
let s = StatementDistributionSubsystem {
|
||||
keystore: Arc::new(LocalKeystore::in_memory()),
|
||||
req_receiver: Some(statement_req_receiver),
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng: AlwaysZeroRng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
@@ -2409,6 +2562,17 @@ fn handle_multiple_seconded_statements() {
|
||||
)))
|
||||
.await;
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(r, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
)
|
||||
if r == relay_parent_hash
|
||||
=> {
|
||||
let _ = tx.send(Err(ASYNC_BACKING_DISABLED_ERROR));
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
@@ -2575,6 +2739,18 @@ fn handle_multiple_seconded_statements() {
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::RuntimeApi(RuntimeApiMessage::Request(
|
||||
_,
|
||||
RuntimeApiRequest::PersistedValidationData(_, assumption, tx),
|
||||
)) if assumption == OccupiedCoreAssumption::Free => {
|
||||
tx.send(Ok(Some(pvd.clone()))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
@@ -2592,7 +2768,7 @@ fn handle_multiple_seconded_statements() {
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) => {
|
||||
assert_eq!(r, relay_parent_hash);
|
||||
assert_eq!(s, statement);
|
||||
assert_eq!(s, statement_with_pvd);
|
||||
}
|
||||
);
|
||||
|
||||
@@ -2676,6 +2852,10 @@ fn handle_multiple_seconded_statements() {
|
||||
})
|
||||
.await;
|
||||
|
||||
let statement_with_pvd = extend_statement_with_pvd(statement.clone(), pvd.clone());
|
||||
|
||||
// Persisted validation data is cached.
|
||||
|
||||
assert_matches!(
|
||||
handle.recv().await,
|
||||
AllMessages::NetworkBridgeTx(
|
||||
@@ -2692,7 +2872,7 @@ fn handle_multiple_seconded_statements() {
|
||||
CandidateBackingMessage::Statement(r, s)
|
||||
) => {
|
||||
assert_eq!(r, relay_parent_hash);
|
||||
assert_eq!(s, statement);
|
||||
assert_eq!(s, statement_with_pvd);
|
||||
}
|
||||
);
|
||||
|
||||
@@ -2784,3 +2964,8 @@ fn derive_metadata_assuming_seconded(
|
||||
signature: statement.unchecked_signature().clone(),
|
||||
}
|
||||
}
|
||||
|
||||
// TODO [now]: adapt most tests to v2 messages.
|
||||
// TODO [now]: test that v2 peers send v1 messages to v1 peers
|
||||
// TODO [now]: test that v2 peers handle v1 messages from v1 peers.
|
||||
// TODO [now]: test that v2 peers send v2 messages to v2 peers.
|
||||
File diff suppressed because it is too large.
Load diff
@@ -29,7 +29,7 @@ struct MetricsInner {
|
||||
received_responses: prometheus::CounterVec<prometheus::U64>,
|
||||
active_leaves_update: prometheus::Histogram,
|
||||
share: prometheus::Histogram,
|
||||
network_bridge_update_v1: prometheus::HistogramVec,
|
||||
network_bridge_update: prometheus::HistogramVec,
|
||||
statements_unexpected: prometheus::CounterVec<prometheus::U64>,
|
||||
created_message_size: prometheus::Gauge<prometheus::U64>,
|
||||
}
|
||||
@@ -77,16 +77,13 @@ impl Metrics {
|
||||
self.0.as_ref().map(|metrics| metrics.share.start_timer())
|
||||
}
|
||||
|
||||
/// Provide a timer for `network_bridge_update_v1` which observes on drop.
|
||||
pub fn time_network_bridge_update_v1(
|
||||
/// Provide a timer for `network_bridge_update` which observes on drop.
|
||||
pub fn time_network_bridge_update(
|
||||
&self,
|
||||
message_type: &'static str,
|
||||
) -> Option<metrics::prometheus::prometheus::HistogramTimer> {
|
||||
self.0.as_ref().map(|metrics| {
|
||||
metrics
|
||||
.network_bridge_update_v1
|
||||
.with_label_values(&[message_type])
|
||||
.start_timer()
|
||||
metrics.network_bridge_update.with_label_values(&[message_type]).start_timer()
|
||||
})
|
||||
}
|
||||
|
||||
@@ -168,11 +165,11 @@ impl metrics::Metrics for Metrics {
|
||||
)?,
|
||||
registry,
|
||||
)?,
|
||||
network_bridge_update_v1: prometheus::register(
|
||||
network_bridge_update: prometheus::register(
|
||||
prometheus::HistogramVec::new(
|
||||
prometheus::HistogramOpts::new(
|
||||
"polkadot_parachain_statement_distribution_network_bridge_update_v1",
|
||||
"Time spent within `statement_distribution::network_bridge_update_v1`",
|
||||
"polkadot_parachain_statement_distribution_network_bridge_update",
|
||||
"Time spent within `statement_distribution::network_bridge_update`",
|
||||
)
|
||||
.buckets(HISTOGRAM_LATENCY_BUCKETS.into()),
|
||||
&["message_type"],
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,70 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! A utility for tracking groups and their members within a session.
|
||||
|
||||
use polkadot_node_primitives::minimum_votes;
|
||||
use polkadot_primitives::vstaging::{GroupIndex, IndexedVec, ValidatorIndex};
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Validator groups within a session, plus some helpful indexing for
|
||||
/// looking up groups by validator indices or authority discovery ID.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Groups {
|
||||
groups: IndexedVec<GroupIndex, Vec<ValidatorIndex>>,
|
||||
by_validator_index: HashMap<ValidatorIndex, GroupIndex>,
|
||||
}
|
||||
|
||||
impl Groups {
|
||||
/// Create a new [`Groups`] tracker with the groups and discovery keys
|
||||
/// from the session.
|
||||
pub fn new(groups: IndexedVec<GroupIndex, Vec<ValidatorIndex>>) -> Self {
|
||||
let mut by_validator_index = HashMap::new();
|
||||
|
||||
for (i, group) in groups.iter().enumerate() {
|
||||
let index = GroupIndex(i as _);
|
||||
for v in group {
|
||||
by_validator_index.insert(*v, index);
|
||||
}
|
||||
}
|
||||
|
||||
Groups { groups, by_validator_index }
|
||||
}
|
||||
|
||||
/// Access all the underlying groups.
|
||||
pub fn all(&self) -> &IndexedVec<GroupIndex, Vec<ValidatorIndex>> {
|
||||
&self.groups
|
||||
}
|
||||
|
||||
/// Get the underlying group validators by group index.
|
||||
pub fn get(&self, group_index: GroupIndex) -> Option<&[ValidatorIndex]> {
|
||||
self.groups.get(group_index).map(|x| &x[..])
|
||||
}
|
||||
|
||||
/// Get the backing group size and backing threshold.
|
||||
pub fn get_size_and_backing_threshold(
|
||||
&self,
|
||||
group_index: GroupIndex,
|
||||
) -> Option<(usize, usize)> {
|
||||
self.get(group_index).map(|g| (g.len(), minimum_votes(g.len())))
|
||||
}
|
||||
|
||||
/// Get the group index for a validator by index.
|
||||
pub fn by_validator_index(&self, validator_index: ValidatorIndex) -> Option<GroupIndex> {
|
||||
self.by_validator_index.get(&validator_index).map(|x| *x)
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,283 @@
|
||||
// Copyright 2022 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! A store of all statements under a given relay-parent.
|
||||
//!
|
||||
//! This structure doesn't attempt to do any spam protection, which must
|
||||
//! be provided at a higher level.
|
||||
//!
|
||||
//! This keeps track of statements submitted with a number of different of
|
||||
//! views into this data: views based on the candidate, views based on the validator
|
||||
//! groups, and views based on the validators themselves.
|
||||
|
||||
use bitvec::{order::Lsb0 as BitOrderLsb0, vec::BitVec};
|
||||
use polkadot_node_network_protocol::vstaging::StatementFilter;
|
||||
use polkadot_primitives::vstaging::{
|
||||
CandidateHash, CompactStatement, GroupIndex, SignedStatement, ValidatorIndex,
|
||||
};
|
||||
use std::collections::hash_map::{Entry as HEntry, HashMap};
|
||||
|
||||
use super::groups::Groups;
|
||||
|
||||
/// Possible origins of a statement.
|
||||
pub enum StatementOrigin {
|
||||
/// The statement originated locally.
|
||||
Local,
|
||||
/// The statement originated from a remote peer.
|
||||
Remote,
|
||||
}
|
||||
|
||||
impl StatementOrigin {
|
||||
fn is_local(&self) -> bool {
|
||||
match *self {
|
||||
StatementOrigin::Local => true,
|
||||
StatementOrigin::Remote => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct StoredStatement {
|
||||
statement: SignedStatement,
|
||||
known_by_backing: bool,
|
||||
}
|
||||
|
||||
/// Storage for statements. Intended to be used for statements signed under
|
||||
/// the same relay-parent. See module docs for more details.
|
||||
pub struct StatementStore {
|
||||
validator_meta: HashMap<ValidatorIndex, ValidatorMeta>,
|
||||
|
||||
// we keep statements per-group because even though only one group _should_ be
|
||||
// producing statements about a candidate, until we have the candidate receipt
|
||||
// itself, we can't tell which group that is.
|
||||
group_statements: HashMap<(GroupIndex, CandidateHash), GroupStatements>,
|
||||
known_statements: HashMap<Fingerprint, StoredStatement>,
|
||||
}
|
||||
|
||||
impl StatementStore {
|
||||
/// Create a new [`StatementStore`]
|
||||
pub fn new(groups: &Groups) -> Self {
|
||||
let mut validator_meta = HashMap::new();
|
||||
for (g, group) in groups.all().iter().enumerate() {
|
||||
for (i, v) in group.iter().enumerate() {
|
||||
validator_meta.insert(
|
||||
*v,
|
||||
ValidatorMeta {
|
||||
seconded_count: 0,
|
||||
within_group_index: i,
|
||||
group: GroupIndex(g as _),
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
StatementStore {
|
||||
validator_meta,
|
||||
group_statements: HashMap::new(),
|
||||
known_statements: HashMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert a statement. Returns `true` if was not known already, `false` if it was.
|
||||
/// Ignores statements by unknown validators and returns an error.
|
||||
pub fn insert(
|
||||
&mut self,
|
||||
groups: &Groups,
|
||||
statement: SignedStatement,
|
||||
origin: StatementOrigin,
|
||||
) -> Result<bool, ValidatorUnknown> {
|
||||
let validator_index = statement.validator_index();
|
||||
let validator_meta = match self.validator_meta.get_mut(&validator_index) {
|
||||
None => return Err(ValidatorUnknown),
|
||||
Some(m) => m,
|
||||
};
|
||||
|
||||
let compact = statement.payload().clone();
|
||||
let fingerprint = (validator_index, compact.clone());
|
||||
match self.known_statements.entry(fingerprint) {
|
||||
HEntry::Occupied(mut e) => {
|
||||
if let StatementOrigin::Local = origin {
|
||||
e.get_mut().known_by_backing = true;
|
||||
}
|
||||
|
||||
return Ok(false)
|
||||
},
|
||||
HEntry::Vacant(e) => {
|
||||
e.insert(StoredStatement { statement, known_by_backing: origin.is_local() });
|
||||
},
|
||||
}
|
||||
|
||||
let candidate_hash = *compact.candidate_hash();
|
||||
let seconded = if let CompactStatement::Seconded(_) = compact { true } else { false };
|
||||
|
||||
// cross-reference updates.
|
||||
{
|
||||
let group_index = validator_meta.group;
|
||||
let group = match groups.get(group_index) {
|
||||
Some(g) => g,
|
||||
None => {
|
||||
gum::error!(
|
||||
target: crate::LOG_TARGET,
|
||||
?group_index,
|
||||
"groups passed into `insert` differ from those used at store creation"
|
||||
);
|
||||
|
||||
return Err(ValidatorUnknown)
|
||||
},
|
||||
};
|
||||
|
||||
let group_statements = self
|
||||
.group_statements
|
||||
.entry((group_index, candidate_hash))
|
||||
.or_insert_with(|| GroupStatements::with_group_size(group.len()));
|
||||
|
||||
if seconded {
|
||||
validator_meta.seconded_count += 1;
|
||||
group_statements.note_seconded(validator_meta.within_group_index);
|
||||
} else {
|
||||
group_statements.note_validated(validator_meta.within_group_index);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(true)
|
||||
}
|
||||
|
||||
/// Fill a `StatementFilter` to be used in the grid topology with all statements
|
||||
/// we are already aware of.
|
||||
pub fn fill_statement_filter(
|
||||
&self,
|
||||
group_index: GroupIndex,
|
||||
candidate_hash: CandidateHash,
|
||||
statement_filter: &mut StatementFilter,
|
||||
) {
|
||||
if let Some(statements) = self.group_statements.get(&(group_index, candidate_hash)) {
|
||||
statement_filter.seconded_in_group |= statements.seconded.as_bitslice();
|
||||
statement_filter.validated_in_group |= statements.valid.as_bitslice();
|
||||
}
|
||||
}
|
||||
|
||||
/// Get an iterator over stored signed statements by the group conforming to the
|
||||
/// given filter.
|
||||
///
|
||||
/// Seconded statements are provided first.
|
||||
pub fn group_statements<'a>(
|
||||
&'a self,
|
||||
groups: &'a Groups,
|
||||
group_index: GroupIndex,
|
||||
candidate_hash: CandidateHash,
|
||||
filter: &'a StatementFilter,
|
||||
) -> impl Iterator<Item = &'a SignedStatement> + 'a {
|
||||
let group_validators = groups.get(group_index);
|
||||
|
||||
let seconded_statements = filter
|
||||
.seconded_in_group
|
||||
.iter_ones()
|
||||
.filter_map(move |i| group_validators.as_ref().and_then(|g| g.get(i)))
|
||||
.filter_map(move |v| {
|
||||
self.known_statements.get(&(*v, CompactStatement::Seconded(candidate_hash)))
|
||||
})
|
||||
.map(|s| &s.statement);
|
||||
|
||||
let valid_statements = filter
|
||||
.validated_in_group
|
||||
.iter_ones()
|
||||
.filter_map(move |i| group_validators.as_ref().and_then(|g| g.get(i)))
|
||||
.filter_map(move |v| {
|
||||
self.known_statements.get(&(*v, CompactStatement::Valid(candidate_hash)))
|
||||
})
|
||||
.map(|s| &s.statement);
|
||||
|
||||
seconded_statements.chain(valid_statements)
|
||||
}
|
||||
|
||||
/// Get the full statement of this kind issued by this validator, if it is known.
|
||||
pub fn validator_statement(
|
||||
&self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: CompactStatement,
|
||||
) -> Option<&SignedStatement> {
|
||||
self.known_statements.get(&(validator_index, statement)).map(|s| &s.statement)
|
||||
}
|
||||
|
||||
/// Get an iterator over all statements marked as being unknown by the backing subsystem.
|
||||
pub fn fresh_statements_for_backing<'a>(
|
||||
&'a self,
|
||||
validators: &'a [ValidatorIndex],
|
||||
candidate_hash: CandidateHash,
|
||||
) -> impl Iterator<Item = &SignedStatement> + 'a {
|
||||
let s_st = CompactStatement::Seconded(candidate_hash);
|
||||
let v_st = CompactStatement::Valid(candidate_hash);
|
||||
|
||||
validators
|
||||
.iter()
|
||||
.flat_map(move |v| {
|
||||
let a = self.known_statements.get(&(*v, s_st.clone()));
|
||||
let b = self.known_statements.get(&(*v, v_st.clone()));
|
||||
|
||||
a.into_iter().chain(b)
|
||||
})
|
||||
.filter(|stored| !stored.known_by_backing)
|
||||
.map(|stored| &stored.statement)
|
||||
}
|
||||
|
||||
/// Get the amount of known `Seconded` statements by the given validator index.
|
||||
pub fn seconded_count(&self, validator_index: &ValidatorIndex) -> usize {
|
||||
self.validator_meta.get(validator_index).map_or(0, |m| m.seconded_count)
|
||||
}
|
||||
|
||||
/// Note that a statement is known by the backing subsystem.
|
||||
pub fn note_known_by_backing(
|
||||
&mut self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: CompactStatement,
|
||||
) {
|
||||
if let Some(stored) = self.known_statements.get_mut(&(validator_index, statement)) {
|
||||
stored.known_by_backing = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Error indicating that the validator was unknown.
|
||||
pub struct ValidatorUnknown;
|
||||
|
||||
type Fingerprint = (ValidatorIndex, CompactStatement);
|
||||
|
||||
struct ValidatorMeta {
|
||||
group: GroupIndex,
|
||||
within_group_index: usize,
|
||||
seconded_count: usize,
|
||||
}
|
||||
|
||||
struct GroupStatements {
|
||||
seconded: BitVec<u8, BitOrderLsb0>,
|
||||
valid: BitVec<u8, BitOrderLsb0>,
|
||||
}
|
||||
|
||||
impl GroupStatements {
|
||||
fn with_group_size(group_size: usize) -> Self {
|
||||
GroupStatements {
|
||||
seconded: BitVec::repeat(false, group_size),
|
||||
valid: BitVec::repeat(false, group_size),
|
||||
}
|
||||
}
|
||||
|
||||
fn note_seconded(&mut self, within_group_index: usize) {
|
||||
self.seconded.set(within_group_index, true);
|
||||
}
|
||||
|
||||
fn note_validated(&mut self, within_group_index: usize) {
|
||||
self.valid.set(within_group_index, true);
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,606 @@
|
||||
// Copyright 2023 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
#![allow(clippy::clone_on_copy)]
|
||||
|
||||
use super::*;
|
||||
use crate::*;
|
||||
use polkadot_node_network_protocol::{
|
||||
grid_topology::TopologyPeerInfo,
|
||||
request_response::{outgoing::Recipient, ReqProtocolNames},
|
||||
view, ObservedRole,
|
||||
};
|
||||
use polkadot_node_primitives::Statement;
|
||||
use polkadot_node_subsystem::messages::{
|
||||
network_bridge_event::NewGossipTopology, AllMessages, ChainApiMessage, FragmentTreeMembership,
|
||||
HypotheticalCandidate, NetworkBridgeEvent, ProspectiveParachainsMessage, ReportPeerMessage,
|
||||
RuntimeApiMessage, RuntimeApiRequest,
|
||||
};
|
||||
use polkadot_node_subsystem_test_helpers as test_helpers;
|
||||
use polkadot_node_subsystem_types::{jaeger, ActivatedLeaf, LeafStatus};
|
||||
use polkadot_node_subsystem_util::TimeoutExt;
|
||||
use polkadot_primitives::vstaging::{
|
||||
AssignmentPair, AsyncBackingParams, BlockNumber, CommittedCandidateReceipt, CoreState,
|
||||
GroupRotationInfo, HeadData, Header, IndexedVec, PersistedValidationData, ScheduledCore,
|
||||
SessionIndex, SessionInfo, ValidatorPair,
|
||||
};
|
||||
use sc_keystore::LocalKeystore;
|
||||
use sp_application_crypto::Pair as PairT;
|
||||
use sp_authority_discovery::AuthorityPair as AuthorityDiscoveryPair;
|
||||
use sp_keyring::Sr25519Keyring;
|
||||
|
||||
use assert_matches::assert_matches;
|
||||
use futures::Future;
|
||||
use parity_scale_codec::Encode;
|
||||
use rand::{Rng, SeedableRng};
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
mod cluster;
|
||||
mod grid;
|
||||
mod requests;
|
||||
|
||||
type VirtualOverseer = test_helpers::TestSubsystemContextHandle<StatementDistributionMessage>;
|
||||
|
||||
const DEFAULT_ASYNC_BACKING_PARAMETERS: AsyncBackingParams =
|
||||
AsyncBackingParams { max_candidate_depth: 4, allowed_ancestry_len: 3 };
|
||||
|
||||
// Some deterministic genesis hash for req/res protocol names
|
||||
const GENESIS_HASH: Hash = Hash::repeat_byte(0xff);
|
||||
|
||||
struct TestConfig {
|
||||
validator_count: usize,
|
||||
// how many validators to place in each group.
|
||||
group_size: usize,
|
||||
// whether the local node should be a validator
|
||||
local_validator: bool,
|
||||
async_backing_params: Option<AsyncBackingParams>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
struct TestLocalValidator {
|
||||
validator_index: ValidatorIndex,
|
||||
group_index: GroupIndex,
|
||||
}
|
||||
|
||||
struct TestState {
|
||||
config: TestConfig,
|
||||
local: Option<TestLocalValidator>,
|
||||
validators: Vec<ValidatorPair>,
|
||||
session_info: SessionInfo,
|
||||
req_sender: async_channel::Sender<sc_network::config::IncomingRequest>,
|
||||
}
|
||||
|
||||
impl TestState {
|
||||
fn from_config(
|
||||
config: TestConfig,
|
||||
req_sender: async_channel::Sender<sc_network::config::IncomingRequest>,
|
||||
rng: &mut impl Rng,
|
||||
) -> Self {
|
||||
if config.group_size == 0 {
|
||||
panic!("group size cannot be 0");
|
||||
}
|
||||
|
||||
let mut validators = Vec::new();
|
||||
let mut discovery_keys = Vec::new();
|
||||
let mut assignment_keys = Vec::new();
|
||||
let mut validator_groups = Vec::new();
|
||||
|
||||
let local_validator_pos = if config.local_validator {
|
||||
// ensure local validator is always in a full group.
|
||||
Some(rng.gen_range(0..config.validator_count).saturating_sub(config.group_size - 1))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
for i in 0..config.validator_count {
|
||||
let validator_pair = if Some(i) == local_validator_pos {
|
||||
// Note: the specific key is used to ensure the keystore holds
|
||||
// this key and the subsystem can detect that it is a validator.
|
||||
Sr25519Keyring::Ferdie.pair().into()
|
||||
} else {
|
||||
ValidatorPair::generate().0
|
||||
};
|
||||
let assignment_id = AssignmentPair::generate().0.public();
|
||||
let discovery_id = AuthorityDiscoveryPair::generate().0.public();
|
||||
|
||||
let group_index = i / config.group_size;
|
||||
validators.push(validator_pair);
|
||||
discovery_keys.push(discovery_id);
|
||||
assignment_keys.push(assignment_id);
|
||||
if validator_groups.len() == group_index {
|
||||
validator_groups.push(vec![ValidatorIndex(i as _)]);
|
||||
} else {
|
||||
validator_groups.last_mut().unwrap().push(ValidatorIndex(i as _));
|
||||
}
|
||||
}
|
||||
|
||||
let local = if let Some(local_pos) = local_validator_pos {
|
||||
Some(TestLocalValidator {
|
||||
validator_index: ValidatorIndex(local_pos as _),
|
||||
group_index: GroupIndex((local_pos / config.group_size) as _),
|
||||
})
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let validator_public = validator_pubkeys(&validators);
|
||||
let session_info = SessionInfo {
|
||||
validators: validator_public,
|
||||
discovery_keys,
|
||||
validator_groups: IndexedVec::from(validator_groups),
|
||||
assignment_keys,
|
||||
n_cores: 0,
|
||||
zeroth_delay_tranche_width: 0,
|
||||
relay_vrf_modulo_samples: 0,
|
||||
n_delay_tranches: 0,
|
||||
no_show_slots: 0,
|
||||
needed_approvals: 0,
|
||||
active_validator_indices: vec![],
|
||||
dispute_period: 6,
|
||||
random_seed: [0u8; 32],
|
||||
};
|
||||
|
||||
TestState { config, local, validators, session_info, req_sender }
|
||||
}
|
||||
|
||||
fn make_dummy_leaf(&self, relay_parent: Hash) -> TestLeaf {
|
||||
TestLeaf {
|
||||
number: 1,
|
||||
hash: relay_parent,
|
||||
parent_hash: Hash::repeat_byte(0),
|
||||
session: 1,
|
||||
availability_cores: self.make_availability_cores(|i| {
|
||||
CoreState::Scheduled(ScheduledCore {
|
||||
para_id: ParaId::from(i as u32),
|
||||
collator: None,
|
||||
})
|
||||
}),
|
||||
para_data: (0..self.session_info.validator_groups.len())
|
||||
.map(|i| (ParaId::from(i as u32), PerParaData::new(1, vec![1, 2, 3].into())))
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
fn make_availability_cores(&self, f: impl Fn(usize) -> CoreState) -> Vec<CoreState> {
|
||||
(0..self.session_info.validator_groups.len()).map(f).collect()
|
||||
}
|
||||
|
||||
fn make_dummy_topology(&self) -> NewGossipTopology {
|
||||
let validator_count = self.config.validator_count;
|
||||
NewGossipTopology {
|
||||
session: 1,
|
||||
topology: SessionGridTopology::new(
|
||||
(0..validator_count).collect(),
|
||||
(0..validator_count)
|
||||
.map(|i| TopologyPeerInfo {
|
||||
peer_ids: Vec::new(),
|
||||
validator_index: ValidatorIndex(i as u32),
|
||||
discovery_id: AuthorityDiscoveryPair::generate().0.public(),
|
||||
})
|
||||
.collect(),
|
||||
),
|
||||
local_index: self.local.as_ref().map(|local| local.validator_index),
|
||||
}
|
||||
}
|
||||
|
||||
fn group_validators(
|
||||
&self,
|
||||
group_index: GroupIndex,
|
||||
exclude_local: bool,
|
||||
) -> Vec<ValidatorIndex> {
|
||||
self.session_info
|
||||
.validator_groups
|
||||
.get(group_index)
|
||||
.unwrap()
|
||||
.iter()
|
||||
.cloned()
|
||||
.filter(|&i| {
|
||||
self.local.as_ref().map_or(true, |l| !exclude_local || l.validator_index != i)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn discovery_id(&self, validator_index: ValidatorIndex) -> AuthorityDiscoveryId {
|
||||
self.session_info.discovery_keys[validator_index.0 as usize].clone()
|
||||
}
|
||||
|
||||
fn sign_statement(
|
||||
&self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: CompactStatement,
|
||||
context: &SigningContext,
|
||||
) -> SignedStatement {
|
||||
let payload = statement.signing_payload(context);
|
||||
let pair = &self.validators[validator_index.0 as usize];
|
||||
let signature = pair.sign(&payload[..]);
|
||||
|
||||
SignedStatement::new(statement, validator_index, signature, context, &pair.public())
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
fn sign_full_statement(
|
||||
&self,
|
||||
validator_index: ValidatorIndex,
|
||||
statement: Statement,
|
||||
context: &SigningContext,
|
||||
pvd: PersistedValidationData,
|
||||
) -> SignedFullStatementWithPVD {
|
||||
let payload = statement.to_compact().signing_payload(context);
|
||||
let pair = &self.validators[validator_index.0 as usize];
|
||||
let signature = pair.sign(&payload[..]);
|
||||
|
||||
SignedFullStatementWithPVD::new(
|
||||
statement.supply_pvd(pvd),
|
||||
validator_index,
|
||||
signature,
|
||||
context,
|
||||
&pair.public(),
|
||||
)
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
// send a request out, returning a future which expects a response.
|
||||
async fn send_request(
|
||||
&mut self,
|
||||
peer: PeerId,
|
||||
request: AttestedCandidateRequest,
|
||||
) -> impl Future<Output = sc_network::config::OutgoingResponse> {
|
||||
let (tx, rx) = futures::channel::oneshot::channel();
|
||||
let req = sc_network::config::IncomingRequest {
|
||||
peer,
|
||||
payload: request.encode(),
|
||||
pending_response: tx,
|
||||
};
|
||||
self.req_sender.send(req).await.unwrap();
|
||||
|
||||
rx.map(|r| r.unwrap())
|
||||
}
|
||||
}
|
||||
|
||||
fn test_harness<T: Future<Output = VirtualOverseer>>(
|
||||
config: TestConfig,
|
||||
test: impl FnOnce(TestState, VirtualOverseer) -> T,
|
||||
) {
|
||||
let pool = sp_core::testing::TaskExecutor::new();
|
||||
let keystore = if config.local_validator {
|
||||
test_helpers::mock::make_ferdie_keystore()
|
||||
} else {
|
||||
Arc::new(LocalKeystore::in_memory()) as KeystorePtr
|
||||
};
|
||||
let req_protocol_names = ReqProtocolNames::new(&GENESIS_HASH, None);
|
||||
let (statement_req_receiver, _) = IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let (candidate_req_receiver, req_cfg) =
|
||||
IncomingRequest::get_config_receiver(&req_protocol_names);
|
||||
let mut rng = rand_chacha::ChaCha8Rng::seed_from_u64(0);
|
||||
|
||||
let test_state = TestState::from_config(config, req_cfg.inbound_queue.unwrap(), &mut rng);
|
||||
|
||||
let (context, virtual_overseer) = test_helpers::make_subsystem_context(pool.clone());
|
||||
let subsystem = async move {
|
||||
let subsystem = crate::StatementDistributionSubsystem {
|
||||
keystore,
|
||||
v1_req_receiver: Some(statement_req_receiver),
|
||||
req_receiver: Some(candidate_req_receiver),
|
||||
metrics: Default::default(),
|
||||
rng,
|
||||
reputation: ReputationAggregator::new(|_| true),
|
||||
};
|
||||
|
||||
if let Err(e) = subsystem.run(context).await {
|
||||
panic!("Fatal error: {:?}", e);
|
||||
}
|
||||
};
|
||||
|
||||
let test_fut = test(test_state, virtual_overseer);
|
||||
|
||||
futures::pin_mut!(test_fut);
|
||||
futures::pin_mut!(subsystem);
|
||||
futures::executor::block_on(future::join(
|
||||
async move {
|
||||
let mut virtual_overseer = test_fut.await;
|
||||
// Ensure we have handled all responses.
|
||||
if let Ok(Some(msg)) = virtual_overseer.rx.try_next() {
|
||||
panic!("Did not handle all responses: {:?}", msg);
|
||||
}
|
||||
// Conclude.
|
||||
virtual_overseer.send(FromOrchestra::Signal(OverseerSignal::Conclude)).await;
|
||||
},
|
||||
subsystem,
|
||||
));
|
||||
}
|
||||
|
||||
struct PerParaData {
|
||||
min_relay_parent: BlockNumber,
|
||||
head_data: HeadData,
|
||||
}
|
||||
|
||||
impl PerParaData {
|
||||
pub fn new(min_relay_parent: BlockNumber, head_data: HeadData) -> Self {
|
||||
Self { min_relay_parent, head_data }
|
||||
}
|
||||
}
|
||||
|
||||
struct TestLeaf {
|
||||
number: BlockNumber,
|
||||
hash: Hash,
|
||||
parent_hash: Hash,
|
||||
session: SessionIndex,
|
||||
availability_cores: Vec<CoreState>,
|
||||
para_data: Vec<(ParaId, PerParaData)>,
|
||||
}
|
||||
|
||||
impl TestLeaf {
|
||||
pub fn para_data(&self, para_id: ParaId) -> &PerParaData {
|
||||
self.para_data
|
||||
.iter()
|
||||
.find_map(|(p_id, data)| if *p_id == para_id { Some(data) } else { None })
|
||||
.unwrap()
|
||||
}
|
||||
}
|
||||
|
||||
async fn activate_leaf(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
leaf: &TestLeaf,
|
||||
test_state: &TestState,
|
||||
expect_session_info_request: bool,
|
||||
) {
|
||||
let activated = ActivatedLeaf {
|
||||
hash: leaf.hash,
|
||||
number: leaf.number,
|
||||
status: LeafStatus::Fresh,
|
||||
span: Arc::new(jaeger::Span::Disabled),
|
||||
};
|
||||
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Signal(OverseerSignal::ActiveLeaves(ActiveLeavesUpdate::start_work(
|
||||
activated,
|
||||
))))
|
||||
.await;
|
||||
|
||||
handle_leaf_activation(virtual_overseer, leaf, test_state, expect_session_info_request).await;
|
||||
}
|
||||
|
||||
async fn handle_leaf_activation(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
leaf: &TestLeaf,
|
||||
test_state: &TestState,
|
||||
expect_session_info_request: bool,
|
||||
) {
|
||||
let TestLeaf { number, hash, parent_hash, para_data, session, availability_cores } = leaf;
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::StagingAsyncBackingParams(tx))
|
||||
) if parent == *hash => {
|
||||
tx.send(Ok(test_state.config.async_backing_params.unwrap_or(DEFAULT_ASYNC_BACKING_PARAMETERS))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let mrp_response: Vec<(ParaId, BlockNumber)> = para_data
|
||||
.iter()
|
||||
.map(|(para_id, data)| (*para_id, data.min_relay_parent))
|
||||
.collect();
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetMinimumRelayParents(parent, tx)
|
||||
) if parent == *hash => {
|
||||
tx.send(mrp_response).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let header = Header {
|
||||
parent_hash: *parent_hash,
|
||||
number: *number,
|
||||
state_root: Hash::zero(),
|
||||
extrinsics_root: Hash::zero(),
|
||||
digest: Default::default(),
|
||||
};
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ChainApi(
|
||||
ChainApiMessage::BlockHeader(parent, tx)
|
||||
) if parent == *hash => {
|
||||
tx.send(Ok(Some(header))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::SessionIndexForChild(tx))) if parent == *hash => {
|
||||
tx.send(Ok(*session)).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::AvailabilityCores(tx))) if parent == *hash => {
|
||||
tx.send(Ok(availability_cores.clone())).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
let validator_groups = test_state.session_info.validator_groups.to_vec();
|
||||
let group_rotation_info =
|
||||
GroupRotationInfo { session_start_block: 1, group_rotation_frequency: 12, now: 1 };
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::ValidatorGroups(tx))) if parent == *hash => {
|
||||
tx.send(Ok((validator_groups, group_rotation_info))).unwrap();
|
||||
}
|
||||
);
|
||||
|
||||
if expect_session_info_request {
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::RuntimeApi(
|
||||
RuntimeApiMessage::Request(parent, RuntimeApiRequest::SessionInfo(s, tx))) if parent == *hash && s == *session => {
|
||||
tx.send(Ok(Some(test_state.session_info.clone()))).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Intercepts an outgoing request, checks the fields, and sends the response.
|
||||
async fn handle_sent_request(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
candidate_hash: CandidateHash,
|
||||
mask: StatementFilter,
|
||||
candidate_receipt: CommittedCandidateReceipt,
|
||||
persisted_validation_data: PersistedValidationData,
|
||||
statements: Vec<UncheckedSignedStatement>,
|
||||
) {
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::NetworkBridgeTx(NetworkBridgeTxMessage::SendRequests(mut requests, IfDisconnected::ImmediateError)) => {
|
||||
assert_eq!(requests.len(), 1);
|
||||
assert_matches!(
|
||||
requests.pop().unwrap(),
|
||||
Requests::AttestedCandidateVStaging(outgoing) => {
|
||||
assert_eq!(outgoing.peer, Recipient::Peer(peer));
|
||||
assert_eq!(outgoing.payload.candidate_hash, candidate_hash);
|
||||
assert_eq!(outgoing.payload.mask, mask);
|
||||
|
||||
let res = AttestedCandidateResponse {
|
||||
candidate_receipt,
|
||||
persisted_validation_data,
|
||||
statements,
|
||||
};
|
||||
outgoing.pending_response.send(Ok(res.encode())).unwrap();
|
||||
}
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
async fn answer_expected_hypothetical_depth_request(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
responses: Vec<(HypotheticalCandidate, FragmentTreeMembership)>,
|
||||
expected_leaf_hash: Option<Hash>,
|
||||
expected_backed_in_path_only: bool,
|
||||
) {
|
||||
assert_matches!(
|
||||
virtual_overseer.recv().await,
|
||||
AllMessages::ProspectiveParachains(
|
||||
ProspectiveParachainsMessage::GetHypotheticalFrontier(req, tx)
|
||||
) => {
|
||||
assert_eq!(req.fragment_tree_relay_parent, expected_leaf_hash);
|
||||
assert_eq!(req.backed_in_path_only, expected_backed_in_path_only);
|
||||
for (i, (candidate, _)) in responses.iter().enumerate() {
|
||||
assert!(
|
||||
req.candidates.iter().any(|c| &c == &candidate),
|
||||
"did not receive request for hypothetical candidate {}",
|
||||
i,
|
||||
);
|
||||
}
|
||||
|
||||
tx.send(responses).unwrap();
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
fn validator_pubkeys(val_ids: &[ValidatorPair]) -> IndexedVec<ValidatorIndex, ValidatorId> {
|
||||
val_ids.iter().map(|v| v.public().into()).collect()
|
||||
}
|
||||
|
||||
async fn connect_peer(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
authority_ids: Option<HashSet<AuthorityDiscoveryId>>,
|
||||
) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerConnected(
|
||||
peer,
|
||||
ObservedRole::Authority,
|
||||
ValidationVersion::VStaging.into(),
|
||||
authority_ids,
|
||||
),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
// TODO: Add some tests using this?
|
||||
#[allow(dead_code)]
|
||||
async fn disconnect_peer(virtual_overseer: &mut VirtualOverseer, peer: PeerId) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerDisconnected(peer),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn send_peer_view_change(virtual_overseer: &mut VirtualOverseer, peer: PeerId, view: View) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerViewChange(peer, view),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn send_peer_message(
|
||||
virtual_overseer: &mut VirtualOverseer,
|
||||
peer: PeerId,
|
||||
message: protocol_vstaging::StatementDistributionMessage,
|
||||
) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::PeerMessage(peer, Versioned::VStaging(message)),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn send_new_topology(virtual_overseer: &mut VirtualOverseer, topology: NewGossipTopology) {
|
||||
virtual_overseer
|
||||
.send(FromOrchestra::Communication {
|
||||
msg: StatementDistributionMessage::NetworkBridgeUpdate(
|
||||
NetworkBridgeEvent::NewGossipTopology(topology),
|
||||
),
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn overseer_recv_with_timeout(
|
||||
overseer: &mut VirtualOverseer,
|
||||
timeout: Duration,
|
||||
) -> Option<AllMessages> {
|
||||
gum::trace!("waiting for message...");
|
||||
overseer.recv().timeout(timeout).await
|
||||
}
|
||||
|
||||
fn next_group_index(
|
||||
group_index: GroupIndex,
|
||||
validator_count: usize,
|
||||
group_size: usize,
|
||||
) -> GroupIndex {
|
||||
let next_group = group_index.0 + 1;
|
||||
let num_groups =
|
||||
validator_count / group_size + if validator_count % group_size > 0 { 1 } else { 0 };
|
||||
GroupIndex::from(next_group % num_groups as u32)
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user