Files
pezkuwi-subxt/polkadot/runtime/parachains/src/paras/tests.rs
T
asynchronous rob 5174b9d2d7 Asynchronous Backing MegaPR (#5022)
* inclusion emulator logic for asynchronous backing (#4790)

* initial stab at candidate_context

* fmt

* docs & more TODOs

* some cleanups

* reframe as inclusion_emulator

* documentations yes

* update types

* add constraint modifications

* watermark

* produce modifications

* v2 primitives: re-export all v1 for consistency

* vstaging primitives

* emulator constraints: handle code upgrades

* produce outbound HRMP modifications

* stack.

* method for applying modifications

* method just for sanity-checking modifications

* fragments produce modifications, not prospectives

* make linear

* add some TODOs

* remove stacking; handle code upgrades

* take `fragment` private

* reintroduce stacking.

* fragment constructor

* add TODO

* allow validating fragments against future constraints

* docs

* relay-parent number and min code size checks

* check code upgrade restriction

* check max hrmp per candidate

* fmt

* remove GoAhead logic because it wasn't helpful

* docs on code upgrade failure

* test stacking

* test modifications against constraints

* fmt

* test fragments

* descending or duplicate test

* fmt

* remove unused imports in vstaging

* wrong primitives

* spellcheck

* Runtime changes for Asynchronous Backing (#4786)

* inclusion: utility for allowed relay-parents

* inclusion: use prev number instead of prev hash

* track most recent context of paras

* inclusion: accept previous relay-parents

* update dmp  advancement rule for async backing

* fmt

* add a comment about validation outputs

* clean up a couple of TODOs

* weights

* fix weights

* fmt

* Resolve dmp todo

* Restore inclusion tests

* Restore paras_inherent tests

* MostRecentContext test

* Benchmark for new paras dispatchable

* Prepare check_validation_outputs for upgrade

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Implementers guide changes

* More tests for allowed relay parents

* Add a github issue link

* Compute group index based on relay parent

* Storage migration

* Move allowed parents tracker to shared

* Compile error

* Get group assigned to core at the next block

* Test group assignment

* fmt

* Error instead of panic

* Update guide

* Extend doc-comment

* Update runtime/parachains/src/shared.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Prospective Parachains Subsystem (#4913)

* docs and skeleton

* subsystem skeleton

* main loop

* fragment tree basics & fmt

* begin fragment trees & view

* flesh out more of view update logic

* further flesh out update logic

* some refcount functions for fragment trees

* add fatal/non-fatal errors

* use non-fatal results

* clear up some TODOs

* ideal format for scheduling info

* add a bunch of TODOs

* some more fluff

* extract fragment graph to submodule

* begin fragment graph API

* trees, not graphs

* improve docs

* scope and constructor for trees

* add some test TODOs

* limit max ancestors and store constraints

* constructor

* constraints: fix bug in HRMP watermarks

* fragment tree population logic

* set::retain

* extract population logic

* implement add_and_populate

* fmt

* add some TODOs in tests

* implement child-selection

* strip out old stuff based on wrong assumptions

* use fatality

* implement pruning

* remove unused ancestor constraints

* fragment tree instantiation

* remove outdated comment

* add message/request types and skeleton for handling

* fmt

* implement handle_candidate_seconded

* candidate storage: handle backed

* implement handle_candidate_backed

* implement answer_get_backable_candidate

* remove async where not needed

* implement fetch_ancestry

* add logic for run_iteration

* add some docs

* remove global allow(unused), fix warnings

* make spellcheck happy (despite English)

* fmt

* bump Cargo.lock

* replace tracing with gum

* introduce PopulateFrom trait

* implement GetHypotheticalDepths

* revise docs slightly

* first fragment tree scope test

* more scope tests

* test add_candidate

* fmt

* test retain

* refactor test code

* test populate is recursive

* test contiguity of depth 0 is maintained

* add_and_populate tests

* cycle tests

* remove PopulateFrom trait

* fmt

* test hypothetical depths (non-recursive)

* have CandidateSeconded return membership

* tree membership requests

* Add a ProspectiveParachainsSubsystem struct

* add a staging API for base constraints

* add a `From` impl

* add runtime API for staging_validity_constraints

* implement fetch_base_constraints

* implement `fetch_upcoming_paras`

* remove reconstruction of candidate receipt; no obvious usecase

* fmt

* export message to broader module

* remove last TODO

* correctly export

* fix compilation and add GetMinimumRelayParent request

* make provisioner into a real subsystem with proper mesage bounds

* fmt

* fix ChannelsOut in overseer test

* fix overseer tests

* fix again

* fmt

* Integrate prospective parachains subsystem into backing: Part 1 (#5557)

* BEGIN ASYNC candidate-backing CHANGES

* rename & document modes

* answer prospective validation data requests

* GetMinimumRelayParents request is now plural

* implement an implicit view utility for backing subsystems

* implicit-view: get allowed relay parents

* refactorings and improvements to implicit view

* add some TODOs for tests

* split implicit view updates into 2 functions

* backing: define State to prepare for functional refactor

* add some docs

* backing: implement bones of new leaf activation logic

* backing: create per-relay-parent-states

* use new handle_active_leaves_update

* begin extracting logic from CandidateBackingJob

* mostly extract statement import from job logic

* handle statement imports outside of job logic

* do some TODO planning for prospective parachains integration

* finish rewriting backing subsystem in functional style

* add prospective parachains mode to relay parent entries

* fmt

* add a RejectedByProspectiveParachains error

* notify prospective parachains of seconded and backed candidates

* always validate candidates exhaustively in backing.

* return persisted_validation_data from validation

* handle rejections by prospective parachains

* implement seconding sanity check

* invoke validate_and_second

* Alter statement table to allow multiple seconded messages per validator

* refactor backing to have statements carry PVD

* clean up all warnings

* Add tests for implicit view

* Improve doc comments

* Prospective parachains mode based on Runtime API version

* Add a TODO

* Rework seconding_sanity_check

* Iterate over responses

* Update backing tests

* collator-protocol: load PVD from runtime

* Fix validator side tests

* Update statement-distribution to fetch PVD

* Fix statement-distribution tests

* Backing tests with prospective paras #1

* fix per_relay_parent pruning in backing

* Test multiple leaves

* Test seconding sanity check

* Import statement order

Before creating an entry in `PerCandidateState` map
wait for the approval from the prospective parachains

* Add a test for correct state updates

* Second multiple candidates per relay parent test

* Add backing tests with prospective paras

* Second more than one test without prospective paras

* Add a test for prospective para blocks

* Update malus

* typos

Co-authored-by: Chris Sosnin <chris125_@live.com>

* Track occupied depth in backing per parachain (#5778)

* provisioner: async backing changes (#5711)

* Provisioner changes for async backing

* Select candidates based on prospective paras mode

* Revert naming

* Update tests

* Update TODO comment

* review

* provisioner: async backing changes (#5711)

* Provisioner changes for async backing

* Select candidates based on prospective paras mode

* Revert naming

* Update tests

* Update TODO comment

* review

* fmt

* Network bridge changes for asynchronous backing + update subsystems to handle versioned packets (#5991)

* BEGIN STATEMENT DISTRIBUTION WORK

create a vstaging network protocol which is the same as v1

* mostly make network bridge amenable to vstaging

* network-bridge: fully adapt to vstaging

* add some TODOs for tests

* fix fallout in bitfield-distribution

* bitfield distribution tests + TODOs

* fix fallout in gossip-support

* collator-protocol: fix message fallout

* collator-protocol: load PVD from runtime

* add TODO for vstaging tests

* make things compile

* set used network protocol version using a feature

* fmt

* get approval-distribution building

* fix approval-distribution tests

* spellcheck

* nits

* approval distribution net protocol test

* bitfield distribution net protocol test

* Revert "collator-protocol: fix message fallout"

This reverts commit 07cc887303e16c6b3843ecb25cdc7cc2080e2ed1.

* Network bridge tests

Co-authored-by: Chris Sosnin <chris125_@live.com>

* remove max_pov_size requirement from prospective pvd request (#6014)

* remove max_pov_size requirement from prospective pvd request

* fmt

* Extract legacy statement distribution to its own module (#6026)

* add compatibility type to v2 statement distribution message

* warning cleanup

* handle compatibility layer for v2

* clean up an unimplemented!() block

* circulate statements based on version

* extract legacy v1 code into separate module

* remove unimplemented

* clean up naming of from_requester/responder

* remove TODOs

* have backing share seconded statements with PVD

* fmt

* fix warning

* Quick fix unused warning for not yet implemented/used staging messages.

* Fix network bridge test

* Fix wrong merge.

We now have 23 subsystems (network bridge split + prospective
parachains)

Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>

* Version 3 is already live.

* Fix tests (#6055)

* Fix backing tests

* Fix warnings.

* fmt

* collator-protocol: asynchronous backing changes (#5740)

* Draft collator side changes

* Start working on collations management

* Handle peer's view change

* Versioning on advertising

* Versioned collation fetching request

* Handle versioned messages

* Improve docs for collation requests

* Add spans

* Add request receiver to overseer

* Fix collator side tests

* Extract relay parent mode to lib

* Validator side draft

* Add more checks for advertisement

* Request pvd based on async backing mode

* review

* Validator side improvements

* Make old tests green

* More fixes

* Collator side tests draft

* Send collation test

* fmt

* Collator side network protocol versioning

* cleanup

* merge artifacts

* Validator side net protocol versioning

* Remove fragment tree membership request

* Resolve todo

* Collator side core state test

* Improve net protocol compatibility

* Validator side tests

* more improvements

* style fixes

* downgrade log

* Track implicit assignments

* Limit the number of seconded candidates per para

* Add a sanity check

* Handle fetched candidate

* fix tests

* Retry fetch

* Guard against dequeueing while already fetching

* Reintegrate connection management

* Timeout on advertisements

* fmt

* spellcheck

* update tests after merge

* validator assignment fixes for backing and collator protocol (#6158)

* Rename depth->ancestry len in tests

* Refactor group assignments

* Remove implicit assignments

* backing: consider occupied core assignments

* Track a single para on validator side

* Refactor prospective parachains mode request (#6179)

* Extract prospective parachains mode into util

* Skip activations depending on the mode

* backing: don't send backed candidate to provisioner (#6185)

* backing: introduce `CanSecond` request for advertisements filtering (#6225)

* Drop BoundToRelayParent

* draft changes

* fix backing tests

* Fix genesis ancestry

* Fix validator side tests

* more tests

* cargo generate-lockfile

* Implement `StagingValidityConstraints` Runtime API method (#6258)

* Implement StagingValidityConstraints

* spellcheck

* fix ump params

* Update hrmp comment

* Introduce ump per candidate limit

* hypothetical earliest block

* refactor primitives usage

* hypothetical earliest block number test

* fix build

* Prepare the Runtime for asynchronous backing upgrade (#6287)

* Introduce async backing params to runtime config

* fix cumulus config

* use config

* finish runtimes

* Introduce new staging API

* Update collator protocol

* Update provisioner

* Update prospective parachains

* Update backing

* Move async backing params lower in the config

* make naming consistent

* misc

* Use real prospective parachains subsystem (#6407)

* Backport `HypotheticalFrontier` into the feature branch (#6605)

* implement more general HypotheticalFrontier

* fmt

* drop unneeded request

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Resolve todo about legacy leaf activation (#6447)

* fix bug/warning in handling membership answers

* Remove `HypotheticalDepthRequest` in favor of `HypotheticalFrontierRequest` (#6521)

* Remove `HypotheticalDepthRequest` for `HypotheticalFrontierRequest`

* Update tests

* Fix (removed wrong docstring)

* Fix can_second request

* Patch some dead_code errors

---------

Co-authored-by: Chris Sosnin <chris125_@live.com>

* Async Backing: Send Statement Distribution "Backed" messages (#6634)

* Backing: Send Statement Distribution "Backed" messages

Closes #6590.

**TODO:**

- [ ] Adjust tests

* Fix compile errors

* (Mostly) fix tests

* Fix comment

* Fix test and compile error

* Test that `StatementDistributionMessage::Backed` is sent

* Fix compile error

* Fix some clippy errors

* Add prospective parachains subsystem tests (#6454)

* Add prospective parachains subsystem test

* Add `should_do_no_work_if_async_backing_disabled_for_leaf` test

* Implement `activate_leaf` helper, up to getting ancestry

* Finish implementing `activate_leaf`

* Small refactor in `activate_leaf`

* Get `CandidateSeconded` working

* Finish `send_candidate_and_check_if_found` test

* Refactor; send more leaves & candidates

* Refactor test

* Implement `check_candidate_parent_leaving_view` test

* Start work on `check_candidate_on_multiple_forks` test

* Don’t associate specific parachains with leaf

* Finish `correctly_updates_leaves` test

* Fix cycle due to reused head data

* Fix `check_backable_query` test

* Fix `check_candidate_on_multiple_forks` test

* Add `check_depth_and_pvd_queries` test

* Address review comments

* Remove TODO

* add a new index for output head data to candidate storage

* Resolve test TODOs

* Fix compile errors

* test candidate storage pruning, make sure new index is cleaned up

---------

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Node-side metrics for asynchronous backing (#6549)

* Add metrics for `prune_view_candidate_storage`

* Add metrics for `request_unblocked_collations`

* Fix docstring

* Couple fixes from review comments

* Fix `check_depth_query` test

* inclusion-emulator: mirror advancement rule check (#6361)

* inclusion-emulator: mirror advancement rule check

* fix build

* prospective-parachains: introduce `backed_in_path_only` flag for advertisements (#6649)

* Introduce `backed_in_path_only` flag for depth request

* fmt

* update doc comment

* fmt

* Add async-backing zombienet tests (#6314)

* Async backing: impl guide for statement distribution (#6738)

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com>

* Asynchronous backing statement distribution: Take III (#5999)

* add notification types for v2 statement-distribution

* improve protocol docs

* add empty vstaging module

* fmt

* add backed candidate packet request types

* start putting down structure of new logic

* handle activated leaf

* some sanity-checking on outbound statements

* fmt

* update vstaging share to use statements with PVD

* tiny refactor, candidate_hash location

* import local statements

* refactor statement import

* first stab at broadcast logic

* fmt

* fill out some TODOs

* start on handling incoming

* split off session info into separate map

* start in on a knowledge tracker

* address some grumbles

* format

* missed comment

* some docs for direct

* add note on slashing

* amend

* simplify 'direct' code

* finish up the 'direct' logic

* add a bunch of tests for the direct-in-group logic

* rename 'direct' to 'cluster', begin a candidate_entry module

* distill candidate_entry

* start in on a statement-store module

* some utilities for the statement store

* rewrite 'send_statement_direct' using new tools

* filter sending logic on peers which have the relay-parent in their view.

* some more logic for handling incoming statements

* req/res: BackedCandidatePacket -> AttestedCandidate + tweaks

* add a `validated_in_group` bitfield to BackedCandidateInventory

* BackedCandidateInventory -> Manifest

* start in on requester module

* add outgoing request for attested candidate

* add a priority mechanism for requester

* some request dispatch logic

* add seconded mask to tagged-request

* amend manifest to hold group index

* handle errors and set up scaffold for response validation

* validate attested candidate responses

* requester -> requests

* add some utilities for manipulating requests

* begin integrating requester

* start grid module

* tiny

* refactor grid topology to expose more info to subsystems

* fix grid_topology test

* fix overseer test

* implement topology group-based view construction logic

* fmt

* flesh out grid slightly more

* add indexed groups utility

* integrate Groups into per-session info

* refactor statement store to borrow Groups

* implement manifest knowledge utility

* add a test for topology setup

* don't send to group members

* test for conflicting manifests

* manifest knowledge tests

* fmt

* rename field

* garbage collection for grid tracker

* routines for finding correct/incorrect advertisers

* add manifest import logic

* tweak naming

* more tests for manifest import

* add comment

* rework candidates into a view-wide tracker

* fmt

* start writing boilerplate for grid sending

* fmt

* some more group boilerplate

* refactor handling of topology and authority IDs

* fmt

* send statements directly to grid peers where possible

* send to cluster only if statement belongs to cluster

* improve handling of cluster statements

* handle incoming statements along the grid

* API for introduction of candidates into the tree

* backing: use new prospective parachains API

* fmt prospective parachains changes

* fmt statement-dist

* fix condition

* get ready for tracking importable candidates

* prospective parachains: add Cow logic

* incomplete and complete hypothetical candidates

* remove keep_if_unneeded

* fmt

* implement more general HypotheticalFrontier

* fmt, cleanup

* add a by_parent_hash index to candidate tracker

* more framework for future code

* utilities for getting all hypothetical candidates for frontier

* track origin in statement store

* fmt

* requests should return peer

* apply post-confirmation reckoning

* flesh out import/announce/circulate logic on new statements

* adjust

* adjust TODO comment

* fix  backing tests

* update statement-distribution to use new indexedvec

* fmt

* query hypothetical candidates

* implement `note_importable_under`

* extract common utility of fragment tree updates

* add a helper function for getting statements unknown by backing

* import fresh statements to backing

* send announcements and acknowledgements over grid

* provide freshly importable statements

also avoid tracking backed candidates in statement distribution

* do not issue requests on newly importable candidates

* add TODO for later when confirming candidate

* write a routine for handling backed candidate notifications

* simplify grid substantially

* add some test TODOs

* handle confirmed candidates & grid announcements

* finish implementing manifest handling, including follow up statements

* send follow-up statements when acknowledging freshly backed

* fmt

* handle incoming acknowledgements

* a little DRYing

* wire up network messages to handlers

* fmt

* some skeleton code for peer view update handling

* more peer view skeleton stuff

* Fix async backing statement distribution tests (#6621)

* Fix compile errors in tests

* Cargo fmt

* Resolve some todos in async backing statement-distribution branch (#6482)

* Implement `remove_by_relay_parent`

* Extract `minimum_votes` to shared primitives.

* Add `can_send_statements_received_with_prejudice` test

* Fix test

* Update docstrings

* Cargo fmt

* Fix compile error

* Fix compile errors in tests

* Cargo fmt

* Add module docs; write `test_priority_ordering` (first draft)

* Fix `test_priority_ordering`

* Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority`

* Address review comments

* Remove `Entry::get_mut`

* fix test compilation

* add a TODO for a test

* clean up a couple of TODOs

* implement sending pending cluster statements

* refactor utility function for sending acknowledgement and statements

* mostly implement catching peers up via grid

* Fix clippy error

* alter grid to track all pending statements

* fix more TODOs and format

* tweak a TODO in requests

* some logic for dispatching requests

* fmt

* skeleton for response receiving

* Async backing statement distribution: cluster tests (#6678)

* Add `pending_statements_set_when_receiving_fresh_statements`

* Add `pending_statements_updated_when_sending_statements` test

* fix up

* fmt

* update TODO

* rework seconded mask in requests

* change doc

* change unhandledresponse not to borrow request manager

* only accept responses sufficient to back

* finish implementing response handling

* extract statement filter to protocol crate

* rework requests: use statement filter in network protocol

* dispatch cluster requests correctly

* rework cluster statement sending

* implement request answering

* fmt

* only send confirmed candidate statement messages on unified relay-parent

* Fix Tests In Statement Distribution Branch

* Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* clean up some review comments

* clean up warnings

* Async backing statement distribution: grid tests (#6673)

* Add `manifest_import_returns_ok_true` test

* cargo fmt

* Add pending_communication_receiving_manifest_on_confirmed_candidate

* Add `senders_can_provide_manifests_in_acknowledgement` test

* Add a couple of tests for pending statements

* Add `pending_statements_cleared_when_sending` test

* Add `pending_statements_respect_remote_knowledge` test

* Refactor group creation in tests

* Clarify docs

* Address some review comments

* Make some clarifications

* Fix post-merge errors

* Clarify test `senders_can_provide_manifests_in_acknowledgement`

* Try writing `pending_statements_are_updated_after_manifest_exchange`

* Document "seconding limit" and `reject_overflowing_manifests` test

* Test that seconding counts are not updated for validators on error

* Fix tests

* Fix manifest exchange test

* Add more tests in `requests.rs` (#6707)

This resolves remaining TODOs in this file.

* remove outdated inventory terminology

* Async backing statement distribution: `Candidates` tests (#6658)

* Async Backing: Fix clippy errors in statement distribution branch (#6720)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* Async Backing: Fix clippy errors in statement distribution branch

* Fix some more clippy lints

* add tests module

* fix warnings in existing tests

* create basic test harness

* create a test state struct

* fmt

* create empty cluster & grid modules for tests

* some TODOs for cluster test suite

* describe test-suite for grid logic

* describe request test suite

* fix seconding-limit bug

* Remove extraneous `pub`

This somehow made it into my clippy PR.

* Fix some test compile warnings

* Remove some unneeded `allow`s

* adapt some new test helpers from Marcin

* add helper for activating a gossip topology

* add utility for signing statements

* helpers for connecting/disconnecting peers

* round out network utilities

* fmt

* fix bug in initializing validator-meta

* fix compilation

* implement first cluster test

* TODOs for incoming request tests

* Remove unneeded `make_committed_candidate` helper

* fmt

* some more tests for cluster

* add a TODO about grid senders

* integrate inbound req/res into test harness

* polish off initial cluster test suite

* keep introduce candidate request

* fix tests after introduce candidate request

* fmt

* Add grid protocol to module docs

* Fix comments

* Test `backed_in_path_only: true`

* Update node/network/protocol/src/lib.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Update node/network/protocol/src/request_response/mod.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Mark receiver with `vstaging`

* validate grid senders based on manifest kind

* fix mask_seconded/valid

* fix unwanted-mask check

* fix build

* resolve todo on leaf mode

* Unify protocol naming to vstaging

* fmt, fix grid test after topology change

* typo

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* address review

* adjust comment, make easier to understand

* Fix typo

---------

Co-authored-by: Marcin S <marcin@bytedude.com>
Co-authored-by: Marcin S <marcin@realemail.net>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>

* miscellaneous fixes to make asynchronous backing work (#6791)

* propagate network-protocol-staging feature

* add feature to adder-collator as well

* allow collation-generation of occupied cores

* prospective parachains: special treatment for pending availability candidates

* runtime: fetch candidates pending availability

* lazily construct PVD for pending candidates

* fix fallout in prospective parachains hypothetical/select_child

* runtime: enact candidates when creating paras-inherent

* make tests compile

* test pending availability in the scope

* add prospective parachains test

* fix validity constraints leftovers

* drop prints

* Fix typos

---------

Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Marcin S <marcin@realemail.net>

* Remove restart from test (#6840)

* Async Backing: Statement Distribution Tests (#6755)

* start on handling incoming

* split off session info into separate map

* start in on a knowledge tracker

* address some grumbles

* format

* missed comment

* some docs for direct

* add note on slashing

* amend

* simplify 'direct' code

* finish up the 'direct' logic

* add a bunch of tests for the direct-in-group logic

* rename 'direct' to 'cluster', begin a candidate_entry module

* distill candidate_entry

* start in on a statement-store module

* some utilities for the statement store

* rewrite 'send_statement_direct' using new tools

* filter sending logic on peers which have the relay-parent in their view.

* some more logic for handling incoming statements

* req/res: BackedCandidatePacket -> AttestedCandidate + tweaks

* add a `validated_in_group` bitfield to BackedCandidateInventory

* BackedCandidateInventory -> Manifest

* start in on requester module

* add outgoing request for attested candidate

* add a priority mechanism for requester

* some request dispatch logic

* add seconded mask to tagged-request

* amend manifest to hold group index

* handle errors and set up scaffold for response validation

* validate attested candidate responses

* requester -> requests

* add some utilities for manipulating requests

* begin integrating requester

* start grid module

* tiny

* refactor grid topology to expose more info to subsystems

* fix grid_topology test

* fix overseer test

* implement topology group-based view construction logic

* fmt

* flesh out grid slightly more

* add indexed groups utility

* integrate Groups into per-session info

* refactor statement store to borrow Groups

* implement manifest knowledge utility

* add a test for topology setup

* don't send to group members

* test for conflicting manifests

* manifest knowledge tests

* fmt

* rename field

* garbage collection for grid tracker

* routines for finding correct/incorrect advertisers

* add manifest import logic

* tweak naming

* more tests for manifest import

* add comment

* rework candidates into a view-wide tracker

* fmt

* start writing boilerplate for grid sending

* fmt

* some more group boilerplate

* refactor handling of topology and authority IDs

* fmt

* send statements directly to grid peers where possible

* send to cluster only if statement belongs to cluster

* improve handling of cluster statements

* handle incoming statements along the grid

* API for introduction of candidates into the tree

* backing: use new prospective parachains API

* fmt prospective parachains changes

* fmt statement-dist

* fix condition

* get ready for tracking importable candidates

* prospective parachains: add Cow logic

* incomplete and complete hypothetical candidates

* remove keep_if_unneeded

* fmt

* implement more general HypotheticalFrontier

* fmt, cleanup

* add a by_parent_hash index to candidate tracker

* more framework for future code

* utilities for getting all hypothetical candidates for frontier

* track origin in statement store

* fmt

* requests should return peer

* apply post-confirmation reckoning

* flesh out import/announce/circulate logic on new statements

* adjust

* adjust TODO comment

* fix  backing tests

* update statement-distribution to use new indexedvec

* fmt

* query hypothetical candidates

* implement `note_importable_under`

* extract common utility of fragment tree updates

* add a helper function for getting statements unknown by backing

* import fresh statements to backing

* send announcements and acknowledgements over grid

* provide freshly importable statements

also avoid tracking backed candidates in statement distribution

* do not issue requests on newly importable candidates

* add TODO for later when confirming candidate

* write a routine for handling backed candidate notifications

* simplify grid substantially

* add some test TODOs

* handle confirmed candidates & grid announcements

* finish implementing manifest handling, including follow up statements

* send follow-up statements when acknowledging freshly backed

* fmt

* handle incoming acknowledgements

* a little DRYing

* wire up network messages to handlers

* fmt

* some skeleton code for peer view update handling

* more peer view skeleton stuff

* Fix async backing statement distribution tests (#6621)

* Fix compile errors in tests

* Cargo fmt

* Resolve some todos in async backing statement-distribution branch (#6482)

* Implement `remove_by_relay_parent`

* Extract `minimum_votes` to shared primitives.

* Add `can_send_statements_received_with_prejudice` test

* Fix test

* Update docstrings

* Cargo fmt

* Fix compile error

* Fix compile errors in tests

* Cargo fmt

* Add module docs; write `test_priority_ordering` (first draft)

* Fix `test_priority_ordering`

* Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority`

* Address review comments

* Remove `Entry::get_mut`

* fix test compilation

* add a TODO for a test

* clean up a couple of TODOs

* implement sending pending cluster statements

* refactor utility function for sending acknowledgement and statements

* mostly implement catching peers up via grid

* Fix clippy error

* alter grid to track all pending statements

* fix more TODOs and format

* tweak a TODO in requests

* some logic for dispatching requests

* fmt

* skeleton for response receiving

* Async backing statement distribution: cluster tests (#6678)

* Add `pending_statements_set_when_receiving_fresh_statements`

* Add `pending_statements_updated_when_sending_statements` test

* fix up

* fmt

* update TODO

* rework seconded mask in requests

* change doc

* change unhandledresponse not to borrow request manager

* only accept responses sufficient to back

* finish implementing response handling

* extract statement filter to protocol crate

* rework requests: use statement filter in network protocol

* dispatch cluster requests correctly

* rework cluster statement sending

* implement request answering

* fmt

* only send confirmed candidate statement messages on unified relay-parent

* Fix Tests In Statement Distribution Branch

* Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* clean up some review comments

* clean up warnings

* Async backing statement distribution: grid tests (#6673)

* Add `manifest_import_returns_ok_true` test

* cargo fmt

* Add pending_communication_receiving_manifest_on_confirmed_candidate

* Add `senders_can_provide_manifests_in_acknowledgement` test

* Add a couple of tests for pending statements

* Add `pending_statements_cleared_when_sending` test

* Add `pending_statements_respect_remote_knowledge` test

* Refactor group creation in tests

* Clarify docs

* Address some review comments

* Make some clarifications

* Fix post-merge errors

* Clarify test `senders_can_provide_manifests_in_acknowledgement`

* Try writing `pending_statements_are_updated_after_manifest_exchange`

* Document "seconding limit" and `reject_overflowing_manifests` test

* Test that seconding counts are not updated for validators on error

* Fix tests

* Fix manifest exchange test

* Add more tests in `requests.rs` (#6707)

This resolves remaining TODOs in this file.

* remove outdated inventory terminology

* Async backing statement distribution: `Candidates` tests (#6658)

* Async Backing: Fix clippy errors in statement distribution branch (#6720)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* Async Backing: Fix clippy errors in statement distribution branch

* Fix some more clippy lints

* add tests module

* fix warnings in existing tests

* create basic test harness

* create a test state struct

* fmt

* create empty cluster & grid modules for tests

* some TODOs for cluster test suite

* describe test-suite for grid logic

* describe request test suite

* fix seconding-limit bug

* Remove extraneous `pub`

This somehow made it into my clippy PR.

* Fix some test compile warnings

* Remove some unneeded `allow`s

* adapt some new test helpers from Marcin

* add helper for activating a gossip topology

* add utility for signing statements

* helpers for connecting/disconnecting peers

* round out network utilities

* fmt

* fix bug in initializing validator-meta

* fix compilation

* implement first cluster test

* TODOs for incoming request tests

* Remove unneeded `make_committed_candidate` helper

* fmt

* Hook up request sender

* Add `valid_statement_without_prior_seconded_is_ignored` test

* Fix `valid_statement_without_prior_seconded_is_ignored` test

* some more tests for cluster

* add a TODO about grid senders

* integrate inbound req/res into test harness

* polish off initial cluster test suite

* keep introduce candidate request

* fix tests after introduce candidate request

* fmt

* Add grid protocol to module docs

* Remove obsolete test

* Fix comments

* Test `backed_in_path_only: true`

* Update node/network/protocol/src/lib.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Update node/network/protocol/src/request_response/mod.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Mark receiver with `vstaging`

* First draft of `ensure_seconding_limit_is_respected` test

* validate grid senders based on manifest kind

* fix mask_seconded/valid

* fix unwanted-mask check

* fix build

* resolve todo on leaf mode

* Unify protocol naming to vstaging

* Fix `ensure_seconding_limit_is_respected` test

* Start `backed_candidate_leads_to_advertisement` test

* fmt, fix grid test after topology change

* Send Backed notification

* Finish `backed_candidate_leads_to_advertisement` test

* Finish `peer_reported_for_duplicate_statements` test

* Finish `received_advertisement_before_confirmation_leads_to_request`

* Add `advertisements_rejected_from_incorrect_peers` test

* Add `manifest_rejected_*` tests

* Add `manifest_rejected_when_group_does_not_match_para` test

* Add `local_node_sanity_checks_incoming_requests` test

* Add `local_node_respects_statement_mask` test

* Add tests where peer is reported for providing invalid signatures

* Add `cluster_peer_allowed_to_send_incomplete_statements` test

* Add `received_advertisement_after_backing_leads_to_acknowledgement`

* Add `received_advertisement_after_confirmation_before_backing` test

* peer_reported_for_advertisement_conflicting_with_confirmed_candidate

* Add `peer_reported_for_not_enough_statements` test

* Add `peer_reported_for_providing_statements_meant_to_be_masked_out`

* Add `additional_statements_are_shared_after_manifest_exchange`

* Add `grid_statements_imported_to_backing` test

* Add `relay_parent_entering_peer_view_leads_to_advertisement` test

* Add `advertisement_not_re_sent_when_peer_re_enters_view` test

* Update node/network/statement-distribution/src/vstaging/tests/grid.rs

Co-authored-by: asynchronous rob <rphmeier@gmail.com>

* Resolve TODOs, update test

* Address unused code

* Add check after every test for unhandled requests

* Refactor (`make_dummy_leaf` and `handle_sent_request`)

* Refactor (`make_dummy_topology`)

* Minor refactor

---------

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>

* Fix some clippy lints in tests

* Async backing: minor fixes (#6920)

* bitfield-distribution test

* implicit view tests

* Refactor parameters -> params

* scheduler: update storage migration (#6963)

* update scheduler migration

* Adjust weight to account for storage read

* Statement Distribution Guide Edits (#7025)

* Statement distribution guide edits

* Addressed Marcin's comments

* Add attested candidate request retry timeouts (#6833)

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
Fix async backing statement distribution tests (#6621)
Resolve some todos in async backing statement-distribution branch (#6482)
Fix clippy errors in statement distribution branch (#6720)

* Async backing: add Prospective Parachains impl guide (#6933)

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Updates to Provisioner Guide for Async Backing (#7106)

* Initial corrections and clarifications

* Partial first draft

* Finished first draft

* Adding back wrongly removed test bit

* fmt

* Update roadmap/implementers-guide/src/node/utility/provisioner.md

Co-authored-by: Marcin S. <marcin@realemail.net>

* Addressing comments

* Reorganization

* fmt

---------

Co-authored-by: Marcin S. <marcin@realemail.net>

* fmt

* Renaming Parathread Mentions (#7287)

* Renaming parathreads

* Renaming module to pallet

* More updates

* PVF: Refactor workers into separate crates, remove host dependency (#7253)

* PVF: Refactor workers into separate crates, remove host dependency

* Fix compile error

* Remove some leftover code

* Fix compile errors

* Update Cargo.lock

* Remove worker main.rs files

I accidentally copied these from the other PR. This PR isn't intended to
introduce standalone workers yet.

* Address review comments

* cargo fmt

* Update a couple of comments

* Update log targets

* Update quote to 1.0.27 (#7280)

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>

* pallets: implement `Default` for `GenesisConfig` in `no_std` (#7271)

* pallets: implement Default for GenesisConfig in no_std

This change is follow-up of: https://github.com/paritytech/substrate/pull/14108

It is a step towards: https://github.com/paritytech/substrate/issues/13334

* Cargo.lock updated

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* cli: enable BEEFY by default on test networks (#7293)

We consider BEEFY mature enough to run by default on all nodes
for test networks (Rococo/Wococo/Versi).

Right now, most nodes are not running it since it's opt-in using
--beefy flag. Switch to an opt-out model for test networks.

Replace --beefy flag from CLI with --no-beefy and have BEEFY
client start by default on test networks.

Signed-off-by: acatangiu <adrian@parity.io>

* runtime: past session slashing runtime API (#6667)

* runtime/vstaging: unapplied_slashes runtime API

* runtime/vstaging: key_ownership_proof runtime API

* runtime/ParachainHost: submit_report_dispute_lost

* fix key_ownership_proof API

* runtime: submit_report_dispute_lost runtime API

* nits

* Update node/subsystem-types/src/messages.rs

Co-authored-by: Marcin S. <marcin@bytedude.com>

* revert unrelated fmt changes

* post merge fixes

* fix compilation

---------

Co-authored-by: Marcin S. <marcin@bytedude.com>

* Correcting git mishap

* Document usage of `gum` crate (#7294)

* Document usage of gum crate

* Small fix

* Add some more basic info

* Update node/gum/src/lib.rs

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* Update target docs

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* XCM: Fix issue with RequestUnlock (#7278)

* XCM: Fix issue with RequestUnlock

* Leave API changes for v4

* Fix clippy errors

* Fix tests

---------

Co-authored-by: parity-processbot <>

* Companion for Substrate#14228 (#7295)

* Companion for Substrate#14228

https://github.com/paritytech/substrate/pull/14228

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* Companion for #14237: Use latest sp-crates (#7300)

* To revert: Update substrate branch to "lexnv/bump_sp_crates"

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "To revert: Update substrate branch to "lexnv/bump_sp_crates""

This reverts commit 5f1db84eac4a226c37b7f6ce6ee19b49dc7e2008.

* Update cargo lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* bounded-collections bump to 0.1.7 (#7305)

* bounded-collections bump to 0.1.7

Companion for: paritytech/substrate#14225

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* bump to quote 1.0.28 (#7306)

* `RollingSessionWindow` cleanup (#7204)

* Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit

* Fix tests in import

* Fix the rest of the tests

* Remove dead code

* Fix todos

* Simplify session caching

* Comments for `SessionInfoProvider`

* Separate `SessionInfoProvider` from `State`

* `cache_session_info_for_head` becomes freestanding function

* Remove unneeded `mut` usage

* fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider`

* Fix SessionInfo retrieval

* Code cleanup

* Don't wrap `SessionInfoProvider` in an `Option`

* Remove `earliest_session()`

* Remove pre-caching -> wip

* Fix some tests and code cleanup

* Fix all tests

* Fixes in tests

* Fix comments, variable names and small style changes

* Fix a warning

* impl From<SessionWindowSize> for NonZeroUsize

* Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG

* Code review feedback

* Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db

* Remove `col_session_data` usages

* Storage migration clearing columns w/o removing them

* Remove session data column usages from `approval-voting` and `dispute-coordinator` tests

* Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests

* Fix formatting in initialized.rs

* Fix a corner case in `SessionInfo` caching for `dispute-coordinator`

* Remove `RollingSessionWindow` ;(

* Revert "Fix formatting in initialized.rs"

This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc.

* v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it

* Fix `NUM_COLUMNS` in `approval-voting`

* Use `columns::v3::NUM_COLUMNS` when opening db

* Update node/service/src/parachains_db/upgrade.rs

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3`

* Fix `NUM+COLUMNS` in approval_voting

* Fix formatting

* Fix columns usage

* Clarification comments about the different db versions

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* pallet-para-config: Remove remnant WeightInfo functions (#7308)

* pallet-para-config: Remove remnant WeightInfo functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* set_config_with_weight begone

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::configuration

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

* XCM: PayOverXcm config (#6900)

* Move XCM query functionality to trait

* Fix tests

* Add PayOverXcm implementation

* fix the PayOverXcm trait to compile

* moved doc comment out of trait implmeentation and to the trait

* PayOverXCM documentation

* Change documentation a bit

* Added empty benchmark methods implementation and changed docs

* update PayOverXCM to convert AccountIds to MultiLocations

* Implement benchmarking method

* Change v3 to latest

* Descend origin to an asset sender (#6970)

* descend origin to an asset sender

* sender as tuple of dest and sender

* Add more variants to the QueryResponseStatus enum

* Change Beneficiary to Into<[u8; 32]>

* update PayOverXcm to return concrete errors and use AccountId as sender

* use polkadot-primitives for AccountId

* fix dependency to use polkadot-core-primitives

* force Unpaid instruction to the top of the instructions list

* modify report_outcome to accept interior argument

* use new_query directly for building final xcm query, instead of report_outcome

* fix usage of new_query to use the XcmQueryHandler

* fix usage of new_query to use the XcmQueryHandler

* tiny method calling fix

* xcm query handler (#7198)

* drop redundant query status

* rename ReportQueryStatus to OuterQueryStatus

* revert rename of QueryResponseStatus

* update mapping

* Update xcm/xcm-builder/src/pay.rs

Co-authored-by: Gavin Wood <gavin@parity.io>

* Updates

* Docs

* Fix benchmarking stuff

* Destination can be determined based on asset_kind

* Tweaking API to minimise clones

* Some repotting and docs

---------

Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>

* Companion for #14265 (#7307)

* Update Cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update Cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>

* bump serde to 1.0.163 (#7315)

* bump serde to 1.0.163

* bump ci

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* fmt

* Updated fmt

* Removing changes accidentally pulled from master

* fix another master pull issue

* Another master pull fix

* fmt

* Fixing implementers guide build

* Revert "Merge branch 'rh-async-backing-feature-while-frozen' of https://github.com/paritytech/polkadot into brad-rename-parathread"

This reverts commit bebc24af52ab61155e3fe02cb3ce66a592bce49c, reversing
changes made to 1b2de662dfb11173679d6da5bd0da9d149c85547.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: acatangiu <adrian@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Marcin S <marcin@realemail.net>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>

* fix bitfield distribution test

* approval distribution tests

* fix bridge tests

* update Cargo.lock

* [async-backing-branch] Optimize collator-protocol validator-side request fetching (#7457)

* Optimize collator-protocol validator-side request fetching

* address feedback: replace tuples with structs

* feedback: add doc comments

* move collation types to subfolder

---------

Signed-off-by: alindima <alin@parity.io>

* Update collation generation for asynchronous backing (#7405)

* break candidate receipt construction and distribution into own function

* update implementers' guide to include SubmitCollation

* implement SubmitCollation for collation-generation

* fmt

* fix test compilation & remove unnecessary submodule

* add some TODOs for a test suite.

* Update roadmap/implementers-guide/src/types/overseer-protocol.md

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* add new test harness and first test

* refactor to avoid requiring background sender

* ensure collation gets packaged and distributed

* tests for the fallback case with no hint

* add parent rp-number hint tests

* fmt

* update uses of CollationGenerationConfig

* fix remaining test

* address review comments

* use subsystemsender for background tasks

* fmt

* remove ValidationCodeHashHint and related tests

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* fix some more fallout from merge

* fmt

* remove staging APIs from Rococo & Westend (#7513)

* send network messages on main protocol name (#7515)

* misc async backing improvements for allowed ancestry blocks (#7532)

* shared: fix acquire_info

* backwards-compat test for prospective parachains

* same relay parent is allowed

* provisioner: request candidate receipt by relay parent (#7527)

* return candidates hash from prospective parachains

* update provisioner

* update tests

* guide changes

* send a single message to backing

* fix test

* revert to old `handle_new_activations` logic in some cases (#7514)

* revert to old `handle_new_activations` logic

* gate sending messages on scheduled cores to max_depth >= 2

* fmt

* 2->1

* Omnibus asynchronous backing bugfix PR (#7529)

* fix a bug in backing

* add some more logs

* prospective parachains: take ancestry only up to session bounds

* add test

* fix zombienet tests (#7614)

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix runtime compilation

* make bitfield distribution tests compile

* attempt to fix zombienet disputes (#7618)

* update metric name

* update some metric names

* avoid cycles when creating fake candidates

* make undying collator more friendly to malformed parents

* fix a bug in malus

* fmt

* clippy

* add RUN_IN_CONTAINER to new ZombieNet tests (#7631)

* remove duplicated migration

happened because of master-merge

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: acatangiu <adrian@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: alindima <alin@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Marcin S <marcin@bytedude.com>
Co-authored-by: Marcin S <marcin@realemail.net>
Co-authored-by: Mattia L.V. Bradascio <28816406+bredamatt@users.noreply.github.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com>
Co-authored-by: BradleyOlson64 <lotrftw9@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Alin Dima <alin@parity.io>
2023-08-18 16:11:56 +00:00

1827 lines
57 KiB
Rust

// Copyright (C) Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
use super::*;
use frame_support::{assert_err, assert_ok, assert_storage_noop};
use keyring::Sr25519Keyring;
use primitives::{BlockNumber, ValidatorId, PARACHAIN_KEY_TYPE_ID};
use sc_keystore::LocalKeystore;
use sp_keystore::{Keystore, KeystorePtr};
use std::sync::Arc;
use test_helpers::{dummy_head_data, dummy_validation_code};
use crate::{
configuration::HostConfiguration,
mock::{
new_test_ext, Configuration, MockGenesisConfig, Paras, ParasShared, RuntimeOrigin, System,
Test,
},
};
static VALIDATORS: &[Sr25519Keyring] = &[
Sr25519Keyring::Alice,
Sr25519Keyring::Bob,
Sr25519Keyring::Charlie,
Sr25519Keyring::Dave,
Sr25519Keyring::Ferdie,
];
fn validator_pubkeys(val_ids: &[Sr25519Keyring]) -> Vec<ValidatorId> {
val_ids.iter().map(|v| v.public().into()).collect()
}
fn sign_and_include_pvf_check_statement(stmt: PvfCheckStatement) {
let validators = &[
Sr25519Keyring::Alice,
Sr25519Keyring::Bob,
Sr25519Keyring::Charlie,
Sr25519Keyring::Dave,
Sr25519Keyring::Ferdie,
];
let signature = validators[stmt.validator_index.0 as usize].sign(&stmt.signing_payload());
Paras::include_pvf_check_statement(None.into(), stmt, signature.into()).unwrap();
}
fn submit_super_majority_pvf_votes(
validation_code: &ValidationCode,
session_index: SessionIndex,
accept: bool,
) {
[0, 1, 2, 3]
.into_iter()
.map(|i| PvfCheckStatement {
accept,
subject: validation_code.hash(),
session_index,
validator_index: i.into(),
})
.for_each(sign_and_include_pvf_check_statement);
}
fn run_to_block(to: BlockNumber, new_session: Option<Vec<BlockNumber>>) {
let keystore: KeystorePtr = Arc::new(LocalKeystore::in_memory());
for validator in VALIDATORS.iter() {
Keystore::sr25519_generate_new(
&*keystore,
PARACHAIN_KEY_TYPE_ID,
Some(&validator.to_seed()),
)
.unwrap();
}
let validator_pubkeys = validator_pubkeys(VALIDATORS);
while System::block_number() < to {
let b = System::block_number();
Paras::initializer_finalize(b);
ParasShared::initializer_finalize();
if new_session.as_ref().map_or(false, |v| v.contains(&(b + 1))) {
let mut session_change_notification = SessionChangeNotification::default();
session_change_notification.session_index = ParasShared::session_index() + 1;
session_change_notification.validators = validator_pubkeys.clone();
ParasShared::initializer_on_new_session(
session_change_notification.session_index,
session_change_notification.random_seed,
&session_change_notification.new_config,
session_change_notification.validators.clone(),
);
ParasShared::set_active_validators_ascending(validator_pubkeys.clone());
Paras::initializer_on_new_session(&session_change_notification);
}
System::on_finalize(b);
System::on_initialize(b + 1);
System::set_block_number(b + 1);
ParasShared::initializer_initialize(b + 1);
Paras::initializer_initialize(b + 1);
}
}
fn upgrade_at(
expected_at: BlockNumber,
activated_at: BlockNumber,
) -> ReplacementTimes<BlockNumber> {
ReplacementTimes { expected_at, activated_at }
}
fn check_code_is_stored(validation_code: &ValidationCode) {
assert!(CodeByHashRefs::<Test>::get(validation_code.hash()) != 0);
assert!(CodeByHash::<Test>::contains_key(validation_code.hash()));
}
fn check_code_is_not_stored(validation_code: &ValidationCode) {
assert!(!CodeByHashRefs::<Test>::contains_key(validation_code.hash()));
assert!(!CodeByHash::<Test>::contains_key(validation_code.hash()));
}
/// An utility for checking that certain events were deposited.
struct EventValidator {
events: Vec<
frame_system::EventRecord<<Test as frame_system::Config>::RuntimeEvent, primitives::Hash>,
>,
}
impl EventValidator {
fn new() -> Self {
Self { events: Vec::new() }
}
fn started(&mut self, code: &ValidationCode, id: ParaId) -> &mut Self {
self.events.push(frame_system::EventRecord {
phase: frame_system::Phase::Initialization,
event: Event::PvfCheckStarted(code.hash(), id).into(),
topics: vec![],
});
self
}
fn rejected(&mut self, code: &ValidationCode, id: ParaId) -> &mut Self {
self.events.push(frame_system::EventRecord {
phase: frame_system::Phase::Initialization,
event: Event::PvfCheckRejected(code.hash(), id).into(),
topics: vec![],
});
self
}
fn accepted(&mut self, code: &ValidationCode, id: ParaId) -> &mut Self {
self.events.push(frame_system::EventRecord {
phase: frame_system::Phase::Initialization,
event: Event::PvfCheckAccepted(code.hash(), id).into(),
topics: vec![],
});
self
}
fn check(&self) {
assert_eq!(&frame_system::Pallet::<Test>::events(), &self.events);
}
}
#[test]
fn para_past_code_pruning_works_correctly() {
let mut past_code = ParaPastCodeMeta::default();
past_code.note_replacement(10u32, 10);
past_code.note_replacement(20, 25);
past_code.note_replacement(30, 35);
let old = past_code.clone();
assert!(past_code.prune_up_to(9).collect::<Vec<_>>().is_empty());
assert_eq!(old, past_code);
assert_eq!(past_code.prune_up_to(10).collect::<Vec<_>>(), vec![10]);
assert_eq!(
past_code,
ParaPastCodeMeta {
upgrade_times: vec![upgrade_at(20, 25), upgrade_at(30, 35)],
last_pruned: Some(10),
}
);
assert!(past_code.prune_up_to(21).collect::<Vec<_>>().is_empty());
assert_eq!(past_code.prune_up_to(26).collect::<Vec<_>>(), vec![20]);
assert_eq!(
past_code,
ParaPastCodeMeta { upgrade_times: vec![upgrade_at(30, 35)], last_pruned: Some(25) }
);
past_code.note_replacement(40, 42);
past_code.note_replacement(50, 53);
past_code.note_replacement(60, 66);
assert_eq!(
past_code,
ParaPastCodeMeta {
upgrade_times: vec![
upgrade_at(30, 35),
upgrade_at(40, 42),
upgrade_at(50, 53),
upgrade_at(60, 66)
],
last_pruned: Some(25),
}
);
assert_eq!(past_code.prune_up_to(60).collect::<Vec<_>>(), vec![30, 40, 50]);
assert_eq!(
past_code,
ParaPastCodeMeta { upgrade_times: vec![upgrade_at(60, 66)], last_pruned: Some(53) }
);
assert_eq!(past_code.most_recent_change(), Some(60));
assert_eq!(past_code.prune_up_to(66).collect::<Vec<_>>(), vec![60]);
assert_eq!(past_code, ParaPastCodeMeta { upgrade_times: Vec::new(), last_pruned: Some(66) });
}
#[test]
fn schedule_para_init_rejects_empty_code() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
assert_err!(
Paras::schedule_para_initialize(
1000.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: dummy_head_data(),
validation_code: ValidationCode(vec![]),
}
),
Error::<Test>::CannotOnboard,
);
assert_ok!(Paras::schedule_para_initialize(
1000.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: dummy_head_data(),
validation_code: ValidationCode(vec![1]),
}
));
});
}
#[test]
fn para_past_code_pruning_in_initialize() {
let code_retention_period = 10;
let paras = vec![
(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: dummy_validation_code(),
},
),
(
1u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: dummy_head_data(),
validation_code: dummy_validation_code(),
},
),
];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration { code_retention_period, ..Default::default() },
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
let id = ParaId::from(0u32);
let at_block: BlockNumber = 10;
let included_block: BlockNumber = 12;
let validation_code = ValidationCode(vec![4, 5, 6]);
Paras::increase_code_ref(&validation_code.hash(), &validation_code);
PastCodeHash::<Test>::insert(&(id, at_block), &validation_code.hash());
PastCodePruning::<Test>::put(&vec![(id, included_block)]);
{
let mut code_meta = Paras::past_code_meta(&id);
code_meta.note_replacement(at_block, included_block);
PastCodeMeta::<Test>::insert(&id, &code_meta);
}
let pruned_at: BlockNumber = included_block + code_retention_period + 1;
assert_eq!(PastCodeHash::<Test>::get(&(id, at_block)), Some(validation_code.hash()));
check_code_is_stored(&validation_code);
run_to_block(pruned_at - 1, None);
assert_eq!(PastCodeHash::<Test>::get(&(id, at_block)), Some(validation_code.hash()));
assert_eq!(Paras::past_code_meta(&id).most_recent_change(), Some(at_block));
check_code_is_stored(&validation_code);
run_to_block(pruned_at, None);
assert!(PastCodeHash::<Test>::get(&(id, at_block)).is_none());
assert!(Paras::past_code_meta(&id).most_recent_change().is_none());
check_code_is_not_stored(&validation_code);
});
}
#[test]
fn note_new_head_sets_head() {
let code_retention_period = 10;
let paras = vec![(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: dummy_validation_code(),
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration { code_retention_period, ..Default::default() },
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
let id_a = ParaId::from(0u32);
assert_eq!(Paras::para_head(&id_a), Some(dummy_head_data()));
Paras::note_new_head(id_a, vec![1, 2, 3].into(), 0);
assert_eq!(Paras::para_head(&id_a), Some(vec![1, 2, 3].into()));
});
}
#[test]
fn note_past_code_sets_up_pruning_correctly() {
let code_retention_period = 10;
let paras = vec![
(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: dummy_validation_code(),
},
),
(
1u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: dummy_head_data(),
validation_code: dummy_validation_code(),
},
),
];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration { code_retention_period, ..Default::default() },
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
let id_a = ParaId::from(0u32);
let id_b = ParaId::from(1u32);
Paras::note_past_code(id_a, 10, 12, ValidationCode(vec![1, 2, 3]).hash());
Paras::note_past_code(id_b, 20, 23, ValidationCode(vec![4, 5, 6]).hash());
assert_eq!(PastCodePruning::<Test>::get(), vec![(id_a, 12), (id_b, 23)]);
assert_eq!(
Paras::past_code_meta(&id_a),
ParaPastCodeMeta { upgrade_times: vec![upgrade_at(10, 12)], last_pruned: None }
);
assert_eq!(
Paras::past_code_meta(&id_b),
ParaPastCodeMeta { upgrade_times: vec![upgrade_at(20, 23)], last_pruned: None }
);
});
}
#[test]
fn code_upgrade_applied_after_delay() {
let code_retention_period = 10;
let validation_upgrade_delay = 5;
let validation_upgrade_cooldown = 10;
let original_code = ValidationCode(vec![1, 2, 3]);
let paras = vec![(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: original_code.clone(),
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
code_retention_period,
validation_upgrade_delay,
validation_upgrade_cooldown,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
check_code_is_stored(&original_code);
let para_id = ParaId::from(0);
let new_code = ValidationCode(vec![4, 5, 6]);
// Wait for at least one session change to set active validators.
const EXPECTED_SESSION: SessionIndex = 1;
run_to_block(2, Some(vec![1]));
assert_eq!(Paras::current_code(&para_id), Some(original_code.clone()));
let (expected_at, next_possible_upgrade_at) = {
// this parablock is in the context of block 1.
let expected_at = 1 + validation_upgrade_delay;
let next_possible_upgrade_at = 1 + validation_upgrade_cooldown;
Paras::schedule_code_upgrade(para_id, new_code.clone(), 1, &Configuration::config());
// Include votes for super-majority.
submit_super_majority_pvf_votes(&new_code, EXPECTED_SESSION, true);
Paras::note_new_head(para_id, Default::default(), 1);
assert!(Paras::past_code_meta(&para_id).most_recent_change().is_none());
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(expected_at));
assert_eq!(FutureCodeHash::<Test>::get(&para_id), Some(new_code.hash()));
assert_eq!(UpcomingUpgrades::<Test>::get(), vec![(para_id, expected_at)]);
assert_eq!(UpgradeCooldowns::<Test>::get(), vec![(para_id, next_possible_upgrade_at)]);
assert_eq!(Paras::current_code(&para_id), Some(original_code.clone()));
check_code_is_stored(&original_code);
check_code_is_stored(&new_code);
(expected_at, next_possible_upgrade_at)
};
run_to_block(expected_at, None);
// the candidate is in the context of the parent of `expected_at`,
// thus does not trigger the code upgrade.
{
Paras::note_new_head(para_id, Default::default(), expected_at - 1);
assert!(Paras::past_code_meta(&para_id).most_recent_change().is_none());
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(expected_at));
assert_eq!(FutureCodeHash::<Test>::get(&para_id), Some(new_code.hash()));
assert_eq!(UpgradeGoAheadSignal::<Test>::get(&para_id), Some(UpgradeGoAhead::GoAhead));
assert_eq!(Paras::current_code(&para_id), Some(original_code.clone()));
check_code_is_stored(&original_code);
check_code_is_stored(&new_code);
}
run_to_block(expected_at + 1, None);
// the candidate is in the context of `expected_at`, and triggers
// the upgrade.
{
Paras::note_new_head(para_id, Default::default(), expected_at);
assert_eq!(Paras::past_code_meta(&para_id).most_recent_change(), Some(expected_at));
assert_eq!(
PastCodeHash::<Test>::get(&(para_id, expected_at)),
Some(original_code.hash()),
);
assert!(FutureCodeUpgrades::<Test>::get(&para_id).is_none());
assert!(FutureCodeHash::<Test>::get(&para_id).is_none());
assert!(UpgradeGoAheadSignal::<Test>::get(&para_id).is_none());
assert_eq!(Paras::current_code(&para_id), Some(new_code.clone()));
assert_eq!(
UpgradeRestrictionSignal::<Test>::get(&para_id),
Some(UpgradeRestriction::Present),
);
assert_eq!(UpgradeCooldowns::<Test>::get(), vec![(para_id, next_possible_upgrade_at)]);
check_code_is_stored(&original_code);
check_code_is_stored(&new_code);
}
run_to_block(next_possible_upgrade_at + 1, None);
{
assert!(UpgradeRestrictionSignal::<Test>::get(&para_id).is_none());
assert!(UpgradeCooldowns::<Test>::get().is_empty());
}
});
}
#[test]
fn code_upgrade_applied_after_delay_even_when_late() {
let code_retention_period = 10;
let validation_upgrade_delay = 5;
let validation_upgrade_cooldown = 10;
let original_code = ValidationCode(vec![1, 2, 3]);
let paras = vec![(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: original_code.clone(),
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
code_retention_period,
validation_upgrade_delay,
validation_upgrade_cooldown,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
let para_id = ParaId::from(0);
let new_code = ValidationCode(vec![4, 5, 6]);
// Wait for at least one session change to set active validators.
const EXPECTED_SESSION: SessionIndex = 1;
run_to_block(2, Some(vec![1]));
assert_eq!(Paras::current_code(&para_id), Some(original_code.clone()));
let expected_at = {
// this parablock is in the context of block 1.
let expected_at = 1 + validation_upgrade_delay;
let next_possible_upgrade_at = 1 + validation_upgrade_cooldown;
Paras::schedule_code_upgrade(para_id, new_code.clone(), 1, &Configuration::config());
// Include votes for super-majority.
submit_super_majority_pvf_votes(&new_code, EXPECTED_SESSION, true);
Paras::note_new_head(para_id, Default::default(), 1);
assert!(Paras::past_code_meta(&para_id).most_recent_change().is_none());
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(expected_at));
assert_eq!(FutureCodeHash::<Test>::get(&para_id), Some(new_code.hash()));
assert_eq!(UpcomingUpgrades::<Test>::get(), vec![(para_id, expected_at)]);
assert_eq!(UpgradeCooldowns::<Test>::get(), vec![(para_id, next_possible_upgrade_at)]);
assert!(UpgradeGoAheadSignal::<Test>::get(&para_id).is_none());
assert_eq!(Paras::current_code(&para_id), Some(original_code.clone()));
expected_at
};
run_to_block(expected_at + 1 + 4, None);
// the candidate is in the context of the first descendant of `expected_at`, and triggers
// the upgrade.
{
// The signal should be set to go-ahead until the new head is actually processed.
assert_eq!(UpgradeGoAheadSignal::<Test>::get(&para_id), Some(UpgradeGoAhead::GoAhead));
Paras::note_new_head(para_id, Default::default(), expected_at + 4);
assert_eq!(Paras::past_code_meta(&para_id).most_recent_change(), Some(expected_at));
assert_eq!(
PastCodeHash::<Test>::get(&(para_id, expected_at)),
Some(original_code.hash()),
);
assert!(FutureCodeUpgrades::<Test>::get(&para_id).is_none());
assert!(FutureCodeHash::<Test>::get(&para_id).is_none());
assert!(UpgradeGoAheadSignal::<Test>::get(&para_id).is_none());
assert_eq!(Paras::current_code(&para_id), Some(new_code.clone()));
}
});
}
#[test]
fn submit_code_change_when_not_allowed_is_err() {
let code_retention_period = 10;
let validation_upgrade_delay = 7;
let validation_upgrade_cooldown = 100;
let paras = vec![(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: vec![1, 2, 3].into(),
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
code_retention_period,
validation_upgrade_delay,
validation_upgrade_cooldown,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
let para_id = ParaId::from(0);
let new_code = ValidationCode(vec![4, 5, 6]);
let newer_code = ValidationCode(vec![4, 5, 6, 7]);
// Wait for at least one session change to set active validators.
const EXPECTED_SESSION: SessionIndex = 1;
run_to_block(1, Some(vec![1]));
Paras::schedule_code_upgrade(para_id, new_code.clone(), 1, &Configuration::config());
// Include votes for super-majority.
submit_super_majority_pvf_votes(&new_code, EXPECTED_SESSION, true);
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(1 + validation_upgrade_delay));
assert_eq!(FutureCodeHash::<Test>::get(&para_id), Some(new_code.hash()));
check_code_is_stored(&new_code);
// We expect that if an upgrade is signalled while there is already one pending we just
// ignore it. Note that this is only true from perspective of this module.
run_to_block(2, None);
assert!(!Paras::can_upgrade_validation_code(para_id));
Paras::schedule_code_upgrade(para_id, newer_code.clone(), 2, &Configuration::config());
assert_eq!(
FutureCodeUpgrades::<Test>::get(&para_id),
Some(1 + validation_upgrade_delay), /* did not change since the same assertion from
* the last time. */
);
assert_eq!(FutureCodeHash::<Test>::get(&para_id), Some(new_code.hash()));
check_code_is_not_stored(&newer_code);
});
}
#[test]
fn upgrade_restriction_elapsed_doesnt_mean_can_upgrade() {
// Situation: parachain scheduled upgrade but it doesn't produce any candidate after
// `expected_at`. When `validation_upgrade_cooldown` elapsed the parachain produces a
// candidate that tries to upgrade the code.
//
// In the current code this is not allowed: the upgrade should be consumed first. This is
// rather an artifact of the current implementation and not necessarily something we want
// to keep in the future.
//
// This test exists that this is not accidentally changed.
let code_retention_period = 10;
let validation_upgrade_delay = 7;
let validation_upgrade_cooldown = 30;
let paras = vec![(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: vec![1, 2, 3].into(),
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
code_retention_period,
validation_upgrade_delay,
validation_upgrade_cooldown,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
let para_id = 0u32.into();
let new_code = ValidationCode(vec![4, 5, 6]);
let newer_code = ValidationCode(vec![4, 5, 6, 7]);
// Wait for at least one session change to set active validators.
const EXPECTED_SESSION: SessionIndex = 1;
run_to_block(1, Some(vec![1]));
Paras::schedule_code_upgrade(para_id, new_code.clone(), 0, &Configuration::config());
// Include votes for super-majority.
submit_super_majority_pvf_votes(&new_code, EXPECTED_SESSION, true);
Paras::note_new_head(para_id, dummy_head_data(), 0);
assert_eq!(
UpgradeRestrictionSignal::<Test>::get(&para_id),
Some(UpgradeRestriction::Present),
);
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(0 + validation_upgrade_delay));
assert!(!Paras::can_upgrade_validation_code(para_id));
run_to_block(31, None);
assert!(UpgradeRestrictionSignal::<Test>::get(&para_id).is_none());
// Note the para still cannot upgrade the validation code.
assert!(!Paras::can_upgrade_validation_code(para_id));
// And scheduling another upgrade does not do anything. `expected_at` is still the same.
Paras::schedule_code_upgrade(para_id, newer_code.clone(), 30, &Configuration::config());
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(0 + validation_upgrade_delay));
});
}
#[test]
fn full_parachain_cleanup_storage() {
let code_retention_period = 20;
let validation_upgrade_delay = 1 + 5;
let original_code = ValidationCode(vec![1, 2, 3]);
let paras = vec![(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: original_code.clone(),
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
code_retention_period,
validation_upgrade_delay,
minimum_validation_upgrade_delay: 2,
// Those are not relevant to this test. However, HostConfiguration is still a
// subject for the consistency check.
paras_availability_period: 1,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
check_code_is_stored(&original_code);
let para_id = ParaId::from(0);
let new_code = ValidationCode(vec![4, 5, 6]);
// Wait for at least one session change to set active validators.
const EXPECTED_SESSION: SessionIndex = 1;
run_to_block(2, Some(vec![1]));
assert_eq!(Paras::current_code(&para_id), Some(original_code.clone()));
check_code_is_stored(&original_code);
let expected_at = {
// this parablock is in the context of block 1.
let expected_at = 1 + validation_upgrade_delay;
Paras::schedule_code_upgrade(para_id, new_code.clone(), 1, &Configuration::config());
// Include votes for super-majority.
submit_super_majority_pvf_votes(&new_code, EXPECTED_SESSION, true);
Paras::note_new_head(para_id, Default::default(), 1);
assert!(Paras::past_code_meta(&para_id).most_recent_change().is_none());
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(expected_at));
assert_eq!(FutureCodeHash::<Test>::get(&para_id), Some(new_code.hash()));
assert_eq!(Paras::current_code(&para_id), Some(original_code.clone()));
check_code_is_stored(&original_code);
check_code_is_stored(&new_code);
expected_at
};
// Enact the upgrade.
//
// For that run to block #7 and submit a new head.
assert_eq!(expected_at, 7);
run_to_block(7, None);
assert_eq!(<frame_system::Pallet<Test>>::block_number(), 7);
Paras::note_new_head(para_id, Default::default(), expected_at);
assert_ok!(Paras::schedule_para_cleanup(para_id));
// run to block #10, with a 2 session changes at the end of the block 7 & 8 (so 8 and 9
// observe the new sessions).
run_to_block(10, Some(vec![8, 9]));
// cleaning up the parachain should place the current parachain code
// into the past code buffer & schedule cleanup.
//
// Why 7 and 8? See above, the clean up scheduled above was processed at the block 8.
// The initial upgrade was enacted at the block 7.
assert_eq!(Paras::past_code_meta(&para_id).most_recent_change(), Some(8));
assert_eq!(PastCodeHash::<Test>::get(&(para_id, 8)), Some(new_code.hash()));
assert_eq!(PastCodePruning::<Test>::get(), vec![(para_id, 7), (para_id, 8)]);
check_code_is_stored(&original_code);
check_code_is_stored(&new_code);
// any future upgrades haven't been used to validate yet, so those
// are cleaned up immediately.
assert!(FutureCodeUpgrades::<Test>::get(&para_id).is_none());
assert!(FutureCodeHash::<Test>::get(&para_id).is_none());
assert!(Paras::current_code(&para_id).is_none());
// run to do the final cleanup
let cleaned_up_at = 8 + code_retention_period + 1;
run_to_block(cleaned_up_at, None);
// now the final cleanup: last past code cleaned up, and this triggers meta cleanup.
assert_eq!(Paras::past_code_meta(&para_id), Default::default());
assert!(PastCodeHash::<Test>::get(&(para_id, 7)).is_none());
assert!(PastCodeHash::<Test>::get(&(para_id, 8)).is_none());
assert!(PastCodePruning::<Test>::get().is_empty());
check_code_is_not_stored(&original_code);
check_code_is_not_stored(&new_code);
});
}
#[test]
fn cannot_offboard_ongoing_pvf_check() {
let para_id = ParaId::from(0);
let existing_code: ValidationCode = vec![1, 2, 3].into();
let new_code: ValidationCode = vec![3, 2, 1].into();
let paras = vec![(
para_id,
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: Default::default(),
validation_code: existing_code,
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
run_to_block(2, Some(vec![1]));
// Relay parent of the block that schedules the upgrade.
const RELAY_PARENT: BlockNumber = 1;
// Expected current session index.
const EXPECTED_SESSION: SessionIndex = 1;
Paras::schedule_code_upgrade(
para_id,
new_code.clone(),
RELAY_PARENT,
&Configuration::config(),
);
assert!(!Paras::pvfs_require_precheck().is_empty());
// Cannot offboard when there's an ongoing pvf-check voting.
assert_err!(Paras::schedule_para_cleanup(para_id), Error::<Test>::CannotOffboard);
// Include votes for super-majority.
submit_super_majority_pvf_votes(&new_code, EXPECTED_SESSION, true);
// Voting concluded, can offboard even though an upgrade is in progress.
assert_ok!(Paras::schedule_para_cleanup(para_id));
});
}
#[test]
fn para_incoming_at_session() {
let code_a = ValidationCode(vec![2]);
let code_b = ValidationCode(vec![1]);
let code_c = ValidationCode(vec![3]);
let genesis_config = MockGenesisConfig::default();
new_test_ext(genesis_config).execute_with(|| {
run_to_block(1, Some(vec![1]));
let b = ParaId::from(525);
let a = ParaId::from(999);
let c = ParaId::from(333);
assert_ok!(Paras::schedule_para_initialize(
b,
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: vec![1].into(),
validation_code: code_b.clone(),
},
));
assert_ok!(Paras::schedule_para_initialize(
a,
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: vec![2].into(),
validation_code: code_a.clone(),
},
));
assert_ok!(Paras::schedule_para_initialize(
c,
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: vec![3].into(),
validation_code: code_c.clone(),
},
));
IntoIterator::into_iter([0, 1, 2, 3])
.map(|i| PvfCheckStatement {
accept: true,
subject: code_a.hash(),
session_index: 1,
validator_index: i.into(),
})
.for_each(sign_and_include_pvf_check_statement);
IntoIterator::into_iter([1, 2, 3, 4])
.map(|i| PvfCheckStatement {
accept: true,
subject: code_b.hash(),
session_index: 1,
validator_index: i.into(),
})
.for_each(sign_and_include_pvf_check_statement);
IntoIterator::into_iter([0, 2, 3, 4])
.map(|i| PvfCheckStatement {
accept: true,
subject: code_c.hash(),
session_index: 1,
validator_index: i.into(),
})
.for_each(sign_and_include_pvf_check_statement);
assert_eq!(ActionsQueue::<Test>::get(Paras::scheduled_session()), vec![c, b, a],);
// Lifecycle is tracked correctly
assert_eq!(ParaLifecycles::<Test>::get(&a), Some(ParaLifecycle::Onboarding));
assert_eq!(ParaLifecycles::<Test>::get(&b), Some(ParaLifecycle::Onboarding));
assert_eq!(ParaLifecycles::<Test>::get(&c), Some(ParaLifecycle::Onboarding));
// run to block without session change.
run_to_block(2, None);
assert_eq!(Paras::parachains(), Vec::new());
assert_eq!(ActionsQueue::<Test>::get(Paras::scheduled_session()), vec![c, b, a],);
// Lifecycle is tracked correctly
assert_eq!(ParaLifecycles::<Test>::get(&a), Some(ParaLifecycle::Onboarding));
assert_eq!(ParaLifecycles::<Test>::get(&b), Some(ParaLifecycle::Onboarding));
assert_eq!(ParaLifecycles::<Test>::get(&c), Some(ParaLifecycle::Onboarding));
// Two sessions pass, so action queue is triggered
run_to_block(4, Some(vec![3, 4]));
assert_eq!(Paras::parachains(), vec![c, b]);
assert_eq!(ActionsQueue::<Test>::get(Paras::scheduled_session()), Vec::new());
// Lifecycle is tracked correctly
assert_eq!(ParaLifecycles::<Test>::get(&a), Some(ParaLifecycle::Parathread));
assert_eq!(ParaLifecycles::<Test>::get(&b), Some(ParaLifecycle::Parachain));
assert_eq!(ParaLifecycles::<Test>::get(&c), Some(ParaLifecycle::Parachain));
assert_eq!(Paras::current_code(&a), Some(vec![2].into()));
assert_eq!(Paras::current_code(&b), Some(vec![1].into()));
assert_eq!(Paras::current_code(&c), Some(vec![3].into()));
})
}
#[test]
fn code_hash_at_returns_up_to_end_of_code_retention_period() {
let code_retention_period = 10;
let validation_upgrade_delay = 2;
let paras = vec![(
0u32.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: dummy_head_data(),
validation_code: vec![1, 2, 3].into(),
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
code_retention_period,
validation_upgrade_delay,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
// Wait for at least one session change to set active validators.
run_to_block(2, Some(vec![1]));
const EXPECTED_SESSION: SessionIndex = 1;
let para_id = ParaId::from(0);
let old_code: ValidationCode = vec![1, 2, 3].into();
let new_code: ValidationCode = vec![4, 5, 6].into();
Paras::schedule_code_upgrade(para_id, new_code.clone(), 0, &Configuration::config());
// Include votes for super-majority.
submit_super_majority_pvf_votes(&new_code, EXPECTED_SESSION, true);
// The new validation code can be applied but a new parablock hasn't gotten in yet,
// so the old code should still be current.
run_to_block(3, None);
assert_eq!(Paras::current_code(&para_id), Some(old_code.clone()));
run_to_block(10, None);
Paras::note_new_head(para_id, Default::default(), 7);
assert_eq!(Paras::past_code_meta(&para_id).upgrade_times, vec![upgrade_at(4, 10)]);
assert_eq!(Paras::current_code(&para_id), Some(new_code.clone()));
// Make sure that the old code is available **before** the code retion period passes.
run_to_block(10 + code_retention_period, None);
assert_eq!(Paras::code_by_hash(&old_code.hash()), Some(old_code.clone()));
assert_eq!(Paras::code_by_hash(&new_code.hash()), Some(new_code.clone()));
run_to_block(10 + code_retention_period + 1, None);
// code entry should be pruned now.
assert_eq!(
Paras::past_code_meta(&para_id),
ParaPastCodeMeta { upgrade_times: Vec::new(), last_pruned: Some(10) },
);
assert_eq!(Paras::code_by_hash(&old_code.hash()), None); // pruned :(
assert_eq!(Paras::code_by_hash(&new_code.hash()), Some(new_code.clone()));
});
}
#[test]
fn code_ref_is_cleaned_correctly() {
new_test_ext(Default::default()).execute_with(|| {
let code: ValidationCode = vec![1, 2, 3].into();
Paras::increase_code_ref(&code.hash(), &code);
Paras::increase_code_ref(&code.hash(), &code);
assert!(CodeByHash::<Test>::contains_key(code.hash()));
assert_eq!(CodeByHashRefs::<Test>::get(code.hash()), 2);
Paras::decrease_code_ref(&code.hash());
assert!(CodeByHash::<Test>::contains_key(code.hash()));
assert_eq!(CodeByHashRefs::<Test>::get(code.hash()), 1);
Paras::decrease_code_ref(&code.hash());
assert!(!CodeByHash::<Test>::contains_key(code.hash()));
assert!(!CodeByHashRefs::<Test>::contains_key(code.hash()));
});
}
#[test]
fn pvf_check_coalescing_onboarding_and_upgrade() {
let validation_upgrade_delay = 5;
let a = ParaId::from(111);
let b = ParaId::from(222);
let existing_code: ValidationCode = vec![1, 2, 3].into();
let validation_code: ValidationCode = vec![3, 2, 1].into();
let paras = vec![(
a,
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: Default::default(),
validation_code: existing_code,
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration { validation_upgrade_delay, ..Default::default() },
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
// At this point `a` is already onboarded. Run to block 1 performing session change at
// the end of block #0.
run_to_block(2, Some(vec![1]));
// Expected current session index.
const EXPECTED_SESSION: SessionIndex = 1;
// Relay parent of the parablock that schedules the upgrade.
const RELAY_PARENT: BlockNumber = 1;
// Now we register `b` with `validation_code`
assert_ok!(Paras::schedule_para_initialize(
b,
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: vec![2].into(),
validation_code: validation_code.clone(),
},
));
// And now at the same time upgrade `a` to `validation_code`
Paras::schedule_code_upgrade(
a,
validation_code.clone(),
RELAY_PARENT,
&Configuration::config(),
);
assert!(!Paras::pvfs_require_precheck().is_empty());
// Supermajority of validators vote for `validation_code`. It should be approved.
submit_super_majority_pvf_votes(&validation_code, EXPECTED_SESSION, true);
// Check that `b` actually onboards.
assert_eq!(ActionsQueue::<Test>::get(EXPECTED_SESSION + 2), vec![b]);
// Check that the upgrade got scheduled.
assert_eq!(
FutureCodeUpgrades::<Test>::get(&a),
Some(RELAY_PARENT + validation_upgrade_delay),
);
// Verify that the required events were emitted.
EventValidator::new()
.started(&validation_code, b)
.started(&validation_code, a)
.accepted(&validation_code, b)
.accepted(&validation_code, a)
.check();
});
}
#[test]
fn pvf_check_onboarding_reject_on_expiry() {
let pvf_voting_ttl = 2;
let a = ParaId::from(111);
let validation_code: ValidationCode = vec![3, 2, 1].into();
let genesis_config = MockGenesisConfig {
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration { pvf_voting_ttl, ..Default::default() },
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
run_to_block(1, Some(vec![1]));
assert_ok!(Paras::schedule_para_initialize(
a,
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: vec![2].into(),
validation_code: validation_code.clone(),
},
));
// Make sure that we kicked off the PVF vote for this validation code and that the
// validation code is stored.
assert!(PvfActiveVoteMap::<Test>::get(&validation_code.hash()).is_some());
check_code_is_stored(&validation_code);
// Skip 2 sessions (i.e. `pvf_voting_ttl`) verifying that the code is still stored in
// the intermediate session.
assert_eq!(pvf_voting_ttl, 2);
run_to_block(2, Some(vec![2]));
check_code_is_stored(&validation_code);
run_to_block(3, Some(vec![3]));
// --- At this point the PVF vote for onboarding should be rejected.
// Verify that the PVF is no longer stored and there is no active PVF vote.
check_code_is_not_stored(&validation_code);
assert!(PvfActiveVoteMap::<Test>::get(&validation_code.hash()).is_none());
assert!(Paras::pvfs_require_precheck().is_empty());
// Verify that at this point we can again try to initialize the same para.
assert!(Paras::can_schedule_para_initialize(&a));
});
}
#[test]
fn pvf_check_upgrade_reject() {
let a = ParaId::from(111);
let old_code: ValidationCode = vec![1, 2, 3].into();
let new_code: ValidationCode = vec![3, 2, 1].into();
let paras = vec![(
a,
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: Default::default(),
validation_code: old_code,
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
// At this point `a` is already onboarded. Run to block 1 performing session change at
// the end of block #0.
run_to_block(2, Some(vec![1]));
// Relay parent of the block that schedules the upgrade.
const RELAY_PARENT: BlockNumber = 1;
// Expected current session index.
const EXPECTED_SESSION: SessionIndex = 1;
Paras::schedule_code_upgrade(a, new_code.clone(), RELAY_PARENT, &Configuration::config());
check_code_is_stored(&new_code);
// 1/3 of validators vote against `new_code`. PVF should not be rejected yet.
sign_and_include_pvf_check_statement(PvfCheckStatement {
accept: false,
subject: new_code.hash(),
session_index: EXPECTED_SESSION,
validator_index: 0.into(),
});
// Verify that the new code is not yet discarded.
check_code_is_stored(&new_code);
// >1/3 of validators vote against `new_code`. PVF should be rejected.
sign_and_include_pvf_check_statement(PvfCheckStatement {
accept: false,
subject: new_code.hash(),
session_index: EXPECTED_SESSION,
validator_index: 1.into(),
});
// Verify that the new code is discarded.
check_code_is_not_stored(&new_code);
assert!(PvfActiveVoteMap::<Test>::get(&new_code.hash()).is_none());
assert!(Paras::pvfs_require_precheck().is_empty());
assert!(FutureCodeHash::<Test>::get(&a).is_none());
// Verify that the required events were emitted.
EventValidator::new().started(&new_code, a).rejected(&new_code, a).check();
});
}
#[test]
fn pvf_check_submit_vote() {
let code_a: ValidationCode = vec![3, 2, 1].into();
let code_b: ValidationCode = vec![1, 2, 3].into();
let check = |stmt: PvfCheckStatement| -> (Result<_, _>, Result<_, _>) {
let validators = &[
Sr25519Keyring::Alice,
Sr25519Keyring::Bob,
Sr25519Keyring::Charlie,
Sr25519Keyring::Dave,
Sr25519Keyring::Ferdie,
Sr25519Keyring::Eve, // <- this validator is not in the set
];
let signature: ValidatorSignature =
validators[stmt.validator_index.0 as usize].sign(&stmt.signing_payload()).into();
let call =
Call::include_pvf_check_statement { stmt: stmt.clone(), signature: signature.clone() };
let validate_unsigned =
<Paras as ValidateUnsigned>::validate_unsigned(TransactionSource::InBlock, &call)
.map(|_| ());
let dispatch_result =
Paras::include_pvf_check_statement(None.into(), stmt.clone(), signature.clone())
.map(|_| ());
(validate_unsigned, dispatch_result)
};
let genesis_config = MockGenesisConfig::default();
new_test_ext(genesis_config).execute_with(|| {
// Important to run this to seed the validators.
run_to_block(1, Some(vec![1]));
assert_ok!(Paras::schedule_para_initialize(
1000.into(),
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: vec![2].into(),
validation_code: code_a.clone(),
},
));
assert_eq!(
check(PvfCheckStatement {
accept: false,
subject: code_a.hash(),
session_index: 1,
validator_index: 1.into(),
}),
(Ok(()), Ok(())),
);
// A vote in the same direction.
let (unsigned, dispatch) = check(PvfCheckStatement {
accept: false,
subject: code_a.hash(),
session_index: 1,
validator_index: 1.into(),
});
assert_eq!(unsigned, Err(InvalidTransaction::Custom(INVALID_TX_DOUBLE_VOTE).into()));
assert_err!(dispatch, Error::<Test>::PvfCheckDoubleVote);
// Equivocation
let (unsigned, dispatch) = check(PvfCheckStatement {
accept: true,
subject: code_a.hash(),
session_index: 1,
validator_index: 1.into(),
});
assert_eq!(unsigned, Err(InvalidTransaction::Custom(INVALID_TX_DOUBLE_VOTE).into()));
assert_err!(dispatch, Error::<Test>::PvfCheckDoubleVote);
// Vote for an earlier session.
let (unsigned, dispatch) = check(PvfCheckStatement {
accept: false,
subject: code_a.hash(),
session_index: 0,
validator_index: 1.into(),
});
assert_eq!(unsigned, Err(InvalidTransaction::Stale.into()));
assert_err!(dispatch, Error::<Test>::PvfCheckStatementStale);
// Vote for an later session.
let (unsigned, dispatch) = check(PvfCheckStatement {
accept: false,
subject: code_a.hash(),
session_index: 2,
validator_index: 1.into(),
});
assert_eq!(unsigned, Err(InvalidTransaction::Future.into()));
assert_err!(dispatch, Error::<Test>::PvfCheckStatementFuture);
// Validator not in the set.
let (unsigned, dispatch) = check(PvfCheckStatement {
accept: false,
subject: code_a.hash(),
session_index: 1,
validator_index: 5.into(),
});
assert_eq!(unsigned, Err(InvalidTransaction::Custom(INVALID_TX_BAD_VALIDATOR_IDX).into()));
assert_err!(dispatch, Error::<Test>::PvfCheckValidatorIndexOutOfBounds);
// Bad subject (code_b)
let (unsigned, dispatch) = check(PvfCheckStatement {
accept: false,
subject: code_b.hash(),
session_index: 1,
validator_index: 1.into(),
});
assert_eq!(unsigned, Err(InvalidTransaction::Custom(INVALID_TX_BAD_SUBJECT).into()));
assert_err!(dispatch, Error::<Test>::PvfCheckSubjectInvalid);
});
}
#[test]
fn include_pvf_check_statement_refunds_weight() {
let a = ParaId::from(111);
let old_code: ValidationCode = vec![1, 2, 3].into();
let new_code: ValidationCode = vec![3, 2, 1].into();
let paras = vec![(
a,
ParaGenesisArgs {
para_kind: ParaKind::Parathread,
genesis_head: Default::default(),
validation_code: old_code,
},
)];
let genesis_config = MockGenesisConfig {
paras: GenesisConfig { paras, ..Default::default() },
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
// At this point `a` is already onboarded. Run to block 1 performing session change at
// the end of block #0.
run_to_block(2, Some(vec![1]));
// Relay parent of the block that schedules the upgrade.
const RELAY_PARENT: BlockNumber = 1;
// Expected current session index.
const EXPECTED_SESSION: SessionIndex = 1;
Paras::schedule_code_upgrade(a, new_code.clone(), RELAY_PARENT, &Configuration::config());
let mut stmts = IntoIterator::into_iter([0, 1, 2, 3])
.map(|i| {
let stmt = PvfCheckStatement {
accept: true,
subject: new_code.hash(),
session_index: EXPECTED_SESSION,
validator_index: (i as u32).into(),
};
let sig = VALIDATORS[i].sign(&stmt.signing_payload());
(stmt, sig)
})
.collect::<Vec<_>>();
let last_one = stmts.pop().unwrap();
// Verify that just vote submission is priced accordingly.
for (stmt, sig) in stmts {
let r = Paras::include_pvf_check_statement(None.into(), stmt, sig.into()).unwrap();
assert_eq!(r.actual_weight, Some(TestWeightInfo::include_pvf_check_statement()));
}
// Verify that the last statement is priced maximally.
let (stmt, sig) = last_one;
let r = Paras::include_pvf_check_statement(None.into(), stmt, sig.into()).unwrap();
assert_eq!(r.actual_weight, None);
});
}
#[test]
fn add_trusted_validation_code_inserts_with_no_users() {
// This test is to ensure that trusted validation code is inserted into the storage
// with the reference count equal to 0.
let validation_code = ValidationCode(vec![1, 2, 3]);
new_test_ext(Default::default()).execute_with(|| {
assert_ok!(Paras::add_trusted_validation_code(
RuntimeOrigin::root(),
validation_code.clone()
));
assert_eq!(CodeByHashRefs::<Test>::get(&validation_code.hash()), 0,);
});
}
#[test]
fn add_trusted_validation_code_idempotent() {
// This test makes sure that calling add_trusted_validation_code twice with the same
// parameters is a no-op.
let validation_code = ValidationCode(vec![1, 2, 3]);
new_test_ext(Default::default()).execute_with(|| {
assert_ok!(Paras::add_trusted_validation_code(
RuntimeOrigin::root(),
validation_code.clone()
));
assert_storage_noop!({
assert_ok!(Paras::add_trusted_validation_code(
RuntimeOrigin::root(),
validation_code.clone()
));
});
});
}
#[test]
fn poke_unused_validation_code_removes_code_cleanly() {
// This test makes sure that calling poke_unused_validation_code with a code that is currently
// in the storage but has no users will remove it cleanly from the storage.
let validation_code = ValidationCode(vec![1, 2, 3]);
new_test_ext(Default::default()).execute_with(|| {
assert_ok!(Paras::add_trusted_validation_code(
RuntimeOrigin::root(),
validation_code.clone()
));
assert_ok!(Paras::poke_unused_validation_code(
RuntimeOrigin::root(),
validation_code.hash()
));
assert_eq!(CodeByHashRefs::<Test>::get(&validation_code.hash()), 0);
assert!(!CodeByHash::<Test>::contains_key(&validation_code.hash()));
});
}
#[test]
fn poke_unused_validation_code_doesnt_remove_code_with_users() {
let para_id = 100.into();
let validation_code = ValidationCode(vec![1, 2, 3]);
new_test_ext(Default::default()).execute_with(|| {
// First we add the code to the storage.
assert_ok!(Paras::add_trusted_validation_code(
RuntimeOrigin::root(),
validation_code.clone()
));
// Then we add a user to the code, say by upgrading.
run_to_block(2, None);
Paras::schedule_code_upgrade(para_id, validation_code.clone(), 1, &Configuration::config());
Paras::note_new_head(para_id, HeadData::default(), 1);
// Finally we poke the code, which should not remove it from the storage.
assert_storage_noop!({
assert_ok!(Paras::poke_unused_validation_code(
RuntimeOrigin::root(),
validation_code.hash()
));
});
check_code_is_stored(&validation_code);
});
}
#[test]
fn increase_code_ref_doesnt_have_allergy_on_add_trusted_validation_code() {
// Verify that accidential calling of increase_code_ref or decrease_code_ref does not lead
// to a disaster.
// NOTE that this test is extra paranoid, as it is not really possible to hit
// `decrease_code_ref` without calling `increase_code_ref` first.
let code = ValidationCode(vec![1, 2, 3]);
new_test_ext(Default::default()).execute_with(|| {
assert_ok!(Paras::add_trusted_validation_code(RuntimeOrigin::root(), code.clone()));
Paras::increase_code_ref(&code.hash(), &code);
Paras::increase_code_ref(&code.hash(), &code);
assert!(CodeByHash::<Test>::contains_key(code.hash()));
assert_eq!(CodeByHashRefs::<Test>::get(code.hash()), 2);
});
new_test_ext(Default::default()).execute_with(|| {
assert_ok!(Paras::add_trusted_validation_code(RuntimeOrigin::root(), code.clone()));
Paras::decrease_code_ref(&code.hash());
assert!(CodeByHash::<Test>::contains_key(code.hash()));
assert_eq!(CodeByHashRefs::<Test>::get(code.hash()), 0);
});
}
#[test]
fn add_trusted_validation_code_insta_approval() {
// In particular, this tests that `kick_off_pvf_check` reacts to the
// `add_trusted_validation_code` and uses the `CodeByHash::contains_key` which is what
// `add_trusted_validation_code` uses.
let para_id = 100.into();
let validation_code = ValidationCode(vec![1, 2, 3]);
let validation_upgrade_delay = 25;
let minimum_validation_upgrade_delay = 2;
let genesis_config = MockGenesisConfig {
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
validation_upgrade_delay,
minimum_validation_upgrade_delay,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
assert_ok!(Paras::add_trusted_validation_code(
RuntimeOrigin::root(),
validation_code.clone()
));
// Then some parachain upgrades it's code with the relay-parent 1.
run_to_block(2, None);
Paras::schedule_code_upgrade(para_id, validation_code.clone(), 1, &Configuration::config());
Paras::note_new_head(para_id, HeadData::default(), 1);
// Verify that the code upgrade has `expected_at` set to `26`.
assert_eq!(FutureCodeUpgrades::<Test>::get(&para_id), Some(1 + validation_upgrade_delay));
// Verify that the required events were emitted.
EventValidator::new()
.started(&validation_code, para_id)
.accepted(&validation_code, para_id)
.check();
});
}
#[test]
fn add_trusted_validation_code_enacts_existing_pvf_vote() {
// This test makes sure that calling `add_trusted_validation_code` with a code that is
// already going through PVF pre-checking voting will conclude the voting and enact the
// code upgrade.
let para_id = 100.into();
let validation_code = ValidationCode(vec![1, 2, 3]);
let validation_upgrade_delay = 25;
let minimum_validation_upgrade_delay = 2;
let genesis_config = MockGenesisConfig {
configuration: crate::configuration::GenesisConfig {
config: HostConfiguration {
validation_upgrade_delay,
minimum_validation_upgrade_delay,
..Default::default()
},
},
..Default::default()
};
new_test_ext(genesis_config).execute_with(|| {
// First, some parachain upgrades it's code with the relay-parent 1.
run_to_block(2, None);
Paras::schedule_code_upgrade(para_id, validation_code.clone(), 1, &Configuration::config());
Paras::note_new_head(para_id, HeadData::default(), 1);
// No upgrade should be scheduled at this point. PVF pre-checking vote should run for
// that PVF.
assert!(FutureCodeUpgrades::<Test>::get(&para_id).is_none());
assert!(PvfActiveVoteMap::<Test>::contains_key(&validation_code.hash()));
// Then we add a trusted validation code. That should conclude the vote.
assert_ok!(Paras::add_trusted_validation_code(
RuntimeOrigin::root(),
validation_code.clone()
));
assert!(FutureCodeUpgrades::<Test>::get(&para_id).is_some());
assert!(!PvfActiveVoteMap::<Test>::contains_key(&validation_code.hash()));
});
}
#[test]
fn verify_upgrade_go_ahead_signal_is_externally_accessible() {
use primitives::well_known_keys;
let a = ParaId::from(2020);
new_test_ext(Default::default()).execute_with(|| {
assert!(sp_io::storage::get(&well_known_keys::upgrade_go_ahead_signal(a)).is_none());
UpgradeGoAheadSignal::<Test>::insert(&a, UpgradeGoAhead::GoAhead);
assert_eq!(
sp_io::storage::get(&well_known_keys::upgrade_go_ahead_signal(a)).unwrap(),
vec![1u8],
);
});
}
#[test]
fn verify_upgrade_restriction_signal_is_externally_accessible() {
use primitives::well_known_keys;
let a = ParaId::from(2020);
new_test_ext(Default::default()).execute_with(|| {
assert!(sp_io::storage::get(&well_known_keys::upgrade_restriction_signal(a)).is_none());
UpgradeRestrictionSignal::<Test>::insert(&a, UpgradeRestriction::Present);
assert_eq!(
sp_io::storage::get(&well_known_keys::upgrade_restriction_signal(a)).unwrap(),
vec![0],
);
});
}
#[test]
fn verify_para_head_is_externally_accessible() {
use primitives::well_known_keys;
let a = ParaId::from(2020);
let expected_head_data = HeadData(vec![0, 1, 2, 3]);
new_test_ext(Default::default()).execute_with(|| {
Heads::<Test>::insert(&a, expected_head_data.clone());
let encoded = sp_io::storage::get(&well_known_keys::para_head(a)).unwrap();
let head_data = HeadData::decode(&mut encoded.as_ref());
assert_eq!(head_data, Ok(expected_head_data));
});
}
#[test]
fn most_recent_context() {
let validation_code: ValidationCode = vec![1, 2, 3].into();
let genesis_config = MockGenesisConfig::default();
new_test_ext(genesis_config).execute_with(|| {
const EXPECTED_SESSION: SessionIndex = 1;
run_to_block(1, Some(vec![1]));
let para_id = ParaId::from(111);
assert_eq!(Paras::para_most_recent_context(para_id), None);
assert_ok!(Paras::schedule_para_initialize(
para_id,
ParaGenesisArgs {
para_kind: ParaKind::Parachain,
genesis_head: vec![1].into(),
validation_code: validation_code.clone(),
},
));
submit_super_majority_pvf_votes(&validation_code, EXPECTED_SESSION, true);
assert_eq!(ParaLifecycles::<Test>::get(&para_id), Some(ParaLifecycle::Onboarding));
// Two sessions pass, so action queue is triggered.
run_to_block(4, Some(vec![3, 4]));
// Double-check the para is onboarded, the context is set to the recent block.
assert_eq!(ParaLifecycles::<Test>::get(&para_id), Some(ParaLifecycle::Parachain));
assert_eq!(Paras::para_most_recent_context(para_id), Some(0));
// Progress para to the new head and check that the recent context is updated.
Paras::note_new_head(para_id, vec![4, 5, 6].into(), 3);
assert_eq!(Paras::para_most_recent_context(para_id), Some(3));
// Finally, offboard the para and expect the context to be cleared.
assert_ok!(Paras::schedule_para_cleanup(para_id));
run_to_block(6, Some(vec![5, 6]));
assert_eq!(Paras::para_most_recent_context(para_id), None);
})
}
#[test]
fn parakind_encodes_decodes_to_bool_scale() {
let chain_kind = ParaKind::Parachain.encode();
let chain_bool = true.encode();
assert_eq!(chain_kind, chain_bool);
let chain_dec = ParaKind::decode(&mut chain_kind.as_slice());
assert_eq!(chain_dec, Ok(ParaKind::Parachain));
let thread_kind = ParaKind::Parathread.encode();
let thread_bool = false.encode();
assert_eq!(thread_kind, thread_bool);
let thread_dec = ParaKind::decode(&mut thread_kind.as_slice());
assert_eq!(thread_dec, Ok(ParaKind::Parathread));
assert_eq!(bool::type_info(), ParaKind::type_info());
}
#[test]
fn parakind_encodes_decodes_to_bool_serde() {
let chain = ParaKind::Parachain;
let ser_chain = serde_json::to_string(&ParaKind::Parachain).unwrap();
let de_chain: ParaKind = serde_json::from_str(&ser_chain).unwrap();
assert_eq!(chain, de_chain);
let ser_true = serde_json::to_string(&true).unwrap();
assert_eq!(ser_true, ser_chain);
let thread = ParaKind::Parathread;
let ser_thread = serde_json::to_string(&thread).unwrap();
let de_thread: ParaKind = serde_json::from_str(&ser_thread).unwrap();
assert_eq!(thread, de_thread);
let ser_false = serde_json::to_string(&false).unwrap();
assert_eq!(ser_false, ser_thread);
}
#[test]
fn parachains_cache_is_set() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let a = ParaId::from(111);
let mut parachains_cache: ParachainsCache<Test> = ParachainsCache::new();
// Add element twice
parachains_cache.add(a);
parachains_cache.add(a);
// Flush cache to storage
drop(parachains_cache);
// In order after addition
assert_eq!(Parachains::<Test>::get(), vec![a]);
let mut parachains_cache: ParachainsCache<Test> = ParachainsCache::new();
// Remove element twice
parachains_cache.remove(a);
parachains_cache.remove(a);
// Flush cache to storage
drop(parachains_cache);
// In order after removal
assert_eq!(Parachains::<Test>::get(), vec![]);
let mut parachains_cache: ParachainsCache<Test> = ParachainsCache::new();
// Remove nonexisting element
parachains_cache.remove(a);
assert_storage_noop!(drop(parachains_cache));
assert_eq!(Parachains::<Test>::get(), vec![]);
});
}
#[test]
fn parachains_cache_preserves_order() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let a = ParaId::from(111);
let b = ParaId::from(222);
let c = ParaId::from(333);
let d = ParaId::from(444);
let mut parachains_cache: ParachainsCache<Test> = ParachainsCache::new();
// Add in mixed order
parachains_cache.add(b);
parachains_cache.add(c);
parachains_cache.add(a);
parachains_cache.add(d);
// Flush cache to storage
drop(parachains_cache);
// In order after addition
assert_eq!(Parachains::<Test>::get(), vec![a, b, c, d]);
let mut parachains_cache: ParachainsCache<Test> = ParachainsCache::new();
// Remove 2 elements
parachains_cache.remove(b);
parachains_cache.remove(d);
// Flush cache to storage
drop(parachains_cache);
// In order after removal
assert_eq!(Parachains::<Test>::get(), vec![a, c]);
});
}