Asynchronous Backing MegaPR (#5022)

* inclusion emulator logic for asynchronous backing (#4790)

* initial stab at candidate_context

* fmt

* docs & more TODOs

* some cleanups

* reframe as inclusion_emulator

* documentations yes

* update types

* add constraint modifications

* watermark

* produce modifications

* v2 primitives: re-export all v1 for consistency

* vstaging primitives

* emulator constraints: handle code upgrades

* produce outbound HRMP modifications

* stack.

* method for applying modifications

* method just for sanity-checking modifications

* fragments produce modifications, not prospectives

* make linear

* add some TODOs

* remove stacking; handle code upgrades

* take `fragment` private

* reintroduce stacking.

* fragment constructor

* add TODO

* allow validating fragments against future constraints

* docs

* relay-parent number and min code size checks

* check code upgrade restriction

* check max hrmp per candidate

* fmt

* remove GoAhead logic because it wasn't helpful

* docs on code upgrade failure

* test stacking

* test modifications against constraints

* fmt

* test fragments

* descending or duplicate test

* fmt

* remove unused imports in vstaging

* wrong primitives

* spellcheck

* Runtime changes for Asynchronous Backing (#4786)

* inclusion: utility for allowed relay-parents

* inclusion: use prev number instead of prev hash

* track most recent context of paras

* inclusion: accept previous relay-parents

* update dmp  advancement rule for async backing

* fmt

* add a comment about validation outputs

* clean up a couple of TODOs

* weights

* fix weights

* fmt

* Resolve dmp todo

* Restore inclusion tests

* Restore paras_inherent tests

* MostRecentContext test

* Benchmark for new paras dispatchable

* Prepare check_validation_outputs for upgrade

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Implementers guide changes

* More tests for allowed relay parents

* Add a github issue link

* Compute group index based on relay parent

* Storage migration

* Move allowed parents tracker to shared

* Compile error

* Get group assigned to core at the next block

* Test group assignment

* fmt

* Error instead of panic

* Update guide

* Extend doc-comment

* Update runtime/parachains/src/shared.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Prospective Parachains Subsystem (#4913)

* docs and skeleton

* subsystem skeleton

* main loop

* fragment tree basics & fmt

* begin fragment trees & view

* flesh out more of view update logic

* further flesh out update logic

* some refcount functions for fragment trees

* add fatal/non-fatal errors

* use non-fatal results

* clear up some TODOs

* ideal format for scheduling info

* add a bunch of TODOs

* some more fluff

* extract fragment graph to submodule

* begin fragment graph API

* trees, not graphs

* improve docs

* scope and constructor for trees

* add some test TODOs

* limit max ancestors and store constraints

* constructor

* constraints: fix bug in HRMP watermarks

* fragment tree population logic

* set::retain

* extract population logic

* implement add_and_populate

* fmt

* add some TODOs in tests

* implement child-selection

* strip out old stuff based on wrong assumptions

* use fatality

* implement pruning

* remove unused ancestor constraints

* fragment tree instantiation

* remove outdated comment

* add message/request types and skeleton for handling

* fmt

* implement handle_candidate_seconded

* candidate storage: handle backed

* implement handle_candidate_backed

* implement answer_get_backable_candidate

* remove async where not needed

* implement fetch_ancestry

* add logic for run_iteration

* add some docs

* remove global allow(unused), fix warnings

* make spellcheck happy (despite English)

* fmt

* bump Cargo.lock

* replace tracing with gum

* introduce PopulateFrom trait

* implement GetHypotheticalDepths

* revise docs slightly

* first fragment tree scope test

* more scope tests

* test add_candidate

* fmt

* test retain

* refactor test code

* test populate is recursive

* test contiguity of depth 0 is maintained

* add_and_populate tests

* cycle tests

* remove PopulateFrom trait

* fmt

* test hypothetical depths (non-recursive)

* have CandidateSeconded return membership

* tree membership requests

* Add a ProspectiveParachainsSubsystem struct

* add a staging API for base constraints

* add a `From` impl

* add runtime API for staging_validity_constraints

* implement fetch_base_constraints

* implement `fetch_upcoming_paras`

* remove reconstruction of candidate receipt; no obvious usecase

* fmt

* export message to broader module

* remove last TODO

* correctly export

* fix compilation and add GetMinimumRelayParent request

* make provisioner into a real subsystem with proper mesage bounds

* fmt

* fix ChannelsOut in overseer test

* fix overseer tests

* fix again

* fmt

* Integrate prospective parachains subsystem into backing: Part 1 (#5557)

* BEGIN ASYNC candidate-backing CHANGES

* rename & document modes

* answer prospective validation data requests

* GetMinimumRelayParents request is now plural

* implement an implicit view utility for backing subsystems

* implicit-view: get allowed relay parents

* refactorings and improvements to implicit view

* add some TODOs for tests

* split implicit view updates into 2 functions

* backing: define State to prepare for functional refactor

* add some docs

* backing: implement bones of new leaf activation logic

* backing: create per-relay-parent-states

* use new handle_active_leaves_update

* begin extracting logic from CandidateBackingJob

* mostly extract statement import from job logic

* handle statement imports outside of job logic

* do some TODO planning for prospective parachains integration

* finish rewriting backing subsystem in functional style

* add prospective parachains mode to relay parent entries

* fmt

* add a RejectedByProspectiveParachains error

* notify prospective parachains of seconded and backed candidates

* always validate candidates exhaustively in backing.

* return persisted_validation_data from validation

* handle rejections by prospective parachains

* implement seconding sanity check

* invoke validate_and_second

* Alter statement table to allow multiple seconded messages per validator

* refactor backing to have statements carry PVD

* clean up all warnings

* Add tests for implicit view

* Improve doc comments

* Prospective parachains mode based on Runtime API version

* Add a TODO

* Rework seconding_sanity_check

* Iterate over responses

* Update backing tests

* collator-protocol: load PVD from runtime

* Fix validator side tests

* Update statement-distribution to fetch PVD

* Fix statement-distribution tests

* Backing tests with prospective paras #1

* fix per_relay_parent pruning in backing

* Test multiple leaves

* Test seconding sanity check

* Import statement order

Before creating an entry in `PerCandidateState` map
wait for the approval from the prospective parachains

* Add a test for correct state updates

* Second multiple candidates per relay parent test

* Add backing tests with prospective paras

* Second more than one test without prospective paras

* Add a test for prospective para blocks

* Update malus

* typos

Co-authored-by: Chris Sosnin <chris125_@live.com>

* Track occupied depth in backing per parachain (#5778)

* provisioner: async backing changes (#5711)

* Provisioner changes for async backing

* Select candidates based on prospective paras mode

* Revert naming

* Update tests

* Update TODO comment

* review

* provisioner: async backing changes (#5711)

* Provisioner changes for async backing

* Select candidates based on prospective paras mode

* Revert naming

* Update tests

* Update TODO comment

* review

* fmt

* Network bridge changes for asynchronous backing + update subsystems to handle versioned packets (#5991)

* BEGIN STATEMENT DISTRIBUTION WORK

create a vstaging network protocol which is the same as v1

* mostly make network bridge amenable to vstaging

* network-bridge: fully adapt to vstaging

* add some TODOs for tests

* fix fallout in bitfield-distribution

* bitfield distribution tests + TODOs

* fix fallout in gossip-support

* collator-protocol: fix message fallout

* collator-protocol: load PVD from runtime

* add TODO for vstaging tests

* make things compile

* set used network protocol version using a feature

* fmt

* get approval-distribution building

* fix approval-distribution tests

* spellcheck

* nits

* approval distribution net protocol test

* bitfield distribution net protocol test

* Revert "collator-protocol: fix message fallout"

This reverts commit 07cc887303e16c6b3843ecb25cdc7cc2080e2ed1.

* Network bridge tests

Co-authored-by: Chris Sosnin <chris125_@live.com>

* remove max_pov_size requirement from prospective pvd request (#6014)

* remove max_pov_size requirement from prospective pvd request

* fmt

* Extract legacy statement distribution to its own module (#6026)

* add compatibility type to v2 statement distribution message

* warning cleanup

* handle compatibility layer for v2

* clean up an unimplemented!() block

* circulate statements based on version

* extract legacy v1 code into separate module

* remove unimplemented

* clean up naming of from_requester/responder

* remove TODOs

* have backing share seconded statements with PVD

* fmt

* fix warning

* Quick fix unused warning for not yet implemented/used staging messages.

* Fix network bridge test

* Fix wrong merge.

We now have 23 subsystems (network bridge split + prospective
parachains)

Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>

* Version 3 is already live.

* Fix tests (#6055)

* Fix backing tests

* Fix warnings.

* fmt

* collator-protocol: asynchronous backing changes (#5740)

* Draft collator side changes

* Start working on collations management

* Handle peer's view change

* Versioning on advertising

* Versioned collation fetching request

* Handle versioned messages

* Improve docs for collation requests

* Add spans

* Add request receiver to overseer

* Fix collator side tests

* Extract relay parent mode to lib

* Validator side draft

* Add more checks for advertisement

* Request pvd based on async backing mode

* review

* Validator side improvements

* Make old tests green

* More fixes

* Collator side tests draft

* Send collation test

* fmt

* Collator side network protocol versioning

* cleanup

* merge artifacts

* Validator side net protocol versioning

* Remove fragment tree membership request

* Resolve todo

* Collator side core state test

* Improve net protocol compatibility

* Validator side tests

* more improvements

* style fixes

* downgrade log

* Track implicit assignments

* Limit the number of seconded candidates per para

* Add a sanity check

* Handle fetched candidate

* fix tests

* Retry fetch

* Guard against dequeueing while already fetching

* Reintegrate connection management

* Timeout on advertisements

* fmt

* spellcheck

* update tests after merge

* validator assignment fixes for backing and collator protocol (#6158)

* Rename depth->ancestry len in tests

* Refactor group assignments

* Remove implicit assignments

* backing: consider occupied core assignments

* Track a single para on validator side

* Refactor prospective parachains mode request (#6179)

* Extract prospective parachains mode into util

* Skip activations depending on the mode

* backing: don't send backed candidate to provisioner (#6185)

* backing: introduce `CanSecond` request for advertisements filtering (#6225)

* Drop BoundToRelayParent

* draft changes

* fix backing tests

* Fix genesis ancestry

* Fix validator side tests

* more tests

* cargo generate-lockfile

* Implement `StagingValidityConstraints` Runtime API method (#6258)

* Implement StagingValidityConstraints

* spellcheck

* fix ump params

* Update hrmp comment

* Introduce ump per candidate limit

* hypothetical earliest block

* refactor primitives usage

* hypothetical earliest block number test

* fix build

* Prepare the Runtime for asynchronous backing upgrade (#6287)

* Introduce async backing params to runtime config

* fix cumulus config

* use config

* finish runtimes

* Introduce new staging API

* Update collator protocol

* Update provisioner

* Update prospective parachains

* Update backing

* Move async backing params lower in the config

* make naming consistent

* misc

* Use real prospective parachains subsystem (#6407)

* Backport `HypotheticalFrontier` into the feature branch (#6605)

* implement more general HypotheticalFrontier

* fmt

* drop unneeded request

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Resolve todo about legacy leaf activation (#6447)

* fix bug/warning in handling membership answers

* Remove `HypotheticalDepthRequest` in favor of `HypotheticalFrontierRequest` (#6521)

* Remove `HypotheticalDepthRequest` for `HypotheticalFrontierRequest`

* Update tests

* Fix (removed wrong docstring)

* Fix can_second request

* Patch some dead_code errors

---------

Co-authored-by: Chris Sosnin <chris125_@live.com>

* Async Backing: Send Statement Distribution "Backed" messages (#6634)

* Backing: Send Statement Distribution "Backed" messages

Closes #6590.

**TODO:**

- [ ] Adjust tests

* Fix compile errors

* (Mostly) fix tests

* Fix comment

* Fix test and compile error

* Test that `StatementDistributionMessage::Backed` is sent

* Fix compile error

* Fix some clippy errors

* Add prospective parachains subsystem tests (#6454)

* Add prospective parachains subsystem test

* Add `should_do_no_work_if_async_backing_disabled_for_leaf` test

* Implement `activate_leaf` helper, up to getting ancestry

* Finish implementing `activate_leaf`

* Small refactor in `activate_leaf`

* Get `CandidateSeconded` working

* Finish `send_candidate_and_check_if_found` test

* Refactor; send more leaves & candidates

* Refactor test

* Implement `check_candidate_parent_leaving_view` test

* Start work on `check_candidate_on_multiple_forks` test

* Don’t associate specific parachains with leaf

* Finish `correctly_updates_leaves` test

* Fix cycle due to reused head data

* Fix `check_backable_query` test

* Fix `check_candidate_on_multiple_forks` test

* Add `check_depth_and_pvd_queries` test

* Address review comments

* Remove TODO

* add a new index for output head data to candidate storage

* Resolve test TODOs

* Fix compile errors

* test candidate storage pruning, make sure new index is cleaned up

---------

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Node-side metrics for asynchronous backing (#6549)

* Add metrics for `prune_view_candidate_storage`

* Add metrics for `request_unblocked_collations`

* Fix docstring

* Couple fixes from review comments

* Fix `check_depth_query` test

* inclusion-emulator: mirror advancement rule check (#6361)

* inclusion-emulator: mirror advancement rule check

* fix build

* prospective-parachains: introduce `backed_in_path_only` flag for advertisements (#6649)

* Introduce `backed_in_path_only` flag for depth request

* fmt

* update doc comment

* fmt

* Add async-backing zombienet tests (#6314)

* Async backing: impl guide for statement distribution (#6738)

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com>

* Asynchronous backing statement distribution: Take III (#5999)

* add notification types for v2 statement-distribution

* improve protocol docs

* add empty vstaging module

* fmt

* add backed candidate packet request types

* start putting down structure of new logic

* handle activated leaf

* some sanity-checking on outbound statements

* fmt

* update vstaging share to use statements with PVD

* tiny refactor, candidate_hash location

* import local statements

* refactor statement import

* first stab at broadcast logic

* fmt

* fill out some TODOs

* start on handling incoming

* split off session info into separate map

* start in on a knowledge tracker

* address some grumbles

* format

* missed comment

* some docs for direct

* add note on slashing

* amend

* simplify 'direct' code

* finish up the 'direct' logic

* add a bunch of tests for the direct-in-group logic

* rename 'direct' to 'cluster', begin a candidate_entry module

* distill candidate_entry

* start in on a statement-store module

* some utilities for the statement store

* rewrite 'send_statement_direct' using new tools

* filter sending logic on peers which have the relay-parent in their view.

* some more logic for handling incoming statements

* req/res: BackedCandidatePacket -> AttestedCandidate + tweaks

* add a `validated_in_group` bitfield to BackedCandidateInventory

* BackedCandidateInventory -> Manifest

* start in on requester module

* add outgoing request for attested candidate

* add a priority mechanism for requester

* some request dispatch logic

* add seconded mask to tagged-request

* amend manifest to hold group index

* handle errors and set up scaffold for response validation

* validate attested candidate responses

* requester -> requests

* add some utilities for manipulating requests

* begin integrating requester

* start grid module

* tiny

* refactor grid topology to expose more info to subsystems

* fix grid_topology test

* fix overseer test

* implement topology group-based view construction logic

* fmt

* flesh out grid slightly more

* add indexed groups utility

* integrate Groups into per-session info

* refactor statement store to borrow Groups

* implement manifest knowledge utility

* add a test for topology setup

* don't send to group members

* test for conflicting manifests

* manifest knowledge tests

* fmt

* rename field

* garbage collection for grid tracker

* routines for finding correct/incorrect advertisers

* add manifest import logic

* tweak naming

* more tests for manifest import

* add comment

* rework candidates into a view-wide tracker

* fmt

* start writing boilerplate for grid sending

* fmt

* some more group boilerplate

* refactor handling of topology and authority IDs

* fmt

* send statements directly to grid peers where possible

* send to cluster only if statement belongs to cluster

* improve handling of cluster statements

* handle incoming statements along the grid

* API for introduction of candidates into the tree

* backing: use new prospective parachains API

* fmt prospective parachains changes

* fmt statement-dist

* fix condition

* get ready for tracking importable candidates

* prospective parachains: add Cow logic

* incomplete and complete hypothetical candidates

* remove keep_if_unneeded

* fmt

* implement more general HypotheticalFrontier

* fmt, cleanup

* add a by_parent_hash index to candidate tracker

* more framework for future code

* utilities for getting all hypothetical candidates for frontier

* track origin in statement store

* fmt

* requests should return peer

* apply post-confirmation reckoning

* flesh out import/announce/circulate logic on new statements

* adjust

* adjust TODO comment

* fix  backing tests

* update statement-distribution to use new indexedvec

* fmt

* query hypothetical candidates

* implement `note_importable_under`

* extract common utility of fragment tree updates

* add a helper function for getting statements unknown by backing

* import fresh statements to backing

* send announcements and acknowledgements over grid

* provide freshly importable statements

also avoid tracking backed candidates in statement distribution

* do not issue requests on newly importable candidates

* add TODO for later when confirming candidate

* write a routine for handling backed candidate notifications

* simplify grid substantially

* add some test TODOs

* handle confirmed candidates & grid announcements

* finish implementing manifest handling, including follow up statements

* send follow-up statements when acknowledging freshly backed

* fmt

* handle incoming acknowledgements

* a little DRYing

* wire up network messages to handlers

* fmt

* some skeleton code for peer view update handling

* more peer view skeleton stuff

* Fix async backing statement distribution tests (#6621)

* Fix compile errors in tests

* Cargo fmt

* Resolve some todos in async backing statement-distribution branch (#6482)

* Implement `remove_by_relay_parent`

* Extract `minimum_votes` to shared primitives.

* Add `can_send_statements_received_with_prejudice` test

* Fix test

* Update docstrings

* Cargo fmt

* Fix compile error

* Fix compile errors in tests

* Cargo fmt

* Add module docs; write `test_priority_ordering` (first draft)

* Fix `test_priority_ordering`

* Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority`

* Address review comments

* Remove `Entry::get_mut`

* fix test compilation

* add a TODO for a test

* clean up a couple of TODOs

* implement sending pending cluster statements

* refactor utility function for sending acknowledgement and statements

* mostly implement catching peers up via grid

* Fix clippy error

* alter grid to track all pending statements

* fix more TODOs and format

* tweak a TODO in requests

* some logic for dispatching requests

* fmt

* skeleton for response receiving

* Async backing statement distribution: cluster tests (#6678)

* Add `pending_statements_set_when_receiving_fresh_statements`

* Add `pending_statements_updated_when_sending_statements` test

* fix up

* fmt

* update TODO

* rework seconded mask in requests

* change doc

* change unhandledresponse not to borrow request manager

* only accept responses sufficient to back

* finish implementing response handling

* extract statement filter to protocol crate

* rework requests: use statement filter in network protocol

* dispatch cluster requests correctly

* rework cluster statement sending

* implement request answering

* fmt

* only send confirmed candidate statement messages on unified relay-parent

* Fix Tests In Statement Distribution Branch

* Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* clean up some review comments

* clean up warnings

* Async backing statement distribution: grid tests (#6673)

* Add `manifest_import_returns_ok_true` test

* cargo fmt

* Add pending_communication_receiving_manifest_on_confirmed_candidate

* Add `senders_can_provide_manifests_in_acknowledgement` test

* Add a couple of tests for pending statements

* Add `pending_statements_cleared_when_sending` test

* Add `pending_statements_respect_remote_knowledge` test

* Refactor group creation in tests

* Clarify docs

* Address some review comments

* Make some clarifications

* Fix post-merge errors

* Clarify test `senders_can_provide_manifests_in_acknowledgement`

* Try writing `pending_statements_are_updated_after_manifest_exchange`

* Document "seconding limit" and `reject_overflowing_manifests` test

* Test that seconding counts are not updated for validators on error

* Fix tests

* Fix manifest exchange test

* Add more tests in `requests.rs` (#6707)

This resolves remaining TODOs in this file.

* remove outdated inventory terminology

* Async backing statement distribution: `Candidates` tests (#6658)

* Async Backing: Fix clippy errors in statement distribution branch (#6720)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* Async Backing: Fix clippy errors in statement distribution branch

* Fix some more clippy lints

* add tests module

* fix warnings in existing tests

* create basic test harness

* create a test state struct

* fmt

* create empty cluster & grid modules for tests

* some TODOs for cluster test suite

* describe test-suite for grid logic

* describe request test suite

* fix seconding-limit bug

* Remove extraneous `pub`

This somehow made it into my clippy PR.

* Fix some test compile warnings

* Remove some unneeded `allow`s

* adapt some new test helpers from Marcin

* add helper for activating a gossip topology

* add utility for signing statements

* helpers for connecting/disconnecting peers

* round out network utilities

* fmt

* fix bug in initializing validator-meta

* fix compilation

* implement first cluster test

* TODOs for incoming request tests

* Remove unneeded `make_committed_candidate` helper

* fmt

* some more tests for cluster

* add a TODO about grid senders

* integrate inbound req/res into test harness

* polish off initial cluster test suite

* keep introduce candidate request

* fix tests after introduce candidate request

* fmt

* Add grid protocol to module docs

* Fix comments

* Test `backed_in_path_only: true`

* Update node/network/protocol/src/lib.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Update node/network/protocol/src/request_response/mod.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Mark receiver with `vstaging`

* validate grid senders based on manifest kind

* fix mask_seconded/valid

* fix unwanted-mask check

* fix build

* resolve todo on leaf mode

* Unify protocol naming to vstaging

* fmt, fix grid test after topology change

* typo

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* address review

* adjust comment, make easier to understand

* Fix typo

---------

Co-authored-by: Marcin S <marcin@bytedude.com>
Co-authored-by: Marcin S <marcin@realemail.net>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>

* miscellaneous fixes to make asynchronous backing work (#6791)

* propagate network-protocol-staging feature

* add feature to adder-collator as well

* allow collation-generation of occupied cores

* prospective parachains: special treatment for pending availability candidates

* runtime: fetch candidates pending availability

* lazily construct PVD for pending candidates

* fix fallout in prospective parachains hypothetical/select_child

* runtime: enact candidates when creating paras-inherent

* make tests compile

* test pending availability in the scope

* add prospective parachains test

* fix validity constraints leftovers

* drop prints

* Fix typos

---------

Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Marcin S <marcin@realemail.net>

* Remove restart from test (#6840)

* Async Backing: Statement Distribution Tests (#6755)

* start on handling incoming

* split off session info into separate map

* start in on a knowledge tracker

* address some grumbles

* format

* missed comment

* some docs for direct

* add note on slashing

* amend

* simplify 'direct' code

* finish up the 'direct' logic

* add a bunch of tests for the direct-in-group logic

* rename 'direct' to 'cluster', begin a candidate_entry module

* distill candidate_entry

* start in on a statement-store module

* some utilities for the statement store

* rewrite 'send_statement_direct' using new tools

* filter sending logic on peers which have the relay-parent in their view.

* some more logic for handling incoming statements

* req/res: BackedCandidatePacket -> AttestedCandidate + tweaks

* add a `validated_in_group` bitfield to BackedCandidateInventory

* BackedCandidateInventory -> Manifest

* start in on requester module

* add outgoing request for attested candidate

* add a priority mechanism for requester

* some request dispatch logic

* add seconded mask to tagged-request

* amend manifest to hold group index

* handle errors and set up scaffold for response validation

* validate attested candidate responses

* requester -> requests

* add some utilities for manipulating requests

* begin integrating requester

* start grid module

* tiny

* refactor grid topology to expose more info to subsystems

* fix grid_topology test

* fix overseer test

* implement topology group-based view construction logic

* fmt

* flesh out grid slightly more

* add indexed groups utility

* integrate Groups into per-session info

* refactor statement store to borrow Groups

* implement manifest knowledge utility

* add a test for topology setup

* don't send to group members

* test for conflicting manifests

* manifest knowledge tests

* fmt

* rename field

* garbage collection for grid tracker

* routines for finding correct/incorrect advertisers

* add manifest import logic

* tweak naming

* more tests for manifest import

* add comment

* rework candidates into a view-wide tracker

* fmt

* start writing boilerplate for grid sending

* fmt

* some more group boilerplate

* refactor handling of topology and authority IDs

* fmt

* send statements directly to grid peers where possible

* send to cluster only if statement belongs to cluster

* improve handling of cluster statements

* handle incoming statements along the grid

* API for introduction of candidates into the tree

* backing: use new prospective parachains API

* fmt prospective parachains changes

* fmt statement-dist

* fix condition

* get ready for tracking importable candidates

* prospective parachains: add Cow logic

* incomplete and complete hypothetical candidates

* remove keep_if_unneeded

* fmt

* implement more general HypotheticalFrontier

* fmt, cleanup

* add a by_parent_hash index to candidate tracker

* more framework for future code

* utilities for getting all hypothetical candidates for frontier

* track origin in statement store

* fmt

* requests should return peer

* apply post-confirmation reckoning

* flesh out import/announce/circulate logic on new statements

* adjust

* adjust TODO comment

* fix  backing tests

* update statement-distribution to use new indexedvec

* fmt

* query hypothetical candidates

* implement `note_importable_under`

* extract common utility of fragment tree updates

* add a helper function for getting statements unknown by backing

* import fresh statements to backing

* send announcements and acknowledgements over grid

* provide freshly importable statements

also avoid tracking backed candidates in statement distribution

* do not issue requests on newly importable candidates

* add TODO for later when confirming candidate

* write a routine for handling backed candidate notifications

* simplify grid substantially

* add some test TODOs

* handle confirmed candidates & grid announcements

* finish implementing manifest handling, including follow up statements

* send follow-up statements when acknowledging freshly backed

* fmt

* handle incoming acknowledgements

* a little DRYing

* wire up network messages to handlers

* fmt

* some skeleton code for peer view update handling

* more peer view skeleton stuff

* Fix async backing statement distribution tests (#6621)

* Fix compile errors in tests

* Cargo fmt

* Resolve some todos in async backing statement-distribution branch (#6482)

* Implement `remove_by_relay_parent`

* Extract `minimum_votes` to shared primitives.

* Add `can_send_statements_received_with_prejudice` test

* Fix test

* Update docstrings

* Cargo fmt

* Fix compile error

* Fix compile errors in tests

* Cargo fmt

* Add module docs; write `test_priority_ordering` (first draft)

* Fix `test_priority_ordering`

* Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority`

* Address review comments

* Remove `Entry::get_mut`

* fix test compilation

* add a TODO for a test

* clean up a couple of TODOs

* implement sending pending cluster statements

* refactor utility function for sending acknowledgement and statements

* mostly implement catching peers up via grid

* Fix clippy error

* alter grid to track all pending statements

* fix more TODOs and format

* tweak a TODO in requests

* some logic for dispatching requests

* fmt

* skeleton for response receiving

* Async backing statement distribution: cluster tests (#6678)

* Add `pending_statements_set_when_receiving_fresh_statements`

* Add `pending_statements_updated_when_sending_statements` test

* fix up

* fmt

* update TODO

* rework seconded mask in requests

* change doc

* change unhandledresponse not to borrow request manager

* only accept responses sufficient to back

* finish implementing response handling

* extract statement filter to protocol crate

* rework requests: use statement filter in network protocol

* dispatch cluster requests correctly

* rework cluster statement sending

* implement request answering

* fmt

* only send confirmed candidate statement messages on unified relay-parent

* Fix Tests In Statement Distribution Branch

* Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* clean up some review comments

* clean up warnings

* Async backing statement distribution: grid tests (#6673)

* Add `manifest_import_returns_ok_true` test

* cargo fmt

* Add pending_communication_receiving_manifest_on_confirmed_candidate

* Add `senders_can_provide_manifests_in_acknowledgement` test

* Add a couple of tests for pending statements

* Add `pending_statements_cleared_when_sending` test

* Add `pending_statements_respect_remote_knowledge` test

* Refactor group creation in tests

* Clarify docs

* Address some review comments

* Make some clarifications

* Fix post-merge errors

* Clarify test `senders_can_provide_manifests_in_acknowledgement`

* Try writing `pending_statements_are_updated_after_manifest_exchange`

* Document "seconding limit" and `reject_overflowing_manifests` test

* Test that seconding counts are not updated for validators on error

* Fix tests

* Fix manifest exchange test

* Add more tests in `requests.rs` (#6707)

This resolves remaining TODOs in this file.

* remove outdated inventory terminology

* Async backing statement distribution: `Candidates` tests (#6658)

* Async Backing: Fix clippy errors in statement distribution branch (#6720)

* Integrate `handle_active_leaves_update`

* Integrate `share_local_statement`/`handle_backed_candidate_message`

* Start hooking up request/response flow

* Finish hooking up request/response flow

* Limit number of parallel requests in responder

* Fix test compilation errors

* Fix missing check for prospective parachains mode

* Fix some more compile errors

* Async Backing: Fix clippy errors in statement distribution branch

* Fix some more clippy lints

* add tests module

* fix warnings in existing tests

* create basic test harness

* create a test state struct

* fmt

* create empty cluster & grid modules for tests

* some TODOs for cluster test suite

* describe test-suite for grid logic

* describe request test suite

* fix seconding-limit bug

* Remove extraneous `pub`

This somehow made it into my clippy PR.

* Fix some test compile warnings

* Remove some unneeded `allow`s

* adapt some new test helpers from Marcin

* add helper for activating a gossip topology

* add utility for signing statements

* helpers for connecting/disconnecting peers

* round out network utilities

* fmt

* fix bug in initializing validator-meta

* fix compilation

* implement first cluster test

* TODOs for incoming request tests

* Remove unneeded `make_committed_candidate` helper

* fmt

* Hook up request sender

* Add `valid_statement_without_prior_seconded_is_ignored` test

* Fix `valid_statement_without_prior_seconded_is_ignored` test

* some more tests for cluster

* add a TODO about grid senders

* integrate inbound req/res into test harness

* polish off initial cluster test suite

* keep introduce candidate request

* fix tests after introduce candidate request

* fmt

* Add grid protocol to module docs

* Remove obsolete test

* Fix comments

* Test `backed_in_path_only: true`

* Update node/network/protocol/src/lib.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Update node/network/protocol/src/request_response/mod.rs

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* Mark receiver with `vstaging`

* First draft of `ensure_seconding_limit_is_respected` test

* validate grid senders based on manifest kind

* fix mask_seconded/valid

* fix unwanted-mask check

* fix build

* resolve todo on leaf mode

* Unify protocol naming to vstaging

* Fix `ensure_seconding_limit_is_respected` test

* Start `backed_candidate_leads_to_advertisement` test

* fmt, fix grid test after topology change

* Send Backed notification

* Finish `backed_candidate_leads_to_advertisement` test

* Finish `peer_reported_for_duplicate_statements` test

* Finish `received_advertisement_before_confirmation_leads_to_request`

* Add `advertisements_rejected_from_incorrect_peers` test

* Add `manifest_rejected_*` tests

* Add `manifest_rejected_when_group_does_not_match_para` test

* Add `local_node_sanity_checks_incoming_requests` test

* Add `local_node_respects_statement_mask` test

* Add tests where peer is reported for providing invalid signatures

* Add `cluster_peer_allowed_to_send_incomplete_statements` test

* Add `received_advertisement_after_backing_leads_to_acknowledgement`

* Add `received_advertisement_after_confirmation_before_backing` test

* peer_reported_for_advertisement_conflicting_with_confirmed_candidate

* Add `peer_reported_for_not_enough_statements` test

* Add `peer_reported_for_providing_statements_meant_to_be_masked_out`

* Add `additional_statements_are_shared_after_manifest_exchange`

* Add `grid_statements_imported_to_backing` test

* Add `relay_parent_entering_peer_view_leads_to_advertisement` test

* Add `advertisement_not_re_sent_when_peer_re_enters_view` test

* Update node/network/statement-distribution/src/vstaging/tests/grid.rs

Co-authored-by: asynchronous rob <rphmeier@gmail.com>

* Resolve TODOs, update test

* Address unused code

* Add check after every test for unhandled requests

* Refactor (`make_dummy_leaf` and `handle_sent_request`)

* Refactor (`make_dummy_topology`)

* Minor refactor

---------

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>

* Fix some clippy lints in tests

* Async backing: minor fixes (#6920)

* bitfield-distribution test

* implicit view tests

* Refactor parameters -> params

* scheduler: update storage migration (#6963)

* update scheduler migration

* Adjust weight to account for storage read

* Statement Distribution Guide Edits (#7025)

* Statement distribution guide edits

* Addressed Marcin's comments

* Add attested candidate request retry timeouts (#6833)

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
Fix async backing statement distribution tests (#6621)
Resolve some todos in async backing statement-distribution branch (#6482)
Fix clippy errors in statement distribution branch (#6720)

* Async backing: add Prospective Parachains impl guide (#6933)

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Updates to Provisioner Guide for Async Backing (#7106)

* Initial corrections and clarifications

* Partial first draft

* Finished first draft

* Adding back wrongly removed test bit

* fmt

* Update roadmap/implementers-guide/src/node/utility/provisioner.md

Co-authored-by: Marcin S. <marcin@realemail.net>

* Addressing comments

* Reorganization

* fmt

---------

Co-authored-by: Marcin S. <marcin@realemail.net>

* fmt

* Renaming Parathread Mentions (#7287)

* Renaming parathreads

* Renaming module to pallet

* More updates

* PVF: Refactor workers into separate crates, remove host dependency (#7253)

* PVF: Refactor workers into separate crates, remove host dependency

* Fix compile error

* Remove some leftover code

* Fix compile errors

* Update Cargo.lock

* Remove worker main.rs files

I accidentally copied these from the other PR. This PR isn't intended to
introduce standalone workers yet.

* Address review comments

* cargo fmt

* Update a couple of comments

* Update log targets

* Update quote to 1.0.27 (#7280)

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>

* pallets: implement `Default` for `GenesisConfig` in `no_std` (#7271)

* pallets: implement Default for GenesisConfig in no_std

This change is follow-up of: https://github.com/paritytech/substrate/pull/14108

It is a step towards: https://github.com/paritytech/substrate/issues/13334

* Cargo.lock updated

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* cli: enable BEEFY by default on test networks (#7293)

We consider BEEFY mature enough to run by default on all nodes
for test networks (Rococo/Wococo/Versi).

Right now, most nodes are not running it since it's opt-in using
--beefy flag. Switch to an opt-out model for test networks.

Replace --beefy flag from CLI with --no-beefy and have BEEFY
client start by default on test networks.

Signed-off-by: acatangiu <adrian@parity.io>

* runtime: past session slashing runtime API (#6667)

* runtime/vstaging: unapplied_slashes runtime API

* runtime/vstaging: key_ownership_proof runtime API

* runtime/ParachainHost: submit_report_dispute_lost

* fix key_ownership_proof API

* runtime: submit_report_dispute_lost runtime API

* nits

* Update node/subsystem-types/src/messages.rs

Co-authored-by: Marcin S. <marcin@bytedude.com>

* revert unrelated fmt changes

* post merge fixes

* fix compilation

---------

Co-authored-by: Marcin S. <marcin@bytedude.com>

* Correcting git mishap

* Document usage of `gum` crate (#7294)

* Document usage of gum crate

* Small fix

* Add some more basic info

* Update node/gum/src/lib.rs

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* Update target docs

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* XCM: Fix issue with RequestUnlock (#7278)

* XCM: Fix issue with RequestUnlock

* Leave API changes for v4

* Fix clippy errors

* Fix tests

---------

Co-authored-by: parity-processbot <>

* Companion for Substrate#14228 (#7295)

* Companion for Substrate#14228

https://github.com/paritytech/substrate/pull/14228

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* Companion for #14237: Use latest sp-crates (#7300)

* To revert: Update substrate branch to "lexnv/bump_sp_crates"

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "To revert: Update substrate branch to "lexnv/bump_sp_crates""

This reverts commit 5f1db84eac4a226c37b7f6ce6ee19b49dc7e2008.

* Update cargo lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* bounded-collections bump to 0.1.7 (#7305)

* bounded-collections bump to 0.1.7

Companion for: paritytech/substrate#14225

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* bump to quote 1.0.28 (#7306)

* `RollingSessionWindow` cleanup (#7204)

* Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit

* Fix tests in import

* Fix the rest of the tests

* Remove dead code

* Fix todos

* Simplify session caching

* Comments for `SessionInfoProvider`

* Separate `SessionInfoProvider` from `State`

* `cache_session_info_for_head` becomes freestanding function

* Remove unneeded `mut` usage

* fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider`

* Fix SessionInfo retrieval

* Code cleanup

* Don't wrap `SessionInfoProvider` in an `Option`

* Remove `earliest_session()`

* Remove pre-caching -> wip

* Fix some tests and code cleanup

* Fix all tests

* Fixes in tests

* Fix comments, variable names and small style changes

* Fix a warning

* impl From<SessionWindowSize> for NonZeroUsize

* Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG

* Code review feedback

* Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db

* Remove `col_session_data` usages

* Storage migration clearing columns w/o removing them

* Remove session data column usages from `approval-voting` and `dispute-coordinator` tests

* Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests

* Fix formatting in initialized.rs

* Fix a corner case in `SessionInfo` caching for `dispute-coordinator`

* Remove `RollingSessionWindow` ;(

* Revert "Fix formatting in initialized.rs"

This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc.

* v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it

* Fix `NUM_COLUMNS` in `approval-voting`

* Use `columns::v3::NUM_COLUMNS` when opening db

* Update node/service/src/parachains_db/upgrade.rs

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3`

* Fix `NUM+COLUMNS` in approval_voting

* Fix formatting

* Fix columns usage

* Clarification comments about the different db versions

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* pallet-para-config: Remove remnant WeightInfo functions (#7308)

* pallet-para-config: Remove remnant WeightInfo functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* set_config_with_weight begone

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::configuration

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

* XCM: PayOverXcm config (#6900)

* Move XCM query functionality to trait

* Fix tests

* Add PayOverXcm implementation

* fix the PayOverXcm trait to compile

* moved doc comment out of trait implmeentation and to the trait

* PayOverXCM documentation

* Change documentation a bit

* Added empty benchmark methods implementation and changed docs

* update PayOverXCM to convert AccountIds to MultiLocations

* Implement benchmarking method

* Change v3 to latest

* Descend origin to an asset sender (#6970)

* descend origin to an asset sender

* sender as tuple of dest and sender

* Add more variants to the QueryResponseStatus enum

* Change Beneficiary to Into<[u8; 32]>

* update PayOverXcm to return concrete errors and use AccountId as sender

* use polkadot-primitives for AccountId

* fix dependency to use polkadot-core-primitives

* force Unpaid instruction to the top of the instructions list

* modify report_outcome to accept interior argument

* use new_query directly for building final xcm query, instead of report_outcome

* fix usage of new_query to use the XcmQueryHandler

* fix usage of new_query to use the XcmQueryHandler

* tiny method calling fix

* xcm query handler (#7198)

* drop redundant query status

* rename ReportQueryStatus to OuterQueryStatus

* revert rename of QueryResponseStatus

* update mapping

* Update xcm/xcm-builder/src/pay.rs

Co-authored-by: Gavin Wood <gavin@parity.io>

* Updates

* Docs

* Fix benchmarking stuff

* Destination can be determined based on asset_kind

* Tweaking API to minimise clones

* Some repotting and docs

---------

Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>

* Companion for #14265 (#7307)

* Update Cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update Cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>

* bump serde to 1.0.163 (#7315)

* bump serde to 1.0.163

* bump ci

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* fmt

* Updated fmt

* Removing changes accidentally pulled from master

* fix another master pull issue

* Another master pull fix

* fmt

* Fixing implementers guide build

* Revert "Merge branch 'rh-async-backing-feature-while-frozen' of https://github.com/paritytech/polkadot into brad-rename-parathread"

This reverts commit bebc24af52ab61155e3fe02cb3ce66a592bce49c, reversing
changes made to 1b2de662dfb11173679d6da5bd0da9d149c85547.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: acatangiu <adrian@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Marcin S <marcin@realemail.net>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>

* fix bitfield distribution test

* approval distribution tests

* fix bridge tests

* update Cargo.lock

* [async-backing-branch] Optimize collator-protocol validator-side request fetching (#7457)

* Optimize collator-protocol validator-side request fetching

* address feedback: replace tuples with structs

* feedback: add doc comments

* move collation types to subfolder

---------

Signed-off-by: alindima <alin@parity.io>

* Update collation generation for asynchronous backing (#7405)

* break candidate receipt construction and distribution into own function

* update implementers' guide to include SubmitCollation

* implement SubmitCollation for collation-generation

* fmt

* fix test compilation & remove unnecessary submodule

* add some TODOs for a test suite.

* Update roadmap/implementers-guide/src/types/overseer-protocol.md

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* add new test harness and first test

* refactor to avoid requiring background sender

* ensure collation gets packaged and distributed

* tests for the fallback case with no hint

* add parent rp-number hint tests

* fmt

* update uses of CollationGenerationConfig

* fix remaining test

* address review comments

* use subsystemsender for background tasks

* fmt

* remove ValidationCodeHashHint and related tests

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* fix some more fallout from merge

* fmt

* remove staging APIs from Rococo & Westend (#7513)

* send network messages on main protocol name (#7515)

* misc async backing improvements for allowed ancestry blocks (#7532)

* shared: fix acquire_info

* backwards-compat test for prospective parachains

* same relay parent is allowed

* provisioner: request candidate receipt by relay parent (#7527)

* return candidates hash from prospective parachains

* update provisioner

* update tests

* guide changes

* send a single message to backing

* fix test

* revert to old `handle_new_activations` logic in some cases (#7514)

* revert to old `handle_new_activations` logic

* gate sending messages on scheduled cores to max_depth >= 2

* fmt

* 2->1

* Omnibus asynchronous backing bugfix PR (#7529)

* fix a bug in backing

* add some more logs

* prospective parachains: take ancestry only up to session bounds

* add test

* fix zombienet tests (#7614)

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix runtime compilation

* make bitfield distribution tests compile

* attempt to fix zombienet disputes (#7618)

* update metric name

* update some metric names

* avoid cycles when creating fake candidates

* make undying collator more friendly to malformed parents

* fix a bug in malus

* fmt

* clippy

* add RUN_IN_CONTAINER to new ZombieNet tests (#7631)

* remove duplicated migration

happened because of master-merge

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: acatangiu <adrian@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: alindima <alin@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Marcin S <marcin@bytedude.com>
Co-authored-by: Marcin S <marcin@realemail.net>
Co-authored-by: Mattia L.V. Bradascio <28816406+bredamatt@users.noreply.github.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com>
Co-authored-by: BradleyOlson64 <lotrftw9@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Alin Dima <alin@parity.io>

polkadot/runtime/parachains/src/configuration/tests.rs

@@ -272,7 +272,7 @@ fn consistency_bypass_works() {
 fn setting_pending_config_members() {
 	new_test_ext(Default::default()).execute_with(|| {
 		let new_config = HostConfiguration {
-			async_backing_params: primitives::vstaging::AsyncBackingParams {
+			async_backing_params: AsyncBackingParams {
 				allowed_ancestry_len: 0,
 				max_candidate_depth: 0,
 			},

polkadot/runtime/parachains/src/dmp.rs

@@ -255,13 +255,27 @@ impl<T: Config> Pallet<T> {
 	/// Checks if the number of processed downward messages is valid.
 	pub(crate) fn check_processed_downward_messages(
 		para: ParaId,
+		relay_parent_number: BlockNumberFor<T>,
 		processed_downward_messages: u32,
 	) -> Result<(), ProcessedDownwardMessagesAcceptanceErr> {
 		let dmq_length = Self::dmq_length(para);
 
 		if dmq_length > 0 && processed_downward_messages == 0 {
-			return Err(ProcessedDownwardMessagesAcceptanceErr::AdvancementRule)
+			// The advancement rule is for at least one downwards message to be processed
+			// if the queue is non-empty at the relay-parent. Downwards messages are annotated
+			// with the block number, so we compare the earliest (first) against the relay parent.
+			let contents = Self::dmq_contents(para);
+
+			// sanity: if dmq_length is >0 this should always be 'Some'.
+			if contents.get(0).map_or(false, |msg| msg.sent_at <= relay_parent_number) {
+				return Err(ProcessedDownwardMessagesAcceptanceErr::AdvancementRule)
+			}
 		}
+
+		// Note that we might be allowing a parachain to signal that it's processed
+		// messages that hadn't been placed in the queue at the relay_parent.
+		// only 'stupid' parachains would do it and we don't (and can't) force anyone
+		// to act on messages, so the lenient approach is fine here.
 		if dmq_length < processed_downward_messages {
 			return Err(ProcessedDownwardMessagesAcceptanceErr::Underflow {
 				processed_downward_messages,

polkadot/runtime/parachains/src/dmp/tests.rs

@@ -125,21 +125,43 @@ fn check_processed_downward_messages() {
 	let a = ParaId::from(1312);
 
 	new_test_ext(default_genesis_config()).execute_with(|| {
+		let block_number = System::block_number();
+
 		// processed_downward_messages=0 is allowed when the DMQ is empty.
-		assert!(Dmp::check_processed_downward_messages(a, 0).is_ok());
+		assert!(Dmp::check_processed_downward_messages(a, block_number, 0).is_ok());
 
 		queue_downward_message(a, vec![1, 2, 3]).unwrap();
 		queue_downward_message(a, vec![4, 5, 6]).unwrap();
 		queue_downward_message(a, vec![7, 8, 9]).unwrap();
 
 		// 0 doesn't pass if the DMQ has msgs.
-		assert!(!Dmp::check_processed_downward_messages(a, 0).is_ok());
+		assert!(Dmp::check_processed_downward_messages(a, block_number, 0).is_err());
 		// a candidate can consume up to 3 messages
-		assert!(Dmp::check_processed_downward_messages(a, 1).is_ok());
-		assert!(Dmp::check_processed_downward_messages(a, 2).is_ok());
-		assert!(Dmp::check_processed_downward_messages(a, 3).is_ok());
+		assert!(Dmp::check_processed_downward_messages(a, block_number, 1).is_ok());
+		assert!(Dmp::check_processed_downward_messages(a, block_number, 2).is_ok());
+		assert!(Dmp::check_processed_downward_messages(a, block_number, 3).is_ok());
 		// there is no 4 messages in the queue
-		assert!(!Dmp::check_processed_downward_messages(a, 4).is_ok());
+		assert!(Dmp::check_processed_downward_messages(a, block_number, 4).is_err());
+	});
+}
+
+#[test]
+fn check_processed_downward_messages_advancement_rule() {
+	let a = ParaId::from(1312);
+
+	new_test_ext(default_genesis_config()).execute_with(|| {
+		let block_number = System::block_number();
+
+		run_to_block(block_number + 1, None);
+		let advanced_block_number = System::block_number();
+
+		queue_downward_message(a, vec![1, 2, 3]).unwrap();
+		queue_downward_message(a, vec![4, 5, 6]).unwrap();
+
+		// The queue was empty at genesis, 0 is OK despite it being non-empty in the further block.
+		assert!(Dmp::check_processed_downward_messages(a, block_number, 0).is_ok());
+		// For the advanced block number, however, the rule is broken in case of 0.
+		assert!(Dmp::check_processed_downward_messages(a, advanced_block_number, 0).is_err());
 	});
 }
 

polkadot/runtime/parachains/src/hrmp.rs

@@ -334,7 +334,7 @@ pub mod pallet {
 		StorageMap<_, Twox64Concat, HrmpChannelId, HrmpOpenChannelRequest>;
 
 	// NOTE: could become bounded, but we don't have a global maximum for this.
-	// `HRMP_MAX_INBOUND_CHANNELS_BOUND` are per parachain/parathread, while this storage tracks the
+	// `HRMP_MAX_INBOUND_CHANNELS_BOUND` are per parachain, while this storage tracks the
 	// global state.
 	#[pallet::storage]
 	pub type HrmpOpenChannelRequestsList<T: Config> =
@@ -951,6 +951,14 @@ impl<T: Config> Pallet<T> {
 		Ok(())
 	}
 
+	/// Returns HRMP watermarks of previously sent messages to a given para.
+	pub(crate) fn valid_watermarks(recipient: ParaId) -> Vec<BlockNumberFor<T>> {
+		HrmpChannelDigests::<T>::get(&recipient)
+			.into_iter()
+			.map(|(block_no, _)| block_no)
+			.collect()
+	}
+
 	pub(crate) fn check_outbound_hrmp(
 		config: &HostConfiguration<BlockNumberFor<T>>,
 		sender: ParaId,
@@ -1015,6 +1023,27 @@ impl<T: Config> Pallet<T> {
 		Ok(())
 	}
 
+	/// Returns remaining outbound channels capacity in messages and in bytes per recipient para.
+	pub(crate) fn outbound_remaining_capacity(sender: ParaId) -> Vec<(ParaId, (u32, u32))> {
+		let recipients = HrmpEgressChannelsIndex::<T>::get(&sender);
+		let mut remaining = Vec::with_capacity(recipients.len());
+
+		for recipient in recipients {
+			let Some(channel) = HrmpChannels::<T>::get(&HrmpChannelId { sender, recipient }) else {
+				continue
+			};
+			remaining.push((
+				recipient,
+				(
+					channel.max_capacity - channel.msg_count,
+					channel.max_total_size - channel.total_size,
+				),
+			));
+		}
+
+		remaining
+	}
+
 	pub(crate) fn prune_hrmp(recipient: ParaId, new_hrmp_watermark: BlockNumberFor<T>) -> Weight {
 		let mut weight = Weight::zero();
 
@@ -1113,12 +1142,12 @@ impl<T: Config> Pallet<T> {
 			HrmpChannels::<T>::insert(&channel_id, channel);
 			HrmpChannelContents::<T>::append(&channel_id, inbound);
 
-			// The digests are sorted in ascending by block number order. Assuming absence of
-			// contextual execution, there are only two possible scenarios here:
+			// The digests are sorted in ascending by block number order. There are only two
+			// possible scenarios here ("the current" is the block of candidate's inclusion):
 			//
 			// (a) It's the first time anybody sends a message to this recipient within this block.
 			//     In this case, the digest vector would be empty or the block number of the latest
-			//     entry  is smaller than the current.
+			//     entry is smaller than the current.
 			//
 			// (b) Somebody has already sent a message within the current block. That means that
 			//     the block number of the latest entry is equal to the current.

polkadot/runtime/parachains/src/inclusion/mod.rs

@@ -14,8 +14,7 @@
 // You should have received a copy of the GNU General Public License
 // along with Polkadot.  If not, see <http://www.gnu.org/licenses/>.
 
-//! The inclusion pallet is responsible for inclusion and availability of scheduled parachains
-//! and parathreads.
+//! The inclusion pallet is responsible for inclusion and availability of scheduled parachains.
 //!
 //! It is responsible for carrying candidates from being backable to being backed, and then from
 //! backed to included.
@@ -23,8 +22,8 @@
 use crate::{
 	configuration::{self, HostConfiguration},
 	disputes, dmp, hrmp, paras,
-	scheduler::common::CoreAssignment,
-	shared,
+	scheduler::{self, common::CoreAssignment},
+	shared::{self, AllowedRelayParentsTracker},
 };
 use bitvec::{order::Lsb0 as BitOrderLsb0, vec::BitVec};
 use frame_support::{
@@ -140,6 +139,14 @@ impl<H, N> CandidatePendingAvailability<H, N> {
 		&self.descriptor
 	}
 
+	/// Get the candidate's relay parent's number.
+	pub(crate) fn relay_parent_number(&self) -> N
+	where
+		N: Clone,
+	{
+		self.relay_parent_number.clone()
+	}
+
 	#[cfg(any(feature = "runtime-benchmarks", test))]
 	pub(crate) fn new(
 		core: CoreIndex,
@@ -194,8 +201,7 @@ impl<H> Default for ProcessedCandidates<H> {
 
 /// Number of backing votes we need for a valid backing.
 ///
-/// WARNING: This check has to be kept in sync with the node side check in the backing
-/// subsystem.
+/// WARNING: This check has to be kept in sync with the node side checks.
 pub fn minimum_backing_votes(n_validators: usize) -> usize {
 	// For considerations on this value see:
 	// https://github.com/paritytech/polkadot/pull/1656#issuecomment-999734650
@@ -269,6 +275,7 @@ pub mod pallet {
 		+ dmp::Config
 		+ hrmp::Config
 		+ configuration::Config
+		+ scheduler::Config
 	{
 		type RuntimeEvent: From<Event<Self>> + IsType<<Self as frame_system::Config>::RuntimeEvent>;
 		type DisputesHandler: disputes::DisputesHandler<BlockNumberFor<Self>>;
@@ -330,8 +337,12 @@ pub mod pallet {
 		PrematureCodeUpgrade,
 		/// Output code is too large
 		NewCodeTooLarge,
-		/// Candidate not in parent context.
-		CandidateNotInParentContext,
+		/// The candidate's relay-parent was not allowed. Either it was
+		/// not recent enough or it didn't advance based on the last parachain block.
+		DisallowedRelayParent,
+		/// Failed to compute group index for the core: either it's out of bounds
+		/// or the relay parent doesn't belong to the current session.
+		InvalidAssignment,
 		/// Invalid group index in core assignment.
 		InvalidGroupIndex,
 		/// Insufficient (non-majority) backing.
@@ -595,7 +606,7 @@ impl<T: Config> Pallet<T> {
 	/// Both should be sorted ascending by core index, and the candidates should be a subset of
 	/// scheduled cores. If these conditions are not met, the execution of the function fails.
 	pub(crate) fn process_candidates<GV>(
-		parent_storage_root: T::Hash,
+		allowed_relay_parents: &AllowedRelayParentsTracker<T::Hash, BlockNumberFor<T>>,
 		candidates: Vec<BackedCandidate<T::Hash>>,
 		scheduled: Vec<CoreAssignment<BlockNumberFor<T>>>,
 		group_validators: GV,
@@ -603,6 +614,8 @@ impl<T: Config> Pallet<T> {
 	where
 		GV: Fn(GroupIndex) -> Option<Vec<ValidatorIndex>>,
 	{
+		let now = <frame_system::Pallet<T>>::block_number();
+
 		ensure!(candidates.len() <= scheduled.len(), Error::<T>::UnscheduledCandidate);
 
 		if scheduled.is_empty() {
@@ -610,13 +623,6 @@ impl<T: Config> Pallet<T> {
 		}
 
 		let validators = shared::Pallet::<T>::active_validator_keys();
-		let parent_hash = <frame_system::Pallet<T>>::parent_hash();
-
-		// At the moment we assume (and in fact enforce, below) that the relay-parent is always one
-		// before of the block where we include a candidate (i.e. this code path).
-		let now = <frame_system::Pallet<T>>::block_number();
-		let relay_parent_number = now - One::one();
-		let check_ctx = CandidateCheckContext::<T>::new(now, relay_parent_number);
 
 		// Collect candidate receipts with backers.
 		let mut candidate_receipt_with_backing_validator_indices =
@@ -639,9 +645,6 @@ impl<T: Config> Pallet<T> {
 					Ok(())
 				};
 
-			let signing_context =
-				SigningContext { parent_hash, session_index: shared::Pallet::<T>::session_index() };
-
 			// We combine an outer loop over candidates with an inner loop over the scheduled,
 			// where each iteration of the outer loop picks up at the position
 			// in scheduled just after the past iteration left off.
@@ -655,18 +658,27 @@ impl<T: Config> Pallet<T> {
 			'next_backed_candidate: for (candidate_idx, backed_candidate) in
 				candidates.iter().enumerate()
 			{
-				match check_ctx.verify_backed_candidate(
-					parent_hash,
-					parent_storage_root,
+				let relay_parent_hash = backed_candidate.descriptor().relay_parent;
+				let para_id = backed_candidate.descriptor().para_id;
+
+				let prev_context = <paras::Pallet<T>>::para_most_recent_context(para_id);
+
+				let check_ctx = CandidateCheckContext::<T>::new(prev_context);
+				let signing_context = SigningContext {
+					parent_hash: relay_parent_hash,
+					session_index: shared::Pallet::<T>::session_index(),
+				};
+
+				let relay_parent_number = match check_ctx.verify_backed_candidate(
+					&allowed_relay_parents,
 					candidate_idx,
 					backed_candidate,
 				)? {
 					Err(FailedToCreatePVD) => {
 						log::debug!(
 							target: LOG_TARGET,
-							"Failed to create PVD for candidate {} on relay parent {:?}",
+							"Failed to create PVD for candidate {}",
 							candidate_idx,
-							parent_hash,
 						);
 						// We don't want to error out here because it will
 						// brick the relay-chain. So we return early without
@@ -674,7 +686,7 @@ impl<T: Config> Pallet<T> {
 						return Ok(ProcessedCandidates::default())
 					},
 					Ok(rpn) => rpn,
-				}
+				};
 
 				let para_id = backed_candidate.descriptor().para_id;
 				let mut backers = bitvec::bitvec![u8, BitOrderLsb0; 0; validators.len()];
@@ -692,7 +704,22 @@ impl<T: Config> Pallet<T> {
 						// account for already skipped, and then skip this one.
 						skip = i + skip + 1;
 
-						let group_vals = group_validators(core_assignment.group_idx)
+						// The candidate based upon relay parent `N` should be backed by a group
+						// assigned to core at block `N + 1`. Thus, `relay_parent_number + 1`
+						// will always land in the current session.
+						let group_idx = <scheduler::Pallet<T>>::group_assigned_to_core(
+							core_assignment.core,
+							relay_parent_number + One::one(),
+						)
+						.ok_or_else(|| {
+							log::warn!(
+								target: LOG_TARGET,
+								"Failed to compute group index for candidate {}",
+								candidate_idx
+							);
+							Error::<T>::InvalidAssignment
+						})?;
+						let group_vals = group_validators(group_idx)
 							.ok_or_else(|| Error::<T>::InvalidGroupIndex)?;
 
 						// check the signatures in the backing and that it is a majority.
@@ -746,7 +773,8 @@ impl<T: Config> Pallet<T> {
 						core_indices_and_backers.push((
 							(core_assignment.core, core_assignment.paras_entry.para_id()),
 							backers,
-							core_assignment.group_idx,
+							group_idx,
+							relay_parent_number,
 						));
 						continue 'next_backed_candidate
 					}
@@ -767,8 +795,8 @@ impl<T: Config> Pallet<T> {
 		};
 
 		// one more sweep for actually writing to storage.
-		let core_indices = core_indices_and_backers.iter().map(|(c, _, _)| *c).collect();
-		for (candidate, (core, backers, group)) in
+		let core_indices = core_indices_and_backers.iter().map(|(c, ..)| *c).collect();
+		for (candidate, (core, backers, group, relay_parent_number)) in
 			candidates.into_iter().zip(core_indices_and_backers)
 		{
 			let para_id = candidate.descriptor().para_id;
@@ -798,7 +826,7 @@ impl<T: Config> Pallet<T> {
 					availability_votes,
 					relay_parent_number,
 					backers: backers.to_bitvec(),
-					backed_in_number: check_ctx.now,
+					backed_in_number: now,
 					backing_group: group,
 				},
 			);
@@ -814,17 +842,16 @@ impl<T: Config> Pallet<T> {
 	/// Run the acceptance criteria checks on the given candidate commitments.
 	pub(crate) fn check_validation_outputs_for_runtime_api(
 		para_id: ParaId,
+		relay_parent_number: BlockNumberFor<T>,
 		validation_outputs: primitives::CandidateCommitments,
 	) -> bool {
-		// This function is meant to be called from the runtime APIs against the relay-parent, hence
-		// `relay_parent_number` is equal to `now`.
-		let now = <frame_system::Pallet<T>>::block_number();
-		let relay_parent_number = now;
-		let check_ctx = CandidateCheckContext::<T>::new(now, relay_parent_number);
+		let prev_context = <paras::Pallet<T>>::para_most_recent_context(para_id);
+		let check_ctx = CandidateCheckContext::<T>::new(prev_context);
 
 		if check_ctx
 			.check_validation_outputs(
 				para_id,
+				relay_parent_number,
 				&validation_outputs.head_data,
 				&validation_outputs.new_validation_code,
 				validation_outputs.processed_downward_messages,
@@ -919,6 +946,11 @@ impl<T: Config> Pallet<T> {
 		))
 	}
 
+	pub(crate) fn relay_dispatch_queue_size(para_id: ParaId) -> (u32, u32) {
+		let fp = T::MessageQueue::footprint(AggregateMessageOrigin::Ump(UmpQueueId::Para(para_id)));
+		(fp.count as u32, fp.size as u32)
+	}
+
 	/// Check that all the upward messages sent by a candidate pass the acceptance criteria.
 	pub(crate) fn check_upward_messages(
 		config: &HostConfiguration<BlockNumberFor<T>>,
@@ -930,46 +962,42 @@ impl<T: Config> Pallet<T> {
 			ensure!(upward_messages.is_empty(), UmpAcceptanceCheckErr::IsOffboarding);
 		}
 
-		let additional_msgs = upward_messages.len();
-		if additional_msgs > config.max_upward_message_num_per_candidate as usize {
+		let additional_msgs = upward_messages.len() as u32;
+		if additional_msgs > config.max_upward_message_num_per_candidate {
 			return Err(UmpAcceptanceCheckErr::MoreMessagesThanPermitted {
-				sent: additional_msgs as u32,
+				sent: additional_msgs,
 				permitted: config.max_upward_message_num_per_candidate,
 			})
 		}
 
-		let fp = T::MessageQueue::footprint(AggregateMessageOrigin::Ump(UmpQueueId::Para(para)));
-		let (para_queue_count, mut para_queue_size) = (fp.count, fp.size);
+		let (para_queue_count, mut para_queue_size) = Self::relay_dispatch_queue_size(para);
 
-		if para_queue_count.saturating_add(additional_msgs as u64) >
-			config.max_upward_queue_count as u64
-		{
+		if para_queue_count.saturating_add(additional_msgs) > config.max_upward_queue_count {
 			return Err(UmpAcceptanceCheckErr::CapacityExceeded {
-				count: para_queue_count.saturating_add(additional_msgs as u64),
-				limit: config.max_upward_queue_count as u64,
+				count: para_queue_count.saturating_add(additional_msgs).into(),
+				limit: config.max_upward_queue_count.into(),
 			})
 		}
 
 		for (idx, msg) in upward_messages.into_iter().enumerate() {
-			let msg_size = msg.len();
-			if msg_size > config.max_upward_message_size as usize {
+			let msg_size = msg.len() as u32;
+			if msg_size > config.max_upward_message_size {
 				return Err(UmpAcceptanceCheckErr::MessageSize {
 					idx: idx as u32,
-					msg_size: msg_size as u32,
+					msg_size,
 					max_size: config.max_upward_message_size,
 				})
 			}
 			// make sure that the queue is not overfilled.
 			// we do it here only once since returning false invalidates the whole relay-chain
 			// block.
-			if para_queue_size.saturating_add(msg_size as u64) > config.max_upward_queue_size as u64
-			{
+			if para_queue_size.saturating_add(msg_size) > config.max_upward_queue_size {
 				return Err(UmpAcceptanceCheckErr::TotalSizeExceeded {
-					total_size: para_queue_size.saturating_add(msg_size as u64),
-					limit: config.max_upward_queue_size as u64,
+					total_size: para_queue_size.saturating_add(msg_size).into(),
+					limit: config.max_upward_queue_size.into(),
 				})
 			}
-			para_queue_size.saturating_accrue(msg_size as u64);
+			para_queue_size.saturating_accrue(msg_size);
 		}
 
 		Ok(())
@@ -1164,8 +1192,7 @@ impl<T: Config> OnQueueChanged<AggregateMessageOrigin> for Pallet<T> {
 /// A collection of data required for checking a candidate.
 pub(crate) struct CandidateCheckContext<T: Config> {
 	config: configuration::HostConfiguration<BlockNumberFor<T>>,
-	now: BlockNumberFor<T>,
-	relay_parent_number: BlockNumberFor<T>,
+	prev_context: Option<BlockNumberFor<T>>,
 }
 
 /// An error indicating that creating Persisted Validation Data failed
@@ -1173,33 +1200,41 @@ pub(crate) struct CandidateCheckContext<T: Config> {
 pub(crate) struct FailedToCreatePVD;
 
 impl<T: Config> CandidateCheckContext<T> {
-	pub(crate) fn new(now: BlockNumberFor<T>, relay_parent_number: BlockNumberFor<T>) -> Self {
-		Self { config: <configuration::Pallet<T>>::config(), now, relay_parent_number }
+	pub(crate) fn new(prev_context: Option<BlockNumberFor<T>>) -> Self {
+		Self { config: <configuration::Pallet<T>>::config(), prev_context }
 	}
 
 	/// Execute verification of the candidate.
 	///
 	/// Assures:
-	///  * correct expected relay parent reference
+	///  * relay-parent in-bounds
 	///  * collator signature check passes
 	///  * code hash of commitments matches current code hash
 	///  * para head in the descriptor and commitments match
+	///
+	/// Returns the relay-parent block number.
 	pub(crate) fn verify_backed_candidate(
 		&self,
-		parent_hash: <T as frame_system::Config>::Hash,
-		parent_storage_root: T::Hash,
+		allowed_relay_parents: &AllowedRelayParentsTracker<T::Hash, BlockNumberFor<T>>,
 		candidate_idx: usize,
 		backed_candidate: &BackedCandidate<<T as frame_system::Config>::Hash>,
-	) -> Result<Result<(), FailedToCreatePVD>, Error<T>> {
+	) -> Result<Result<BlockNumberFor<T>, FailedToCreatePVD>, Error<T>> {
 		let para_id = backed_candidate.descriptor().para_id;
-		let now = <frame_system::Pallet<T>>::block_number();
-		let relay_parent_number = now - One::one();
+		let relay_parent = backed_candidate.descriptor().relay_parent;
+
+		// Check that the relay-parent is one of the allowed relay-parents.
+		let (relay_parent_storage_root, relay_parent_number) = {
+			match allowed_relay_parents.acquire_info(relay_parent, self.prev_context) {
+				None => return Err(Error::<T>::DisallowedRelayParent),
+				Some(info) => info,
+			}
+		};
 
 		{
 			let persisted_validation_data = match crate::util::make_persisted_validation_data::<T>(
 				para_id,
 				relay_parent_number,
-				parent_storage_root,
+				relay_parent_storage_root,
 			)
 			.defensive_proof("the para is registered")
 			{
@@ -1215,11 +1250,6 @@ impl<T: Config> CandidateCheckContext<T> {
 			);
 		}
 
-		// we require that the candidate is in the context of the parent block.
-		ensure!(
-			backed_candidate.descriptor().relay_parent == parent_hash,
-			Error::<T>::CandidateNotInParentContext,
-		);
 		ensure!(
 			backed_candidate.descriptor().check_collator_signature().is_ok(),
 			Error::<T>::NotCollatorSigned,
@@ -1241,6 +1271,7 @@ impl<T: Config> CandidateCheckContext<T> {
 
 		if let Err(err) = self.check_validation_outputs(
 			para_id,
+			relay_parent_number,
 			&backed_candidate.candidate.commitments.head_data,
 			&backed_candidate.candidate.commitments.new_validation_code,
 			backed_candidate.candidate.commitments.processed_downward_messages,
@@ -1256,14 +1287,29 @@ impl<T: Config> CandidateCheckContext<T> {
 			);
 			Err(err.strip_into_dispatch_err::<T>())?;
 		};
-		Ok(Ok(()))
+		Ok(Ok(relay_parent_number))
 	}
 
 	/// Check the given outputs after candidate validation on whether it passes the acceptance
 	/// criteria.
+	///
+	/// The things that are checked can be roughly divided into limits and minimums.
+	///
+	/// Limits are things like max message queue sizes and max head data size.
+	///
+	/// Minimums are things like the minimum amount of messages that must be processed
+	/// by the parachain block.
+	///
+	/// Limits are checked against the current state. The parachain block must be acceptable
+	/// by the current relay-chain state regardless of whether it was acceptable at some relay-chain
+	/// state in the past.
+	///
+	/// Minimums are checked against the current state but modulated by
+	/// considering the information available at the relay-parent of the parachain block.
 	fn check_validation_outputs(
 		&self,
 		para_id: ParaId,
+		relay_parent_number: BlockNumberFor<T>,
 		head_data: &HeadData,
 		new_validation_code: &Option<primitives::ValidationCode>,
 		processed_downward_messages: u32,
@@ -1289,9 +1335,13 @@ impl<T: Config> CandidateCheckContext<T> {
 		}
 
 		// check if the candidate passes the messaging acceptance criteria
-		<dmp::Pallet<T>>::check_processed_downward_messages(para_id, processed_downward_messages)?;
+		<dmp::Pallet<T>>::check_processed_downward_messages(
+			para_id,
+			relay_parent_number,
+			processed_downward_messages,
+		)?;
 		Pallet::<T>::check_upward_messages(&self.config, para_id, upward_messages)?;
-		<hrmp::Pallet<T>>::check_hrmp_watermark(para_id, self.relay_parent_number, hrmp_watermark)?;
+		<hrmp::Pallet<T>>::check_hrmp_watermark(para_id, relay_parent_number, hrmp_watermark)?;
 		<hrmp::Pallet<T>>::check_outbound_hrmp(&self.config, para_id, horizontal_messages)?;
 
 		Ok(())

polkadot/runtime/parachains/src/inclusion/tests.rs

@@ -19,11 +19,12 @@ use crate::{
 	configuration::HostConfiguration,
 	initializer::SessionChangeNotification,
 	mock::{
-		new_test_ext, Configuration, MockGenesisConfig, ParaInclusion, Paras, ParasShared, System,
-		Test,
+		new_test_ext, Configuration, MockGenesisConfig, ParaInclusion, Paras, ParasShared,
+		Scheduler, System, Test,
 	},
 	paras::{ParaGenesisArgs, ParaKind},
 	paras_inherent::DisputedBitfield,
+	shared::AllowedRelayParentsTracker,
 };
 use primitives::{SignedAvailabilityBitfields, UncheckedSignedAvailabilityBitfields};
 
@@ -47,6 +48,7 @@ fn default_config() -> HostConfiguration<BlockNumber> {
 	config.on_demand_cores = 1;
 	config.max_code_size = 0b100000;
 	config.max_head_data_size = 0b100000;
+	config.group_rotation_frequency = u32::MAX;
 	config
 }
 
@@ -73,6 +75,16 @@ pub(crate) fn genesis_config(paras: Vec<(ParaId, ParaKind)>) -> MockGenesisConfi
 	}
 }
 
+fn default_allowed_relay_parent_tracker() -> AllowedRelayParentsTracker<Hash, BlockNumber> {
+	let mut allowed = AllowedRelayParentsTracker::default();
+
+	let relay_parent = System::parent_hash();
+	let parent_number = System::block_number().saturating_sub(1);
+
+	allowed.update(relay_parent, Hash::zero(), parent_number, 1);
+	allowed
+}
+
 #[derive(Debug, Clone, Copy, PartialEq)]
 pub(crate) enum BackingKind {
 	#[allow(unused)]
@@ -301,6 +313,13 @@ impl TestCandidateBuilder {
 
 pub(crate) fn make_vdata_hash(para_id: ParaId) -> Option<Hash> {
 	let relay_parent_number = <frame_system::Pallet<Test>>::block_number() - 1;
+	make_vdata_hash_with_block_number(para_id, relay_parent_number)
+}
+
+fn make_vdata_hash_with_block_number(
+	para_id: ParaId,
+	relay_parent_number: BlockNumber,
+) -> Option<Hash> {
 	let persisted_validation_data = crate::util::make_persisted_validation_data::<Test>(
 		para_id,
 		relay_parent_number,
@@ -877,26 +896,33 @@ fn candidate_checks() {
 			.map(|m| m.into_iter().map(ValidatorIndex).collect::<Vec<_>>())
 		};
 
+		// When processing candidates, we compute the group index from scheduler.
+		let validator_groups = vec![
+			vec![ValidatorIndex(0), ValidatorIndex(1)],
+			vec![ValidatorIndex(2), ValidatorIndex(3)],
+			vec![ValidatorIndex(4)],
+		];
+		Scheduler::set_validator_groups(validator_groups);
+
 		let entry_ttl = 10_000;
 		let thread_collator: CollatorId = Sr25519Keyring::Two.public().into();
 		let chain_a_assignment = CoreAssignment {
 			core: CoreIndex::from(0),
 			paras_entry: ParasEntry::new(Assignment::new(chain_a), entry_ttl),
-			group_idx: GroupIndex::from(0),
 		};
 
 		let chain_b_assignment = CoreAssignment {
 			core: CoreIndex::from(1),
 			paras_entry: ParasEntry::new(Assignment::new(chain_b), entry_ttl),
-			group_idx: GroupIndex::from(1),
 		};
 
 		let thread_a_assignment = CoreAssignment {
 			core: CoreIndex::from(2),
 			paras_entry: ParasEntry::new(Assignment::new(thread_a), entry_ttl),
-			group_idx: GroupIndex::from(2),
 		};
 
+		let allowed_relay_parents = default_allowed_relay_parent_tracker();
+
 		// unscheduled candidate.
 		{
 			let mut candidate = TestCandidateBuilder {
@@ -921,7 +947,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_b_assignment.clone()],
 					&group_validators,
@@ -976,7 +1002,7 @@ fn candidate_checks() {
 			// out-of-order manifests as unscheduled.
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed_b, backed_a],
 					vec![chain_a_assignment.clone(), chain_b_assignment.clone()],
 					&group_validators,
@@ -1009,7 +1035,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_a_assignment.clone()],
 					&group_validators,
@@ -1018,12 +1044,12 @@ fn candidate_checks() {
 			);
 		}
 
-		// candidate not in parent context.
+		// one of candidates is not based on allowed relay parent.
 		{
 			let wrong_parent_hash = Hash::repeat_byte(222);
 			assert!(System::parent_hash() != wrong_parent_hash);
 
-			let mut candidate = TestCandidateBuilder {
+			let mut candidate_a = TestCandidateBuilder {
 				para_id: chain_a,
 				relay_parent: wrong_parent_hash,
 				pov_hash: Hash::repeat_byte(1),
@@ -1031,10 +1057,23 @@ fn candidate_checks() {
 				..Default::default()
 			}
 			.build();
-			collator_sign_candidate(Sr25519Keyring::One, &mut candidate);
 
-			let backed = back_candidate(
-				candidate,
+			let mut candidate_b = TestCandidateBuilder {
+				para_id: chain_b,
+				relay_parent: System::parent_hash(),
+				pov_hash: Hash::repeat_byte(2),
+				persisted_validation_data_hash: make_vdata_hash(chain_b).unwrap(),
+				hrmp_watermark: RELAY_PARENT_NUM,
+				..Default::default()
+			}
+			.build();
+
+			collator_sign_candidate(Sr25519Keyring::One, &mut candidate_a);
+
+			collator_sign_candidate(Sr25519Keyring::Two, &mut candidate_b);
+
+			let backed_a = back_candidate(
+				candidate_a,
 				&validators,
 				group_validators(GroupIndex::from(0)).unwrap().as_ref(),
 				&keystore,
@@ -1042,14 +1081,23 @@ fn candidate_checks() {
 				BackingKind::Threshold,
 			);
 
+			let backed_b = back_candidate(
+				candidate_b,
+				&validators,
+				group_validators(GroupIndex::from(1)).unwrap().as_ref(),
+				&keystore,
+				&signing_context,
+				BackingKind::Threshold,
+			);
+
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
-					vec![backed],
-					vec![chain_a_assignment.clone()],
+					&allowed_relay_parents,
+					vec![backed_b, backed_a],
+					vec![chain_a_assignment.clone(), chain_b_assignment.clone()],
 					&group_validators,
 				),
-				Error::<Test>::CandidateNotInParentContext
+				Error::<Test>::DisallowedRelayParent
 			);
 		}
 
@@ -1082,7 +1130,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![thread_a_assignment.clone()],
 					&group_validators,
@@ -1132,7 +1180,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_a_assignment.clone()],
 					&group_validators,
@@ -1172,7 +1220,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_a_assignment.clone()],
 					&group_validators,
@@ -1216,7 +1264,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_a_assignment.clone()],
 					&group_validators,
@@ -1250,7 +1298,7 @@ fn candidate_checks() {
 
 			assert_eq!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_a_assignment.clone()],
 					&group_validators,
@@ -1285,7 +1333,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_a_assignment.clone()],
 					&group_validators,
@@ -1320,7 +1368,7 @@ fn candidate_checks() {
 
 			assert_noop!(
 				ParaInclusion::process_candidates(
-					Default::default(),
+					&allowed_relay_parents,
 					vec![backed],
 					vec![chain_a_assignment.clone()],
 					&group_validators,
@@ -1382,24 +1430,30 @@ fn backing_works() {
 			.map(|vs| vs.into_iter().map(ValidatorIndex).collect::<Vec<_>>())
 		};
 
-		let entry_ttl = 10_000;
+		// When processing candidates, we compute the group index from scheduler.
+		let validator_groups = vec![
+			vec![ValidatorIndex(0), ValidatorIndex(1)],
+			vec![ValidatorIndex(2), ValidatorIndex(3)],
+			vec![ValidatorIndex(4)],
+		];
+		Scheduler::set_validator_groups(validator_groups);
 
+		let allowed_relay_parents = default_allowed_relay_parent_tracker();
+
+		let entry_ttl = 10_000;
 		let chain_a_assignment = CoreAssignment {
 			core: CoreIndex::from(0),
 			paras_entry: ParasEntry::new(Assignment::new(chain_a), entry_ttl),
-			group_idx: GroupIndex::from(0),
 		};
 
 		let chain_b_assignment = CoreAssignment {
 			core: CoreIndex::from(1),
 			paras_entry: ParasEntry::new(Assignment::new(chain_b), entry_ttl),
-			group_idx: GroupIndex::from(1),
 		};
 
 		let thread_a_assignment = CoreAssignment {
 			core: CoreIndex::from(2),
 			paras_entry: ParasEntry::new(Assignment::new(thread_a), entry_ttl),
-			group_idx: GroupIndex::from(2),
 		};
 
 		let mut candidate_a = TestCandidateBuilder {
@@ -1486,7 +1540,7 @@ fn backing_works() {
 			core_indices: occupied_cores,
 			candidate_receipt_with_backing_validator_indices,
 		} = ParaInclusion::process_candidates(
-			Default::default(),
+			&allowed_relay_parents,
 			backed_candidates.clone(),
 			vec![
 				chain_a_assignment.clone(),
@@ -1661,12 +1715,21 @@ fn can_include_candidate_with_ok_code_upgrade() {
 			.map(|vs| vs.into_iter().map(ValidatorIndex).collect::<Vec<_>>())
 		};
 
-		let entry_ttl = 10_000;
+		// When processing candidates, we compute the group index from scheduler.
+		let validator_groups = vec![vec![
+			ValidatorIndex(0),
+			ValidatorIndex(1),
+			ValidatorIndex(2),
+			ValidatorIndex(3),
+			ValidatorIndex(4),
+		]];
+		Scheduler::set_validator_groups(validator_groups);
 
+		let allowed_relay_parents = default_allowed_relay_parent_tracker();
+		let entry_ttl = 10_000;
 		let chain_a_assignment = CoreAssignment {
 			core: CoreIndex::from(0),
 			paras_entry: ParasEntry::new(Assignment::new(chain_a), entry_ttl),
-			group_idx: GroupIndex::from(0),
 		};
 
 		let mut candidate_a = TestCandidateBuilder {
@@ -1692,7 +1755,7 @@ fn can_include_candidate_with_ok_code_upgrade() {
 
 		let ProcessedCandidates { core_indices: occupied_cores, .. } =
 			ParaInclusion::process_candidates(
-				Default::default(),
+				&allowed_relay_parents,
 				vec![backed_a],
 				vec![chain_a_assignment.clone()],
 				&group_validators,
@@ -1725,6 +1788,212 @@ fn can_include_candidate_with_ok_code_upgrade() {
 	});
 }
 
+#[test]
+fn check_allowed_relay_parents() {
+	let chain_a = ParaId::from(1);
+	let chain_b = ParaId::from(2);
+	let thread_a = ParaId::from(3);
+
+	let paras = vec![
+		(chain_a, ParaKind::Parachain),
+		(chain_b, ParaKind::Parachain),
+		(thread_a, ParaKind::Parathread),
+	];
+	let validators = vec![
+		Sr25519Keyring::Alice,
+		Sr25519Keyring::Bob,
+		Sr25519Keyring::Charlie,
+		Sr25519Keyring::Dave,
+		Sr25519Keyring::Ferdie,
+	];
+	let keystore: KeystorePtr = Arc::new(LocalKeystore::in_memory());
+	for validator in validators.iter() {
+		Keystore::sr25519_generate_new(
+			&*keystore,
+			PARACHAIN_KEY_TYPE_ID,
+			Some(&validator.to_seed()),
+		)
+		.unwrap();
+	}
+	let validator_public = validator_pubkeys(&validators);
+	let mut config = genesis_config(paras);
+	config.configuration.config.group_rotation_frequency = 1;
+
+	new_test_ext(config).execute_with(|| {
+		shared::Pallet::<Test>::set_active_validators_ascending(validator_public.clone());
+		shared::Pallet::<Test>::set_session_index(5);
+
+		run_to_block(5, |_| None);
+
+		let group_validators = |group_index: GroupIndex| {
+			match group_index {
+				group_index if group_index == GroupIndex::from(0) => Some(vec![0, 1]),
+				group_index if group_index == GroupIndex::from(1) => Some(vec![2, 3]),
+				group_index if group_index == GroupIndex::from(2) => Some(vec![4]),
+				_ => panic!("Group index out of bounds for 2 parachains and 1 parathread core"),
+			}
+			.map(|vs| vs.into_iter().map(ValidatorIndex).collect::<Vec<_>>())
+		};
+
+		// When processing candidates, we compute the group index from scheduler.
+		let validator_groups = vec![
+			vec![ValidatorIndex(0), ValidatorIndex(1)],
+			vec![ValidatorIndex(2), ValidatorIndex(3)],
+			vec![ValidatorIndex(4)],
+		];
+		Scheduler::set_validator_groups(validator_groups);
+
+		// Base each candidate on one of allowed relay parents.
+		//
+		// Note that the group rotation frequency is set to 1 above,
+		// which means groups shift at each relay parent.
+		//
+		// For example, candidate `a` is based on block 1,
+		// thus it will be included in block 2, its group index is
+		// core = 0 shifted 2 times: one for group rotation and one for
+		// fetching the group assigned to the next block.
+		//
+		// Candidates `b` and `c` are constructed accordingly.
+
+		let relay_parent_a = (1, Hash::repeat_byte(0x1));
+		let relay_parent_b = (2, Hash::repeat_byte(0x2));
+		let relay_parent_c = (3, Hash::repeat_byte(0x3));
+
+		let mut allowed_relay_parents = AllowedRelayParentsTracker::default();
+		let max_ancestry_len = 3;
+		allowed_relay_parents.update(
+			relay_parent_a.1,
+			Hash::zero(),
+			relay_parent_a.0,
+			max_ancestry_len,
+		);
+		allowed_relay_parents.update(
+			relay_parent_b.1,
+			Hash::zero(),
+			relay_parent_b.0,
+			max_ancestry_len,
+		);
+		allowed_relay_parents.update(
+			relay_parent_c.1,
+			Hash::zero(),
+			relay_parent_c.0,
+			max_ancestry_len,
+		);
+
+		let chain_a_assignment = CoreAssignment {
+			core: CoreIndex::from(0),
+			paras_entry: ParasEntry {
+				assignment: Assignment { para_id: chain_a },
+				availability_timeouts: 0,
+				ttl: 5,
+			},
+		};
+
+		let chain_b_assignment = CoreAssignment {
+			core: CoreIndex::from(1),
+			paras_entry: ParasEntry {
+				assignment: Assignment { para_id: chain_b },
+				availability_timeouts: 0,
+				ttl: 5,
+			},
+		};
+
+		let thread_a_assignment = CoreAssignment {
+			core: CoreIndex::from(2),
+			paras_entry: ParasEntry::new(Assignment::new(thread_a), 5),
+		};
+
+		let mut candidate_a = TestCandidateBuilder {
+			para_id: chain_a,
+			relay_parent: relay_parent_a.1,
+			pov_hash: Hash::repeat_byte(1),
+			persisted_validation_data_hash: make_vdata_hash_with_block_number(
+				chain_a,
+				relay_parent_a.0,
+			)
+			.unwrap(),
+			hrmp_watermark: relay_parent_a.0,
+			..Default::default()
+		}
+		.build();
+		collator_sign_candidate(Sr25519Keyring::One, &mut candidate_a);
+		let signing_context_a = SigningContext { parent_hash: relay_parent_a.1, session_index: 5 };
+
+		let mut candidate_b = TestCandidateBuilder {
+			para_id: chain_b,
+			relay_parent: relay_parent_b.1,
+			pov_hash: Hash::repeat_byte(2),
+			persisted_validation_data_hash: make_vdata_hash_with_block_number(
+				chain_b,
+				relay_parent_b.0,
+			)
+			.unwrap(),
+			hrmp_watermark: relay_parent_b.0,
+			..Default::default()
+		}
+		.build();
+		collator_sign_candidate(Sr25519Keyring::One, &mut candidate_b);
+		let signing_context_b = SigningContext { parent_hash: relay_parent_b.1, session_index: 5 };
+
+		let mut candidate_c = TestCandidateBuilder {
+			para_id: thread_a,
+			relay_parent: relay_parent_c.1,
+			pov_hash: Hash::repeat_byte(3),
+			persisted_validation_data_hash: make_vdata_hash_with_block_number(
+				thread_a,
+				relay_parent_c.0,
+			)
+			.unwrap(),
+			hrmp_watermark: relay_parent_c.0,
+			..Default::default()
+		}
+		.build();
+		collator_sign_candidate(Sr25519Keyring::Two, &mut candidate_c);
+		let signing_context_c = SigningContext { parent_hash: relay_parent_c.1, session_index: 5 };
+
+		let backed_a = back_candidate(
+			candidate_a.clone(),
+			&validators,
+			group_validators(GroupIndex::from(2)).unwrap().as_ref(),
+			&keystore,
+			&signing_context_a,
+			BackingKind::Threshold,
+		);
+
+		let backed_b = back_candidate(
+			candidate_b.clone(),
+			&validators,
+			group_validators(GroupIndex::from(1)).unwrap().as_ref(),
+			&keystore,
+			&signing_context_b,
+			BackingKind::Threshold,
+		);
+
+		let backed_c = back_candidate(
+			candidate_c.clone(),
+			&validators,
+			group_validators(GroupIndex::from(0)).unwrap().as_ref(),
+			&keystore,
+			&signing_context_c,
+			BackingKind::Threshold,
+		);
+
+		let backed_candidates = vec![backed_a, backed_b, backed_c];
+
+		ParaInclusion::process_candidates(
+			&allowed_relay_parents,
+			backed_candidates.clone(),
+			vec![
+				chain_a_assignment.clone(),
+				chain_b_assignment.clone(),
+				thread_a_assignment.clone(),
+			],
+			&group_validators,
+		)
+		.expect("candidates scheduled, in order, and backed");
+	});
+}
+
 #[test]
 fn session_change_wipes() {
 	let chain_a = ParaId::from(1_u32);
@@ -1911,11 +2180,23 @@ fn para_upgrade_delay_scheduled_from_inclusion() {
 			.map(|vs| vs.into_iter().map(ValidatorIndex).collect::<Vec<_>>())
 		};
 
+		// When processing candidates, we compute the group index from scheduler.
+		let validator_groups = vec![vec![
+			ValidatorIndex(0),
+			ValidatorIndex(1),
+			ValidatorIndex(2),
+			ValidatorIndex(3),
+			ValidatorIndex(4),
+		]];
+		Scheduler::set_validator_groups(validator_groups);
+
 		let core_lookup = |core| match core {
 			core if core == CoreIndex::from(0) => Some(chain_a),
 			_ => None,
 		};
 
+		let allowed_relay_parents = default_allowed_relay_parent_tracker();
+
 		let chain_a_assignment = CoreAssignment {
 			core: CoreIndex::from(0),
 			paras_entry: ParasEntry {
@@ -1923,7 +2204,6 @@ fn para_upgrade_delay_scheduled_from_inclusion() {
 				availability_timeouts: 0,
 				ttl: 5,
 			},
-			group_idx: GroupIndex::from(0),
 		};
 
 		let mut candidate_a = TestCandidateBuilder {
@@ -1949,7 +2229,7 @@ fn para_upgrade_delay_scheduled_from_inclusion() {
 
 		let ProcessedCandidates { core_indices: occupied_cores, .. } =
 			ParaInclusion::process_candidates(
-				Default::default(),
+				&allowed_relay_parents,
 				vec![backed_a],
 				vec![chain_a_assignment.clone()],
 				&group_validators,

polkadot/runtime/parachains/src/lib.rs

@@ -79,12 +79,12 @@ pub fn schedule_para_cleanup<T: paras::Config>(id: primitives::Id) -> Result<(),
 	<paras::Pallet<T>>::schedule_para_cleanup(id).map_err(|_| ())
 }
 
-/// Schedule a parathread to be upgraded to a parachain.
+/// Schedule a parathread (on-demand parachain) to be upgraded to a lease holding parachain.
 pub fn schedule_parathread_upgrade<T: paras::Config>(id: ParaId) -> Result<(), ()> {
 	paras::Pallet::<T>::schedule_parathread_upgrade(id).map_err(|_| ())
 }
 
-/// Schedule a parachain to be downgraded to a parathread.
+/// Schedule a lease holding parachain to be downgraded to an on-demand parachain.
 pub fn schedule_parachain_downgrade<T: paras::Config>(id: ParaId) -> Result<(), ()> {
 	paras::Pallet::<T>::schedule_parachain_downgrade(id).map_err(|_| ())
 }

polkadot/runtime/parachains/src/paras/benchmarking.rs

@@ -99,6 +99,10 @@ benchmarks! {
 	verify {
 		assert_last_event::<T>(Event::CurrentHeadUpdated(para_id).into());
 	}
+	force_set_most_recent_context {
+		let para_id = ParaId::from(1000);
+		let context = BlockNumberFor::<T>::from(1000u32);
+	}: _(RawOrigin::Root, para_id, context)
 	force_schedule_code_upgrade {
 		let c in 1 .. MAX_CODE_SIZE;
 		let new_code = ValidationCode(vec![0; c as usize]);

polkadot/runtime/parachains/src/paras/mod.rs

@@ -18,15 +18,15 @@
 //!
 //! # Tracking State of Paras
 //!
-//! The most important responsibility of this module is to track which parachains and parathreads
+//! The most important responsibility of this module is to track which parachains
 //! are active and what their current state is. The current state of a para consists of the current
 //! head data and the current validation code (AKA Parachain Validation Function (PVF)).
 //!
 //! A para is not considered live until it is registered and activated in this pallet.
 //!
-//! The set of parachains and parathreads cannot change except at session boundaries. This is
-//! primarily to ensure that the number and meaning of bits required for the availability bitfields
-//! does not change except at session boundaries.
+//! The set of parachains cannot change except at session boundaries. This is primarily to ensure
+//! that the number and meaning of bits required for the availability bitfields does not change
+//! except at session boundaries.
 //!
 //! # Validation Code Upgrades
 //!
@@ -61,7 +61,8 @@
 //!
 //! # Para Lifecycle Management
 //!
-//! A para can be in one of the two stable states: it is either a parachain or a parathread.
+//! A para can be in one of the two stable states: it is either a lease holding parachain or an
+//! on-demand parachain.
 //!
 //! However, in order to get into one of those two states, it must first be onboarded. Onboarding
 //! can be only enacted at session boundaries. Onboarding must take at least one full session.
@@ -179,17 +180,17 @@ pub struct ParaPastCodeMeta<N> {
 /// state will be used to determine the state transition to apply to the para.
 #[derive(PartialEq, Eq, Clone, Encode, Decode, RuntimeDebug, TypeInfo)]
 pub enum ParaLifecycle {
-	/// Para is new and is onboarding as a Parathread or Parachain.
+	/// Para is new and is onboarding as an on-demand or lease holding Parachain.
 	Onboarding,
-	/// Para is a Parathread.
+	/// Para is a Parathread (on-demand parachain).
 	Parathread,
-	/// Para is a Parachain.
+	/// Para is a lease holding Parachain.
 	Parachain,
-	/// Para is a Parathread which is upgrading to a Parachain.
+	/// Para is a Parathread (on-demand parachain) which is upgrading to a lease holding Parachain.
 	UpgradingParathread,
-	/// Para is a Parachain which is downgrading to a Parathread.
+	/// Para is a lease holding Parachain which is downgrading to an on-demand parachain.
 	DowngradingParachain,
-	/// Parathread is queued to be offboarded.
+	/// Parathread (on-demand parachain) is queued to be offboarded.
 	OffboardingParathread,
 	/// Parachain is queued to be offboarded.
 	OffboardingParachain,
@@ -197,14 +198,14 @@ pub enum ParaLifecycle {
 
 impl ParaLifecycle {
 	/// Returns true if parachain is currently onboarding. To learn if the
-	/// parachain is onboarding as a parachain or parathread, look at the
+	/// parachain is onboarding as a lease holding or on-demand parachain, look at the
 	/// `UpcomingGenesis` storage item.
 	pub fn is_onboarding(&self) -> bool {
 		matches!(self, ParaLifecycle::Onboarding)
 	}
 
 	/// Returns true if para is in a stable state, i.e. it is currently
-	/// a parachain or parathread, and not in any transition state.
+	/// a lease holding or on-demand parachain, and not in any transition state.
 	pub fn is_stable(&self) -> bool {
 		matches!(self, ParaLifecycle::Parathread | ParaLifecycle::Parachain)
 	}
@@ -221,7 +222,7 @@ impl ParaLifecycle {
 		)
 	}
 
-	/// Returns true if para is currently treated as a parathread.
+	/// Returns true if para is currently treated as a parathread (on-demand parachain).
 	/// This also includes transitioning states, so you may want to combine
 	/// this check with `is_stable` if you specifically want `Paralifecycle::Parathread`.
 	pub fn is_parathread(&self) -> bool {
@@ -294,12 +295,12 @@ pub struct ParaGenesisArgs {
 	pub genesis_head: HeadData,
 	/// The initial validation code to use.
 	pub validation_code: ValidationCode,
-	/// Parachain or Parathread.
+	/// Lease holding or on-demand parachain.
 	#[serde(rename = "parachain")]
 	pub para_kind: ParaKind,
 }
 
-/// Distinguishes between Parachain and Parathread
+/// Distinguishes between lease holding Parachain and Parathread (on-demand parachain)
 #[derive(PartialEq, Eq, Clone, RuntimeDebug)]
 pub enum ParaKind {
 	Parathread,
@@ -483,6 +484,7 @@ impl<BlockNumber> PvfCheckActiveVoteState<BlockNumber> {
 pub trait WeightInfo {
 	fn force_set_current_code(c: u32) -> Weight;
 	fn force_set_current_head(s: u32) -> Weight;
+	fn force_set_most_recent_context() -> Weight;
 	fn force_schedule_code_upgrade(c: u32) -> Weight;
 	fn force_note_new_head(s: u32) -> Weight;
 	fn force_queue_action() -> Weight;
@@ -504,6 +506,9 @@ impl WeightInfo for TestWeightInfo {
 	fn force_set_current_head(_s: u32) -> Weight {
 		Weight::MAX
 	}
+	fn force_set_most_recent_context() -> Weight {
+		Weight::MAX
+	}
 	fn force_schedule_code_upgrade(_c: u32) -> Weight {
 		Weight::MAX
 	}
@@ -606,9 +611,9 @@ pub mod pallet {
 		CannotOnboard,
 		/// Para cannot be offboarded at this time.
 		CannotOffboard,
-		/// Para cannot be upgraded to a parachain.
+		/// Para cannot be upgraded to a lease holding parachain.
 		CannotUpgrade,
-		/// Para cannot be downgraded to a parathread.
+		/// Para cannot be downgraded to an on-demand parachain.
 		CannotDowngrade,
 		/// The statement for PVF pre-checking is stale.
 		PvfCheckStatementStale,
@@ -644,7 +649,8 @@ pub mod pallet {
 	pub(super) type PvfActiveVoteList<T: Config> =
 		StorageValue<_, Vec<ValidationCodeHash>, ValueQuery>;
 
-	/// All parachains. Ordered ascending by `ParaId`. Parathreads are not included.
+	/// All lease holding parachains. Ordered ascending by `ParaId`. On demand parachains are not
+	/// included.
 	///
 	/// Consider using the [`ParachainsCache`] type of modifying.
 	#[pallet::storage]
@@ -660,6 +666,12 @@ pub mod pallet {
 	#[pallet::getter(fn para_head)]
 	pub(super) type Heads<T: Config> = StorageMap<_, Twox64Concat, ParaId, HeadData>;
 
+	/// The context (relay-chain block number) of the most recent parachain head.
+	#[pallet::storage]
+	#[pallet::getter(fn para_most_recent_context)]
+	pub(super) type MostRecentContext<T: Config> =
+		StorageMap<_, Twox64Concat, ParaId, BlockNumberFor<T>>;
+
 	/// The validation code hash of every live para.
 	///
 	/// Corresponding code can be retrieved with [`CodeByHash`].
@@ -706,6 +718,7 @@ pub mod pallet {
 	///
 	/// Corresponding code can be retrieved with [`CodeByHash`].
 	#[pallet::storage]
+	#[pallet::getter(fn future_code_hash)]
 	pub(super) type FutureCodeHash<T: Config> =
 		StorageMap<_, Twox64Concat, ParaId, ValidationCodeHash>;
 
@@ -733,6 +746,7 @@ pub mod pallet {
 	/// NOTE that this field is used by parachains via merkle storage proofs, therefore changing
 	/// the format will require migration of parachains.
 	#[pallet::storage]
+	#[pallet::getter(fn upgrade_restriction_signal)]
 	pub(super) type UpgradeRestrictionSignal<T: Config> =
 		StorageMap<_, Twox64Concat, ParaId, UpgradeRestriction>;
 
@@ -1059,6 +1073,19 @@ pub mod pallet {
 				Ok(Some(<T as Config>::WeightInfo::include_pvf_check_statement()).into())
 			}
 		}
+
+		/// Set the storage for the current parachain head data immediately.
+		#[pallet::call_index(8)]
+		#[pallet::weight(<T as Config>::WeightInfo::force_set_most_recent_context())]
+		pub fn force_set_most_recent_context(
+			origin: OriginFor<T>,
+			para: ParaId,
+			context: BlockNumberFor<T>,
+		) -> DispatchResult {
+			ensure_root(origin)?;
+			MostRecentContext::<T>::insert(&para, context);
+			Ok(())
+		}
 	}
 
 	#[pallet::validate_unsigned]
@@ -1200,7 +1227,7 @@ impl<T: Config> Pallet<T> {
 	// The actions to take are based on the lifecycle of of the paras.
 	//
 	// The final state of any para after the actions queue should be as a
-	// parachain, parathread, or not registered. (stable states)
+	// lease holding parachain, on-demand parachain, or not registered. (stable states)
 	//
 	// Returns the list of outgoing paras from the actions queue.
 	fn apply_actions_queue(session: SessionIndex) -> Vec<ParaId> {
@@ -1219,22 +1246,23 @@ impl<T: Config> Pallet<T> {
 						Self::initialize_para_now(&mut parachains, para, &genesis_data);
 					}
 				},
-				// Upgrade a parathread to a parachain
+				// Upgrade an on-demand parachain to a lease holding parachain
 				Some(ParaLifecycle::UpgradingParathread) => {
 					parachains.add(para);
 					ParaLifecycles::<T>::insert(&para, ParaLifecycle::Parachain);
 				},
-				// Downgrade a parachain to a parathread
+				// Downgrade a lease holding parachain to an on-demand parachain
 				Some(ParaLifecycle::DowngradingParachain) => {
 					parachains.remove(para);
 					ParaLifecycles::<T>::insert(&para, ParaLifecycle::Parathread);
 				},
-				// Offboard a parathread or parachain from the system
+				// Offboard a lease holding or on-demand parachain from the system
 				Some(ParaLifecycle::OffboardingParachain) |
 				Some(ParaLifecycle::OffboardingParathread) => {
 					parachains.remove(para);
 
 					Heads::<T>::remove(&para);
+					MostRecentContext::<T>::remove(&para);
 					FutureCodeUpgrades::<T>::remove(&para);
 					UpgradeGoAheadSignal::<T>::remove(&para);
 					UpgradeRestrictionSignal::<T>::remove(&para);
@@ -1676,8 +1704,7 @@ impl<T: Config> Pallet<T> {
 	///
 	/// Will return error if either is true:
 	///
-	/// - para is not a stable parachain or parathread (i.e. [`ParaLifecycle::is_stable`] is
-	///   `false`)
+	/// - para is not a stable parachain (i.e. [`ParaLifecycle::is_stable`] is `false`)
 	/// - para has a pending upgrade.
 	/// - para has unprocessed messages in its UMP queue.
 	///
@@ -1728,7 +1755,7 @@ impl<T: Config> Pallet<T> {
 		Ok(())
 	}
 
-	/// Schedule a parathread to be upgraded to a parachain.
+	/// Schedule a parathread (on-demand parachain) to be upgraded to a lease holding parachain.
 	///
 	/// Will return error if `ParaLifecycle` is not `Parathread`.
 	pub(crate) fn schedule_parathread_upgrade(id: ParaId) -> DispatchResult {
@@ -1747,7 +1774,7 @@ impl<T: Config> Pallet<T> {
 		Ok(())
 	}
 
-	/// Schedule a parachain to be downgraded to a parathread.
+	/// Schedule a lease holding parachain to be downgraded to an on-demand parachain.
 	///
 	/// Noop if `ParaLifecycle` is not `Parachain`.
 	pub(crate) fn schedule_parachain_downgrade(id: ParaId) -> DispatchResult {
@@ -1936,6 +1963,7 @@ impl<T: Config> Pallet<T> {
 		execution_context: BlockNumberFor<T>,
 	) -> Weight {
 		Heads::<T>::insert(&id, new_head);
+		MostRecentContext::<T>::insert(&id, execution_context);
 
 		if let Some(expected_at) = FutureCodeUpgrades::<T>::get(&id) {
 			if expected_at <= execution_context {
@@ -2028,9 +2056,10 @@ impl<T: Config> Pallet<T> {
 		ParaLifecycles::<T>::get(&id).map_or(false, |state| state.is_offboarding())
 	}
 
-	/// Whether a para ID corresponds to any live parachain.
+	/// Whether a para ID corresponds to any live lease holding parachain.
 	///
-	/// Includes parachains which will downgrade to a parathread in the future.
+	/// Includes lease holding parachains which will downgrade to a on-demand parachains in the
+	/// future.
 	pub fn is_parachain(id: ParaId) -> bool {
 		if let Some(state) = ParaLifecycles::<T>::get(&id) {
 			state.is_parachain()
@@ -2039,9 +2068,9 @@ impl<T: Config> Pallet<T> {
 		}
 	}
 
-	/// Whether a para ID corresponds to any live parathread.
+	/// Whether a para ID corresponds to any live parathread (on-demand parachain).
 	///
-	/// Includes parathreads which will upgrade to parachains in the future.
+	/// Includes on-demand parachains which will upgrade to lease holding parachains in the future.
 	pub fn is_parathread(id: ParaId) -> bool {
 		if let Some(state) = ParaLifecycles::<T>::get(&id) {
 			state.is_parathread()
@@ -2138,6 +2167,7 @@ impl<T: Config> Pallet<T> {
 		}
 
 		Heads::<T>::insert(&id, &genesis_data.genesis_head);
+		MostRecentContext::<T>::insert(&id, BlockNumberFor::<T>::from(0u32));
 	}
 
 	#[cfg(test)]

polkadot/runtime/parachains/src/paras/tests.rs

@@ -1669,6 +1669,50 @@ fn verify_para_head_is_externally_accessible() {
 	});
 }
 
+#[test]
+fn most_recent_context() {
+	let validation_code: ValidationCode = vec![1, 2, 3].into();
+
+	let genesis_config = MockGenesisConfig::default();
+
+	new_test_ext(genesis_config).execute_with(|| {
+		const EXPECTED_SESSION: SessionIndex = 1;
+		run_to_block(1, Some(vec![1]));
+
+		let para_id = ParaId::from(111);
+
+		assert_eq!(Paras::para_most_recent_context(para_id), None);
+
+		assert_ok!(Paras::schedule_para_initialize(
+			para_id,
+			ParaGenesisArgs {
+				para_kind: ParaKind::Parachain,
+				genesis_head: vec![1].into(),
+				validation_code: validation_code.clone(),
+			},
+		));
+		submit_super_majority_pvf_votes(&validation_code, EXPECTED_SESSION, true);
+
+		assert_eq!(ParaLifecycles::<Test>::get(&para_id), Some(ParaLifecycle::Onboarding));
+
+		// Two sessions pass, so action queue is triggered.
+		run_to_block(4, Some(vec![3, 4]));
+
+		// Double-check the para is onboarded, the context is set to the recent block.
+		assert_eq!(ParaLifecycles::<Test>::get(&para_id), Some(ParaLifecycle::Parachain));
+		assert_eq!(Paras::para_most_recent_context(para_id), Some(0));
+
+		// Progress para to the new head and check that the recent context is updated.
+		Paras::note_new_head(para_id, vec![4, 5, 6].into(), 3);
+		assert_eq!(Paras::para_most_recent_context(para_id), Some(3));
+
+		// Finally, offboard the para and expect the context to be cleared.
+		assert_ok!(Paras::schedule_para_cleanup(para_id));
+		run_to_block(6, Some(vec![5, 6]));
+		assert_eq!(Paras::para_most_recent_context(para_id), None);
+	})
+}
+
 #[test]
 fn parakind_encodes_decodes_to_bool_scale() {
 	let chain_kind = ParaKind::Parachain.encode();

polkadot/runtime/parachains/src/paras_inherent/mod.rs

@@ -28,8 +28,11 @@ use crate::{
 	inclusion::CandidateCheckContext,
 	initializer,
 	metrics::METRICS,
-	scheduler,
-	scheduler::common::{CoreAssignment, FreedReason},
+	paras,
+	scheduler::{
+		self,
+		common::{CoreAssignment, FreedReason},
+	},
 	shared, ParaId,
 };
 use bitvec::prelude::BitVec;
@@ -356,6 +359,23 @@ impl<T: Config> Pallet<T> {
 		);
 
 		let now = <frame_system::Pallet<T>>::block_number();
+		let config = <configuration::Pallet<T>>::config();
+
+		// Before anything else, update the allowed relay-parents.
+		{
+			let parent_number = now - One::one();
+			let parent_storage_root = *parent_header.state_root();
+
+			shared::AllowedRelayParents::<T>::mutate(|tracker| {
+				tracker.update(
+					parent_hash,
+					parent_storage_root,
+					parent_number,
+					config.async_backing_params.allowed_ancestry_len,
+				);
+			});
+		}
+		let allowed_relay_parents = <shared::Pallet<T>>::allowed_relay_parents();
 
 		let candidates_weight = backed_candidates_weight::<T>(&backed_candidates);
 		let bitfields_weight = signed_bitfields_weight::<T>(&bitfields);
@@ -407,7 +427,6 @@ impl<T: Config> Pallet<T> {
 			log::debug!(target: LOG_TARGET, "Found duplicate statement sets, retaining the first");
 		}
 
-		let config = <configuration::Pallet<T>>::config();
 		let post_conclusion_acceptance_period = config.dispute_post_conclusion_acceptance_period;
 
 		let dispute_statement_set_valid = move |set: DisputeStatementSet| {
@@ -566,28 +585,28 @@ impl<T: Config> Pallet<T> {
 
 		let scheduled = <scheduler::Pallet<T>>::update_claimqueue(freed, now);
 
-		let relay_parent_number = now - One::one();
-		let parent_storage_root = *parent_header.state_root();
-
-		let check_ctx = CandidateCheckContext::<T>::new(now, relay_parent_number);
-
 		METRICS.on_candidates_processed_total(backed_candidates.len() as u64);
 
 		let backed_candidates = sanitize_backed_candidates::<T, _>(
-			parent_hash,
 			backed_candidates,
-			move |candidate_idx: usize,
-			      backed_candidate: &BackedCandidate<<T as frame_system::Config>::Hash>|
-			      -> bool {
+			|candidate_idx: usize,
+			 backed_candidate: &BackedCandidate<<T as frame_system::Config>::Hash>|
+			 -> bool {
+				let para_id = backed_candidate.descriptor().para_id;
+				let prev_context = <paras::Pallet<T>>::para_most_recent_context(para_id);
+				let check_ctx = CandidateCheckContext::<T>::new(prev_context);
+
 				// never include a concluded-invalid candidate
 				current_concluded_invalid_disputes.contains(&backed_candidate.hash()) ||
 					// Instead of checking the candidates with code upgrades twice
 					// move the checking up here and skip it in the training wheels fallback.
 					// That way we avoid possible duplicate checks while assuring all
 					// backed candidates fine to pass on.
+					//
+					// NOTE: this is the only place where we check the relay-parent.
 					check_ctx
-					.verify_backed_candidate(parent_hash, parent_storage_root, candidate_idx, backed_candidate)
-					.is_err()
+						.verify_backed_candidate(&allowed_relay_parents, candidate_idx, backed_candidate)
+						.is_err()
 			},
 			&scheduled[..],
 		);
@@ -595,12 +614,11 @@ impl<T: Config> Pallet<T> {
 		METRICS.on_candidates_sanitized(backed_candidates.len() as u64);
 
 		// Process backed candidates according to scheduled cores.
-		let parent_storage_root = *parent_header.state_root();
 		let inclusion::ProcessedCandidates::<<HeaderFor<T> as HeaderT>::Hash> {
 			core_indices: occupied,
 			candidate_receipt_with_backing_validator_indices,
 		} = <inclusion::Pallet<T>>::process_candidates(
-			parent_storage_root,
+			&allowed_relay_parents,
 			backed_candidates.clone(),
 			scheduled,
 			<scheduler::Pallet<T>>::group_validators,
@@ -897,7 +915,6 @@ fn sanitize_backed_candidates<
 	T: crate::inclusion::Config,
 	F: FnMut(usize, &BackedCandidate<T::Hash>) -> bool,
 >(
-	relay_parent: T::Hash,
 	mut backed_candidates: Vec<BackedCandidate<T::Hash>>,
 	mut candidate_has_concluded_invalid_dispute_or_is_invalid: F,
 	scheduled: &[CoreAssignment<BlockNumberFor<T>>],
@@ -915,12 +932,13 @@ fn sanitize_backed_candidates<
 
 	// Assure the backed candidate's `ParaId`'s core is free.
 	// This holds under the assumption that `Scheduler::schedule` is called _before_.
-	// Also checks the candidate references the correct relay parent.
+	// We don't check the relay-parent because this is done in the closure when
+	// constructing the inherent and during actual processing otherwise.
 
 	backed_candidates.retain(|backed_candidate| {
 		let desc = backed_candidate.descriptor();
-		desc.relay_parent == relay_parent &&
-			scheduled_paras_to_core_idx.get(&desc.para_id).is_some()
+
+		scheduled_paras_to_core_idx.get(&desc.para_id).is_some()
 	});
 
 	// Sort the `Vec` last, once there is a guarantee that these

polkadot/runtime/parachains/src/paras_inherent/tests.rs

@@ -601,8 +601,8 @@ mod enter {
 		sum
 	}
 
-	#[test]
 	// Ensure that when a block is over weight due to disputes and bitfields, we filter.
+	#[test]
 	fn limit_candidates_over_weight_1() {
 		let config = MockGenesisConfig::default();
 		assert!(config.configuration.config.scheduling_lookahead > 0);
@@ -671,6 +671,7 @@ mod enter {
 				limit_inherent_data,
 			));
 
+			// TODO [now]: this assertion fails with async backing runtime.
 			assert_eq!(
 				// The length of this vec is equal to the number of candidates, so we know our 2
 				// backed candidates did not get filtered out
@@ -1244,7 +1245,6 @@ mod sanitizers {
 						Assignment::new(ParaId::from(1_u32 + idx as u32)),
 						entry_ttl,
 					),
-					group_idx: GroupIndex::from(idx as u32),
 					core: core_idx,
 				};
 				ca
@@ -1291,7 +1291,6 @@ mod sanitizers {
 		// happy path
 		assert_eq!(
 			sanitize_backed_candidates::<Test, _>(
-				relay_parent,
 				backed_candidates.clone(),
 				has_concluded_invalid,
 				&scheduled
@@ -1303,19 +1302,6 @@ mod sanitizers {
 		{
 			let scheduled = &Vec::new();
 			assert!(sanitize_backed_candidates::<Test, _>(
-				relay_parent,
-				backed_candidates.clone(),
-				has_concluded_invalid,
-				scheduled
-			)
-			.is_empty());
-		}
-
-		// relay parent mismatch
-		{
-			let relay_parent = Hash::repeat_byte(0xFA);
-			assert!(sanitize_backed_candidates::<Test, _>(
-				relay_parent,
 				backed_candidates.clone(),
 				has_concluded_invalid,
 				&scheduled
@@ -1339,7 +1325,6 @@ mod sanitizers {
 				|_idx: usize, candidate: &BackedCandidate| set.contains(&candidate.hash());
 			assert_eq!(
 				sanitize_backed_candidates::<Test, _>(
-					relay_parent,
 					backed_candidates.clone(),
 					has_concluded_invalid,
 					&scheduled

polkadot/runtime/parachains/src/runtime_api_impl/v5.rs

@@ -126,7 +126,7 @@ pub fn availability_cores<T: initializer::Config>() -> Vec<CoreState<T::Hash, Bl
 		.collect();
 
 	// This will overwrite only `Free` cores if the scheduler module is working as intended.
-	for scheduled in <scheduler::Pallet<T>>::scheduled_claimqueue(now) {
+	for scheduled in <scheduler::Pallet<T>>::scheduled_claimqueue() {
 		core_states[scheduled.core.0 as usize] = CoreState::Scheduled(primitives::ScheduledCore {
 			para_id: scheduled.paras_entry.para_id(),
 			collator: None,
@@ -221,7 +221,12 @@ pub fn check_validation_outputs<T: initializer::Config>(
 	para_id: ParaId,
 	outputs: primitives::CandidateCommitments,
 ) -> bool {
-	<inclusion::Pallet<T>>::check_validation_outputs_for_runtime_api(para_id, outputs)
+	let relay_parent_number = <frame_system::Pallet<T>>::block_number();
+	<inclusion::Pallet<T>>::check_validation_outputs_for_runtime_api(
+		para_id,
+		relay_parent_number,
+		outputs,
+	)
 }
 
 /// Implementation for the `session_index_for_child` function of the runtime API.

polkadot/runtime/parachains/src/runtime_api_impl/vstaging.rs

@@ -15,3 +15,106 @@
 // along with Polkadot.  If not, see <http://www.gnu.org/licenses/>.
 
 //! Put implementations of functions from staging APIs here.
+
+use crate::{configuration, dmp, hrmp, inclusion, initializer, paras, shared};
+use frame_system::pallet_prelude::BlockNumberFor;
+use primitives::{
+	vstaging::{
+		AsyncBackingParams, BackingState, CandidatePendingAvailability, Constraints,
+		InboundHrmpLimitations, OutboundHrmpChannelLimitations,
+	},
+	Id as ParaId,
+};
+use sp_std::prelude::*;
+
+/// Implementation for `StagingParaBackingState` function from the runtime API
+pub fn backing_state<T: initializer::Config>(
+	para_id: ParaId,
+) -> Option<BackingState<T::Hash, BlockNumberFor<T>>> {
+	let config = <configuration::Pallet<T>>::config();
+	// Async backing is only expected to be enabled with a tracker capacity of 1.
+	// Subsequent configuration update gets applied on new session, which always
+	// clears the buffer.
+	//
+	// Thus, minimum relay parent is ensured to have asynchronous backing enabled.
+	let now = <frame_system::Pallet<T>>::block_number();
+	let min_relay_parent_number = <shared::Pallet<T>>::allowed_relay_parents()
+		.hypothetical_earliest_block_number(now, config.async_backing_params.allowed_ancestry_len);
+
+	let required_parent = <paras::Pallet<T>>::para_head(para_id)?;
+	let validation_code_hash = <paras::Pallet<T>>::current_code_hash(para_id)?;
+
+	let upgrade_restriction = <paras::Pallet<T>>::upgrade_restriction_signal(para_id);
+	let future_validation_code =
+		<paras::Pallet<T>>::future_code_upgrade_at(para_id).and_then(|block_num| {
+			// Only read the storage if there's a pending upgrade.
+			Some(block_num).zip(<paras::Pallet<T>>::future_code_hash(para_id))
+		});
+
+	let (ump_msg_count, ump_total_bytes) =
+		<inclusion::Pallet<T>>::relay_dispatch_queue_size(para_id);
+	let ump_remaining = config.max_upward_queue_count - ump_msg_count;
+	let ump_remaining_bytes = config.max_upward_queue_size - ump_total_bytes;
+
+	let dmp_remaining_messages = <dmp::Pallet<T>>::dmq_contents(para_id)
+		.into_iter()
+		.map(|msg| msg.sent_at)
+		.collect();
+
+	let valid_watermarks = <hrmp::Pallet<T>>::valid_watermarks(para_id);
+	let hrmp_inbound = InboundHrmpLimitations { valid_watermarks };
+	let hrmp_channels_out = <hrmp::Pallet<T>>::outbound_remaining_capacity(para_id)
+		.into_iter()
+		.map(|(para, (messages_remaining, bytes_remaining))| {
+			(para, OutboundHrmpChannelLimitations { messages_remaining, bytes_remaining })
+		})
+		.collect();
+
+	let constraints = Constraints {
+		min_relay_parent_number,
+		max_pov_size: config.max_pov_size,
+		max_code_size: config.max_code_size,
+		ump_remaining,
+		ump_remaining_bytes,
+		max_ump_num_per_candidate: config.max_upward_message_num_per_candidate,
+		dmp_remaining_messages,
+		hrmp_inbound,
+		hrmp_channels_out,
+		max_hrmp_num_per_candidate: config.hrmp_max_message_num_per_candidate,
+		required_parent,
+		validation_code_hash,
+		upgrade_restriction,
+		future_validation_code,
+	};
+
+	let pending_availability = {
+		// Note: the API deals with a `Vec` as it is future-proof for cases
+		// where there may be multiple candidates pending availability at a time.
+		// But at the moment only one candidate can be pending availability per
+		// parachain.
+		crate::inclusion::PendingAvailability::<T>::get(&para_id)
+			.and_then(|pending| {
+				let commitments =
+					crate::inclusion::PendingAvailabilityCommitments::<T>::get(&para_id);
+				commitments.map(move |c| (pending, c))
+			})
+			.map(|(pending, commitments)| {
+				CandidatePendingAvailability {
+					candidate_hash: pending.candidate_hash(),
+					descriptor: pending.candidate_descriptor().clone(),
+					commitments,
+					relay_parent_number: pending.relay_parent_number(),
+					max_pov_size: constraints.max_pov_size, // assume always same in session.
+				}
+			})
+			.into_iter()
+			.collect()
+	};
+
+	Some(BackingState { constraints, pending_availability })
+}
+
+/// Implementation for `StagingAsyncBackingParams` function from the runtime API
+pub fn async_backing_params<T: configuration::Config>() -> AsyncBackingParams {
+	<configuration::Pallet<T>>::config().async_backing_params
+}

polkadot/runtime/parachains/src/scheduler.rs

@@ -587,7 +587,7 @@ impl<T: Config> Pallet<T> {
 			debug_assert!(timedout_paras.is_empty());
 			debug_assert!(concluded_paras.is_empty());
 
-			Self::scheduled_claimqueue(now)
+			Self::scheduled_claimqueue()
 		}
 	}
 
@@ -634,9 +634,7 @@ impl<T: Config> Pallet<T> {
 
 	// TODO: Temporary to imitate the old schedule() call. Will be adjusted when we make the
 	// scheduler AB ready
-	pub(crate) fn scheduled_claimqueue(
-		now: BlockNumberFor<T>,
-	) -> Vec<CoreAssignment<BlockNumberFor<T>>> {
+	pub(crate) fn scheduled_claimqueue() -> Vec<CoreAssignment<BlockNumberFor<T>>> {
 		let claimqueue = ClaimQueue::<T>::get();
 
 		claimqueue
@@ -645,20 +643,11 @@ impl<T: Config> Pallet<T> {
 				v.front()
 					.cloned()
 					.flatten()
-					.and_then(|pe| Self::paras_entry_to_core_assignment(now, core_idx, pe))
+					.map(|pe| CoreAssignment { core: core_idx, paras_entry: pe })
 			})
 			.collect()
 	}
 
-	fn paras_entry_to_core_assignment(
-		now: BlockNumberFor<T>,
-		core_idx: CoreIndex,
-		pe: ParasEntry<BlockNumberFor<T>>,
-	) -> Option<CoreAssignment<BlockNumberFor<T>>> {
-		let group_idx = Self::group_assigned_to_core(core_idx, now)?;
-		Some(CoreAssignment { core: core_idx, group_idx, paras_entry: pe })
-	}
-
 	#[cfg(any(feature = "runtime-benchmarks", test))]
 	pub(crate) fn assignment_provider_config(
 		core_idx: CoreIndex,
@@ -675,4 +664,9 @@ impl<T: Config> Pallet<T> {
 	pub(crate) fn claimqueue_is_empty() -> bool {
 		Self::claimqueue_len() == 0
 	}
+
+	#[cfg(test)]
+	pub(crate) fn set_validator_groups(validator_groups: Vec<Vec<ValidatorIndex>>) {
+		ValidatorGroups::<T>::set(validator_groups);
+	}
 }

polkadot/runtime/parachains/src/scheduler/common.rs

@@ -19,7 +19,7 @@
 use frame_support::pallet_prelude::*;
 use primitives::{
 	v5::{Assignment, ParasEntry},
-	CoreIndex, GroupIndex, Id as ParaId,
+	CoreIndex, Id as ParaId,
 };
 use scale_info::TypeInfo;
 use sp_std::prelude::*;
@@ -81,8 +81,6 @@ pub struct CoreAssignment<BlockNumber> {
 	pub core: CoreIndex,
 	/// The para id and accompanying information needed to collate and back a parablock.
 	pub paras_entry: ParasEntry<BlockNumber>,
-	/// The index of the validator group assigned to the core.
-	pub group_idx: GroupIndex,
 }
 
 impl<BlockNumber> CoreAssignment<BlockNumber> {

polkadot/runtime/parachains/src/scheduler/tests.rs

@@ -400,11 +400,11 @@ fn fill_claimqueue_fills() {
 	new_test_ext(genesis_config).execute_with(|| {
 		assert_eq!(default_config().on_demand_cores, 3);
 
-		// register 2 parachains
+		// register 2 lease holding parachains
 		schedule_blank_para(chain_a, ParaKind::Parachain);
 		schedule_blank_para(chain_b, ParaKind::Parachain);
 
-		// and 3 parathreads
+		// and 3 parathreads (on-demand parachains)
 		schedule_blank_para(thread_a, ParaKind::Parathread);
 		schedule_blank_para(thread_b, ParaKind::Parathread);
 		schedule_blank_para(thread_c, ParaKind::Parathread);
@@ -427,7 +427,7 @@ fn fill_claimqueue_fills() {
 
 		{
 			assert_eq!(Scheduler::claimqueue_len(), 2 * lookahead);
-			let scheduled = Scheduler::scheduled_claimqueue(1);
+			let scheduled = Scheduler::scheduled_claimqueue();
 
 			// Cannot assert on indices anymore as they depend on the assignment providers
 			assert!(claimqueue_contains_para_ids::<Test>(vec![chain_a, chain_b]));
@@ -441,7 +441,6 @@ fn fill_claimqueue_fills() {
 						availability_timeouts: 0,
 						ttl: 6
 					},
-					group_idx: GroupIndex(0),
 				}
 			);
 
@@ -454,7 +453,6 @@ fn fill_claimqueue_fills() {
 						availability_timeouts: 0,
 						ttl: 6
 					},
-					group_idx: GroupIndex(1),
 				}
 			);
 		}
@@ -483,7 +481,7 @@ fn fill_claimqueue_fills() {
 
 		{
 			assert_eq!(Scheduler::claimqueue_len(), 5);
-			let scheduled = Scheduler::scheduled_claimqueue(3);
+			let scheduled = Scheduler::scheduled_claimqueue();
 
 			assert_eq!(
 				scheduled[0],
@@ -494,7 +492,6 @@ fn fill_claimqueue_fills() {
 						availability_timeouts: 0,
 						ttl: 6
 					},
-					group_idx: GroupIndex(0),
 				}
 			);
 			assert_eq!(
@@ -506,7 +503,6 @@ fn fill_claimqueue_fills() {
 						availability_timeouts: 0,
 						ttl: 6
 					},
-					group_idx: GroupIndex(1),
 				}
 			);
 
@@ -520,7 +516,6 @@ fn fill_claimqueue_fills() {
 						availability_timeouts: 0,
 						ttl: 7
 					},
-					group_idx: GroupIndex(2),
 				}
 			);
 			// Sits on the same core as `thread_a`
@@ -541,7 +536,6 @@ fn fill_claimqueue_fills() {
 						availability_timeouts: 0,
 						ttl: 7
 					},
-					group_idx: GroupIndex(3),
 				}
 			);
 		}
@@ -574,11 +568,11 @@ fn schedule_schedules_including_just_freed() {
 	new_test_ext(genesis_config).execute_with(|| {
 		assert_eq!(default_config().on_demand_cores, 3);
 
-		// register 2 parachains
+		// register 2 lease holding parachains
 		schedule_blank_para(chain_a, ParaKind::Parachain);
 		schedule_blank_para(chain_b, ParaKind::Parachain);
 
-		// and 5 parathreads
+		// and 5 parathreads (on-demand parachains)
 		schedule_blank_para(thread_a, ParaKind::Parathread);
 		schedule_blank_para(thread_b, ParaKind::Parathread);
 		schedule_blank_para(thread_c, ParaKind::Parathread);
@@ -614,7 +608,7 @@ fn schedule_schedules_including_just_freed() {
 		let mut now = 2;
 		run_to_block(now, |_| None);
 
-		assert_eq!(Scheduler::scheduled_claimqueue(now).len(), 4);
+		assert_eq!(Scheduler::scheduled_claimqueue().len(), 4);
 
 		// cores 0, 1, 2, and 3 should be occupied. mark them as such.
 		let mut occupied_map: BTreeMap<CoreIndex, ParaId> = BTreeMap::new();
@@ -636,7 +630,7 @@ fn schedule_schedules_including_just_freed() {
 			// core 4 is free
 			assert!(cores[4] == CoreOccupied::Free);
 
-			assert!(Scheduler::scheduled_claimqueue(now).is_empty());
+			assert!(Scheduler::scheduled_claimqueue().is_empty());
 
 			// All core index entries in the claimqueue should have `None` in them.
 			Scheduler::claimqueue().iter().for_each(|(_core_idx, core_queue)| {
@@ -663,10 +657,10 @@ fn schedule_schedules_including_just_freed() {
 		run_to_block(now, |_| None);
 
 		{
-			let scheduled = Scheduler::scheduled_claimqueue(now);
+			let scheduled = Scheduler::scheduled_claimqueue();
 
-			// cores 0 and 1 are occupied by parachains. cores 2 and 3 are occupied by parathread
-			// claims. core 4 was free.
+			// cores 0 and 1 are occupied by lease holding parachains. cores 2 and 3 are occupied by
+			// on-demand parachain claims. core 4 was free.
 			assert_eq!(scheduled.len(), 1);
 			assert_eq!(
 				scheduled[0],
@@ -677,7 +671,6 @@ fn schedule_schedules_including_just_freed() {
 						availability_timeouts: 0,
 						ttl: 8
 					},
-					group_idx: GroupIndex(4),
 				}
 			);
 		}
@@ -693,7 +686,7 @@ fn schedule_schedules_including_just_freed() {
 		Scheduler::update_claimqueue(just_updated, now);
 
 		{
-			let scheduled = Scheduler::scheduled_claimqueue(now);
+			let scheduled = Scheduler::scheduled_claimqueue();
 
 			// 1 thing scheduled before, + 3 cores freed.
 			assert_eq!(scheduled.len(), 4);
@@ -706,7 +699,6 @@ fn schedule_schedules_including_just_freed() {
 						availability_timeouts: 0,
 						ttl: 8
 					},
-					group_idx: GroupIndex(0),
 				}
 			);
 			assert_eq!(
@@ -718,7 +710,6 @@ fn schedule_schedules_including_just_freed() {
 						availability_timeouts: 0,
 						ttl: 8
 					},
-					group_idx: GroupIndex(2),
 				}
 			);
 			// Although C was descheduled, the core `4` was occupied so C goes back to the queue.
@@ -731,7 +722,6 @@ fn schedule_schedules_including_just_freed() {
 						availability_timeouts: 1,
 						ttl: 8
 					},
-					group_idx: GroupIndex(3),
 				}
 			);
 			assert_eq!(
@@ -743,7 +733,6 @@ fn schedule_schedules_including_just_freed() {
 						availability_timeouts: 0,
 						ttl: 8
 					},
-					group_idx: GroupIndex(4),
 				}
 			);
 
@@ -911,10 +900,16 @@ fn schedule_rotates_groups() {
 		run_to_block(now, |_| None);
 
 		let assert_groups_rotated = |rotations: u32, now: &BlockNumberFor<Test>| {
-			let scheduled = Scheduler::scheduled_claimqueue(*now);
+			let scheduled = Scheduler::scheduled_claimqueue();
 			assert_eq!(scheduled.len(), 2);
-			assert_eq!(scheduled[0].group_idx, GroupIndex((0u32 + rotations) % on_demand_cores));
-			assert_eq!(scheduled[1].group_idx, GroupIndex((1u32 + rotations) % on_demand_cores));
+			assert_eq!(
+				Scheduler::group_assigned_to_core(scheduled[0].core, *now).unwrap(),
+				GroupIndex((0u32 + rotations) % on_demand_cores)
+			);
+			assert_eq!(
+				Scheduler::group_assigned_to_core(scheduled[1].core, *now).unwrap(),
+				GroupIndex((1u32 + rotations) % on_demand_cores)
+			);
 		};
 
 		assert_groups_rotated(0, &now);

polkadot/runtime/parachains/src/shared.rs

@@ -22,7 +22,8 @@
 use frame_support::pallet_prelude::*;
 use frame_system::pallet_prelude::BlockNumberFor;
 use primitives::{SessionIndex, ValidatorId, ValidatorIndex};
-use sp_std::vec::Vec;
+use sp_runtime::traits::AtLeast32BitUnsigned;
+use sp_std::{collections::vec_deque::VecDeque, vec::Vec};
 
 use rand::{seq::SliceRandom, SeedableRng};
 use rand_chacha::ChaCha20Rng;
@@ -39,6 +40,86 @@ pub(crate) const SESSION_DELAY: SessionIndex = 2;
 #[cfg(test)]
 mod tests;
 
+/// Information about past relay-parents.
+#[derive(Encode, Decode, Default, TypeInfo)]
+pub struct AllowedRelayParentsTracker<Hash, BlockNumber> {
+	// The past relay parents, paired with state roots, that are viable to build upon.
+	//
+	// They are in ascending chronologic order, so the newest relay parents are at
+	// the back of the deque.
+	//
+	// (relay_parent, state_root)
+	buffer: VecDeque<(Hash, Hash)>,
+
+	// The number of the most recent relay-parent, if any.
+	// If the buffer is empty, this value has no meaning and may
+	// be nonsensical.
+	latest_number: BlockNumber,
+}
+
+impl<Hash: PartialEq + Copy, BlockNumber: AtLeast32BitUnsigned + Copy>
+	AllowedRelayParentsTracker<Hash, BlockNumber>
+{
+	/// Add a new relay-parent to the allowed relay parents, along with info about the header.
+	/// Provide a maximum ancestry length for the buffer, which will cause old relay-parents to be
+	/// pruned.
+	pub(crate) fn update(
+		&mut self,
+		relay_parent: Hash,
+		state_root: Hash,
+		number: BlockNumber,
+		max_ancestry_len: u32,
+	) {
+		// + 1 for the most recent block, which is always allowed.
+		let buffer_size_limit = max_ancestry_len as usize + 1;
+
+		self.buffer.push_back((relay_parent, state_root));
+		self.latest_number = number;
+		while self.buffer.len() > buffer_size_limit {
+			let _ = self.buffer.pop_front();
+		}
+
+		// We only allow relay parents within the same sessions, the buffer
+		// gets cleared on session changes.
+	}
+
+	/// Attempt to acquire the state root and block number to be used when building
+	/// upon the given relay-parent.
+	///
+	/// This only succeeds if the relay-parent is one of the allowed relay-parents.
+	/// If a previous relay-parent number is passed, then this only passes if the new relay-parent
+	/// is more recent than the previous.
+	pub(crate) fn acquire_info(
+		&self,
+		relay_parent: Hash,
+		prev: Option<BlockNumber>,
+	) -> Option<(Hash, BlockNumber)> {
+		let pos = self.buffer.iter().position(|(rp, _)| rp == &relay_parent)?;
+		let age = (self.buffer.len() - 1) - pos;
+		let number = self.latest_number - BlockNumber::from(age as u32);
+
+		if let Some(prev) = prev {
+			if prev > number {
+				return None
+			}
+		}
+
+		Some((self.buffer[pos].1, number))
+	}
+
+	/// Returns block number of the earliest block the buffer would contain if
+	/// `now` is pushed into it.
+	pub(crate) fn hypothetical_earliest_block_number(
+		&self,
+		now: BlockNumber,
+		max_ancestry_len: u32,
+	) -> BlockNumber {
+		let allowed_ancestry_len = max_ancestry_len.min(self.buffer.len() as u32);
+
+		now - allowed_ancestry_len.into()
+	}
+}
+
 #[frame_support::pallet]
 pub mod pallet {
 	use super::*;
@@ -68,6 +149,12 @@ pub mod pallet {
 	#[pallet::getter(fn active_validator_keys)]
 	pub(super) type ActiveValidatorKeys<T: Config> = StorageValue<_, Vec<ValidatorId>, ValueQuery>;
 
+	/// All allowed relay-parents.
+	#[pallet::storage]
+	#[pallet::getter(fn allowed_relay_parents)]
+	pub(crate) type AllowedRelayParents<T: Config> =
+		StorageValue<_, AllowedRelayParentsTracker<T::Hash, BlockNumberFor<T>>, ValueQuery>;
+
 	#[pallet::call]
 	impl<T: Config> Pallet<T> {}
 }
@@ -90,6 +177,17 @@ impl<T: Config> Pallet<T> {
 		new_config: &HostConfiguration<BlockNumberFor<T>>,
 		all_validators: Vec<ValidatorId>,
 	) -> Vec<ValidatorId> {
+		// Drop allowed relay parents buffer on a session change.
+		//
+		// During the initialization of the next block we always add its parent
+		// to the tracker.
+		//
+		// With asynchronous backing candidates built on top of relay
+		// parent `R` are still restricted by the runtime to be backed
+		// by the group assigned at `number(R) + 1`, which is guaranteed
+		// to be in the current session.
+		AllowedRelayParents::<T>::mutate(|tracker| tracker.buffer.clear());
+
 		CurrentSessionIndex::<T>::set(session_index);
 		let mut rng: ChaCha20Rng = SeedableRng::from_seed(random_seed);
 

polkadot/runtime/parachains/src/shared/tests.rs

@@ -19,12 +19,81 @@ use crate::{
 	configuration::HostConfiguration,
 	mock::{new_test_ext, MockGenesisConfig, ParasShared},
 };
+use assert_matches::assert_matches;
 use keyring::Sr25519Keyring;
+use primitives::Hash;
 
 fn validator_pubkeys(val_ids: &[Sr25519Keyring]) -> Vec<ValidatorId> {
 	val_ids.iter().map(|v| v.public().into()).collect()
 }
 
+#[test]
+fn tracker_earliest_block_number() {
+	let mut tracker = AllowedRelayParentsTracker::default();
+
+	// Test it on an empty tracker.
+	let now: u32 = 1;
+	let max_ancestry_len = 5;
+	assert_eq!(tracker.hypothetical_earliest_block_number(now, max_ancestry_len), now);
+
+	// Push a single block into the tracker, suppose max capacity is 1.
+	let max_ancestry_len = 0;
+	tracker.update(Hash::zero(), Hash::zero(), 0, max_ancestry_len);
+	assert_eq!(tracker.hypothetical_earliest_block_number(now, max_ancestry_len), now);
+
+	// Test a greater capacity.
+	let max_ancestry_len = 4;
+	let now = 4;
+	for i in 1..now {
+		tracker.update(Hash::zero(), Hash::zero(), i, max_ancestry_len);
+		assert_eq!(tracker.hypothetical_earliest_block_number(i + 1, max_ancestry_len), 0);
+	}
+
+	// Capacity exceeded.
+	tracker.update(Hash::zero(), Hash::zero(), now, max_ancestry_len);
+	assert_eq!(tracker.hypothetical_earliest_block_number(now + 1, max_ancestry_len), 1);
+}
+
+#[test]
+fn tracker_acquire_info() {
+	let mut tracker = AllowedRelayParentsTracker::<Hash, u32>::default();
+	let max_ancestry_len = 2;
+
+	// (relay_parent, state_root) pairs.
+	let blocks = &[
+		(Hash::repeat_byte(0), Hash::repeat_byte(10)),
+		(Hash::repeat_byte(1), Hash::repeat_byte(11)),
+		(Hash::repeat_byte(2), Hash::repeat_byte(12)),
+	];
+
+	let (relay_parent, state_root) = blocks[0];
+	tracker.update(relay_parent, state_root, 0, max_ancestry_len);
+	assert_matches!(
+		tracker.acquire_info(relay_parent, None),
+		Some((s, b)) if s == state_root && b == 0
+	);
+
+	let (relay_parent, state_root) = blocks[1];
+	tracker.update(relay_parent, state_root, 1u32, max_ancestry_len);
+	let (relay_parent, state_root) = blocks[2];
+	tracker.update(relay_parent, state_root, 2u32, max_ancestry_len);
+	for (block_num, (rp, state_root)) in blocks.iter().enumerate().take(2) {
+		assert_matches!(
+			tracker.acquire_info(*rp, None),
+			Some((s, b)) if &s == state_root && b == block_num as u32
+		);
+
+		assert!(tracker.acquire_info(*rp, Some(2)).is_none());
+	}
+
+	for (block_num, (rp, state_root)) in blocks.iter().enumerate().skip(1) {
+		assert_matches!(
+			tracker.acquire_info(*rp, Some(block_num as u32 - 1)),
+			Some((s, b)) if &s == state_root && b == block_num as u32
+		);
+	}
+}
+
 #[test]
 fn sets_and_shuffles_validators() {
 	let validators = vec![