82 Commits

Author	SHA1	Message	Date
Tsvetomir Dimitrov	f8954093b4	Filter votes from disabled validators in BackedCandidates in process_inherent_data (#2889) ... Backport of https://github.com/paritytech/polkadot-sdk/pull/1863 to master Extend candidate sanitation in paras_inherent by removing backing votes from disabled validators. Check https://github.com/paritytech/polkadot-sdk/issues/1592 for more details. This change is related to the disabling strategy implementation (https://github.com/paritytech/polkadot-sdk/pull/2226). --------- Co-authored-by: ordian <noreply@reusable.software> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>	2024-01-18 07:33:58 +00:00
eskimor	69434d9a32	Coretime Feature branch (relay chain) (#1694) ... Also fixes: https://github.com/paritytech/polkadot-sdk/issues/1417 - [x] CoreIndex -> AssignmentProvider mapping will be able to change any time. - [x] Implement - [x] Provide Migrations - [x] Add and fix tests - [x] Implement bulk assigner logic - [x] bulk assigner tests - [x] Port over current assigner to use bulk designer (+ share on-demand with bulk): top-level assigner has core ranges: legacy, bulk - [x] Adjust migrations to reflect new assigner structure - [x] Move migration code to Assignment code directly and make it recursive (make it possible to skip releases) -> follow up ticket. - [x] Test migrations - [x] Add migration PR to runtimes repo -> follow up ticket. - [x] Wire up with actual UMP messages - [x] Write PR docs --------- Co-authored-by: eskimor <eskimor@no-such-url.com> Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: BradleyOlson64 <lotrftw9@gmail.com> Co-authored-by: Anton Vilhelm Ásgeirsson <antonva@users.noreply.github.com> Co-authored-by: antonva <anton.asgeirsson@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Marcin S. <marcin@realemail.net> Co-authored-by: Bastian Köcher <info@kchr.de> Co-authored-by: command-bot <>	2023-12-21 19:06:58 +01:00
vuittont60	4caa3d8d8e	docs: fix typos (#2193)	2023-11-07 13:11:06 +01:00
Chevdor	a30092ab42	Markdown linter (#1309) ... * Add markdown linting - add linter default rules - adapt rules to current code - fix the code for linting to pass - add CI check fix #1243 * Fix markdown for Substrate * Fix tooling install * Fix workflow * Add documentation * Remove trailing spaces * Update .github/.markdownlint.yaml Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix mangled markdown/lists * Fix captalization issues on known words	2023-09-04 12:02:32 +03:00
asynchronous rob	5174b9d2d7	Asynchronous Backing MegaPR (#5022) ... * inclusion emulator logic for asynchronous backing (#4790) * initial stab at candidate_context * fmt * docs & more TODOs * some cleanups * reframe as inclusion_emulator * documentations yes * update types * add constraint modifications * watermark * produce modifications * v2 primitives: re-export all v1 for consistency * vstaging primitives * emulator constraints: handle code upgrades * produce outbound HRMP modifications * stack. * method for applying modifications * method just for sanity-checking modifications * fragments produce modifications, not prospectives * make linear * add some TODOs * remove stacking; handle code upgrades * take `fragment` private * reintroduce stacking. * fragment constructor * add TODO * allow validating fragments against future constraints * docs * relay-parent number and min code size checks * check code upgrade restriction * check max hrmp per candidate * fmt * remove GoAhead logic because it wasn't helpful * docs on code upgrade failure * test stacking * test modifications against constraints * fmt * test fragments * descending or duplicate test * fmt * remove unused imports in vstaging * wrong primitives * spellcheck * Runtime changes for Asynchronous Backing (#4786) * inclusion: utility for allowed relay-parents * inclusion: use prev number instead of prev hash * track most recent context of paras * inclusion: accept previous relay-parents * update dmp advancement rule for async backing * fmt * add a comment about validation outputs * clean up a couple of TODOs * weights * fix weights * fmt * Resolve dmp todo * Restore inclusion tests * Restore paras_inherent tests * MostRecentContext test * Benchmark for new paras dispatchable * Prepare check_validation_outputs for upgrade * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs * Implementers guide changes * More tests for allowed relay parents * Add a github issue link * Compute group index based on relay parent * Storage migration * Move allowed parents tracker to shared * Compile error * Get group assigned to core at the next block * Test group assignment * fmt * Error instead of panic * Update guide * Extend doc-comment * Update runtime/parachains/src/shared.rs Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Prospective Parachains Subsystem (#4913) * docs and skeleton * subsystem skeleton * main loop * fragment tree basics & fmt * begin fragment trees & view * flesh out more of view update logic * further flesh out update logic * some refcount functions for fragment trees * add fatal/non-fatal errors * use non-fatal results * clear up some TODOs * ideal format for scheduling info * add a bunch of TODOs * some more fluff * extract fragment graph to submodule * begin fragment graph API * trees, not graphs * improve docs * scope and constructor for trees * add some test TODOs * limit max ancestors and store constraints * constructor * constraints: fix bug in HRMP watermarks * fragment tree population logic * set::retain * extract population logic * implement add_and_populate * fmt * add some TODOs in tests * implement child-selection * strip out old stuff based on wrong assumptions * use fatality * implement pruning * remove unused ancestor constraints * fragment tree instantiation * remove outdated comment * add message/request types and skeleton for handling * fmt * implement handle_candidate_seconded * candidate storage: handle backed * implement handle_candidate_backed * implement answer_get_backable_candidate * remove async where not needed * implement fetch_ancestry * add logic for run_iteration * add some docs * remove global allow(unused), fix warnings * make spellcheck happy (despite English) * fmt * bump Cargo.lock * replace tracing with gum * introduce PopulateFrom trait * implement GetHypotheticalDepths * revise docs slightly * first fragment tree scope test * more scope tests * test add_candidate * fmt * test retain * refactor test code * test populate is recursive * test contiguity of depth 0 is maintained * add_and_populate tests * cycle tests * remove PopulateFrom trait * fmt * test hypothetical depths (non-recursive) * have CandidateSeconded return membership * tree membership requests * Add a ProspectiveParachainsSubsystem struct * add a staging API for base constraints * add a `From` impl * add runtime API for staging_validity_constraints * implement fetch_base_constraints * implement `fetch_upcoming_paras` * remove reconstruction of candidate receipt; no obvious usecase * fmt * export message to broader module * remove last TODO * correctly export * fix compilation and add GetMinimumRelayParent request * make provisioner into a real subsystem with proper mesage bounds * fmt * fix ChannelsOut in overseer test * fix overseer tests * fix again * fmt * Integrate prospective parachains subsystem into backing: Part 1 (#5557) * BEGIN ASYNC candidate-backing CHANGES * rename & document modes * answer prospective validation data requests * GetMinimumRelayParents request is now plural * implement an implicit view utility for backing subsystems * implicit-view: get allowed relay parents * refactorings and improvements to implicit view * add some TODOs for tests * split implicit view updates into 2 functions * backing: define State to prepare for functional refactor * add some docs * backing: implement bones of new leaf activation logic * backing: create per-relay-parent-states * use new handle_active_leaves_update * begin extracting logic from CandidateBackingJob * mostly extract statement import from job logic * handle statement imports outside of job logic * do some TODO planning for prospective parachains integration * finish rewriting backing subsystem in functional style * add prospective parachains mode to relay parent entries * fmt * add a RejectedByProspectiveParachains error * notify prospective parachains of seconded and backed candidates * always validate candidates exhaustively in backing. * return persisted_validation_data from validation * handle rejections by prospective parachains * implement seconding sanity check * invoke validate_and_second * Alter statement table to allow multiple seconded messages per validator * refactor backing to have statements carry PVD * clean up all warnings * Add tests for implicit view * Improve doc comments * Prospective parachains mode based on Runtime API version * Add a TODO * Rework seconding_sanity_check * Iterate over responses * Update backing tests * collator-protocol: load PVD from runtime * Fix validator side tests * Update statement-distribution to fetch PVD * Fix statement-distribution tests * Backing tests with prospective paras #1 * fix per_relay_parent pruning in backing * Test multiple leaves * Test seconding sanity check * Import statement order Before creating an entry in `PerCandidateState` map wait for the approval from the prospective parachains * Add a test for correct state updates * Second multiple candidates per relay parent test * Add backing tests with prospective paras * Second more than one test without prospective paras * Add a test for prospective para blocks * Update malus * typos Co-authored-by: Chris Sosnin <chris125_@live.com> * Track occupied depth in backing per parachain (#5778) * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * fmt * Network bridge changes for asynchronous backing + update subsystems to handle versioned packets (#5991) * BEGIN STATEMENT DISTRIBUTION WORK create a vstaging network protocol which is the same as v1 * mostly make network bridge amenable to vstaging * network-bridge: fully adapt to vstaging * add some TODOs for tests * fix fallout in bitfield-distribution * bitfield distribution tests + TODOs * fix fallout in gossip-support * collator-protocol: fix message fallout * collator-protocol: load PVD from runtime * add TODO for vstaging tests * make things compile * set used network protocol version using a feature * fmt * get approval-distribution building * fix approval-distribution tests * spellcheck * nits * approval distribution net protocol test * bitfield distribution net protocol test * Revert "collator-protocol: fix message fallout" This reverts commit 07cc887303e16c6b3843ecb25cdc7cc2080e2ed1. * Network bridge tests Co-authored-by: Chris Sosnin <chris125_@live.com> * remove max_pov_size requirement from prospective pvd request (#6014) * remove max_pov_size requirement from prospective pvd request * fmt * Extract legacy statement distribution to its own module (#6026) * add compatibility type to v2 statement distribution message * warning cleanup * handle compatibility layer for v2 * clean up an unimplemented!() block * circulate statements based on version * extract legacy v1 code into separate module * remove unimplemented * clean up naming of from_requester/responder * remove TODOs * have backing share seconded statements with PVD * fmt * fix warning * Quick fix unused warning for not yet implemented/used staging messages. * Fix network bridge test * Fix wrong merge. We now have 23 subsystems (network bridge split + prospective parachains) Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at> * Version 3 is already live. * Fix tests (#6055) * Fix backing tests * Fix warnings. * fmt * collator-protocol: asynchronous backing changes (#5740) * Draft collator side changes * Start working on collations management * Handle peer's view change * Versioning on advertising * Versioned collation fetching request * Handle versioned messages * Improve docs for collation requests * Add spans * Add request receiver to overseer * Fix collator side tests * Extract relay parent mode to lib * Validator side draft * Add more checks for advertisement * Request pvd based on async backing mode * review * Validator side improvements * Make old tests green * More fixes * Collator side tests draft * Send collation test * fmt * Collator side network protocol versioning * cleanup * merge artifacts * Validator side net protocol versioning * Remove fragment tree membership request * Resolve todo * Collator side core state test * Improve net protocol compatibility * Validator side tests * more improvements * style fixes * downgrade log * Track implicit assignments * Limit the number of seconded candidates per para * Add a sanity check * Handle fetched candidate * fix tests * Retry fetch * Guard against dequeueing while already fetching * Reintegrate connection management * Timeout on advertisements * fmt * spellcheck * update tests after merge * validator assignment fixes for backing and collator protocol (#6158) * Rename depth->ancestry len in tests * Refactor group assignments * Remove implicit assignments * backing: consider occupied core assignments * Track a single para on validator side * Refactor prospective parachains mode request (#6179) * Extract prospective parachains mode into util * Skip activations depending on the mode * backing: don't send backed candidate to provisioner (#6185) * backing: introduce `CanSecond` request for advertisements filtering (#6225) * Drop BoundToRelayParent * draft changes * fix backing tests * Fix genesis ancestry * Fix validator side tests * more tests * cargo generate-lockfile * Implement `StagingValidityConstraints` Runtime API method (#6258) * Implement StagingValidityConstraints * spellcheck * fix ump params * Update hrmp comment * Introduce ump per candidate limit * hypothetical earliest block * refactor primitives usage * hypothetical earliest block number test * fix build * Prepare the Runtime for asynchronous backing upgrade (#6287) * Introduce async backing params to runtime config * fix cumulus config * use config * finish runtimes * Introduce new staging API * Update collator protocol * Update provisioner * Update prospective parachains * Update backing * Move async backing params lower in the config * make naming consistent * misc * Use real prospective parachains subsystem (#6407) * Backport `HypotheticalFrontier` into the feature branch (#6605) * implement more general HypotheticalFrontier * fmt * drop unneeded request Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Resolve todo about legacy leaf activation (#6447) * fix bug/warning in handling membership answers * Remove `HypotheticalDepthRequest` in favor of `HypotheticalFrontierRequest` (#6521) * Remove `HypotheticalDepthRequest` for `HypotheticalFrontierRequest` * Update tests * Fix (removed wrong docstring) * Fix can_second request * Patch some dead_code errors --------- Co-authored-by: Chris Sosnin <chris125_@live.com> * Async Backing: Send Statement Distribution "Backed" messages (#6634) * Backing: Send Statement Distribution "Backed" messages Closes #6590. **TODO:** - [ ] Adjust tests * Fix compile errors * (Mostly) fix tests * Fix comment * Fix test and compile error * Test that `StatementDistributionMessage::Backed` is sent * Fix compile error * Fix some clippy errors * Add prospective parachains subsystem tests (#6454) * Add prospective parachains subsystem test * Add `should_do_no_work_if_async_backing_disabled_for_leaf` test * Implement `activate_leaf` helper, up to getting ancestry * Finish implementing `activate_leaf` * Small refactor in `activate_leaf` * Get `CandidateSeconded` working * Finish `send_candidate_and_check_if_found` test * Refactor; send more leaves & candidates * Refactor test * Implement `check_candidate_parent_leaving_view` test * Start work on `check_candidate_on_multiple_forks` test * Don’t associate specific parachains with leaf * Finish `correctly_updates_leaves` test * Fix cycle due to reused head data * Fix `check_backable_query` test * Fix `check_candidate_on_multiple_forks` test * Add `check_depth_and_pvd_queries` test * Address review comments * Remove TODO * add a new index for output head data to candidate storage * Resolve test TODOs * Fix compile errors * test candidate storage pruning, make sure new index is cleaned up --------- Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Node-side metrics for asynchronous backing (#6549) * Add metrics for `prune_view_candidate_storage` * Add metrics for `request_unblocked_collations` * Fix docstring * Couple fixes from review comments * Fix `check_depth_query` test * inclusion-emulator: mirror advancement rule check (#6361) * inclusion-emulator: mirror advancement rule check * fix build * prospective-parachains: introduce `backed_in_path_only` flag for advertisements (#6649) * Introduce `backed_in_path_only` flag for depth request * fmt * update doc comment * fmt * Add async-backing zombienet tests (#6314) * Async backing: impl guide for statement distribution (#6738) Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com> * Asynchronous backing statement distribution: Take III (#5999) * add notification types for v2 statement-distribution * improve protocol docs * add empty vstaging module * fmt * add backed candidate packet request types * start putting down structure of new logic * handle activated leaf * some sanity-checking on outbound statements * fmt * update vstaging share to use statements with PVD * tiny refactor, candidate_hash location * import local statements * refactor statement import * first stab at broadcast logic * fmt * fill out some TODOs * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Mark receiver with `vstaging` * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * fmt, fix grid test after topology change * typo Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * address review * adjust comment, make easier to understand * Fix typo --------- Co-authored-by: Marcin S <marcin@bytedude.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Chris Sosnin <chris125_@live.com> * miscellaneous fixes to make asynchronous backing work (#6791) * propagate network-protocol-staging feature * add feature to adder-collator as well * allow collation-generation of occupied cores * prospective parachains: special treatment for pending availability candidates * runtime: fetch candidates pending availability * lazily construct PVD for pending candidates * fix fallout in prospective parachains hypothetical/select_child * runtime: enact candidates when creating paras-inherent * make tests compile * test pending availability in the scope * add prospective parachains test * fix validity constraints leftovers * drop prints * Fix typos --------- Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Marcin S <marcin@realemail.net> * Remove restart from test (#6840) * Async Backing: Statement Distribution Tests (#6755) * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * Hook up request sender * Add `valid_statement_without_prior_seconded_is_ignored` test * Fix `valid_statement_without_prior_seconded_is_ignored` test * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Remove obsolete test * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * Mark receiver with `vstaging` * First draft of `ensure_seconding_limit_is_respected` test * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * Fix `ensure_seconding_limit_is_respected` test * Start `backed_candidate_leads_to_advertisement` test * fmt, fix grid test after topology change * Send Backed notification * Finish `backed_candidate_leads_to_advertisement` test * Finish `peer_reported_for_duplicate_statements` test * Finish `received_advertisement_before_confirmation_leads_to_request` * Add `advertisements_rejected_from_incorrect_peers` test * Add `manifest_rejected_*` tests * Add `manifest_rejected_when_group_does_not_match_para` test * Add `local_node_sanity_checks_incoming_requests` test * Add `local_node_respects_statement_mask` test * Add tests where peer is reported for providing invalid signatures * Add `cluster_peer_allowed_to_send_incomplete_statements` test * Add `received_advertisement_after_backing_leads_to_acknowledgement` * Add `received_advertisement_after_confirmation_before_backing` test * peer_reported_for_advertisement_conflicting_with_confirmed_candidate * Add `peer_reported_for_not_enough_statements` test * Add `peer_reported_for_providing_statements_meant_to_be_masked_out` * Add `additional_statements_are_shared_after_manifest_exchange` * Add `grid_statements_imported_to_backing` test * Add `relay_parent_entering_peer_view_leads_to_advertisement` test * Add `advertisement_not_re_sent_when_peer_re_enters_view` test * Update node/network/statement-distribution/src/vstaging/tests/grid.rs Co-authored-by: asynchronous rob <rphmeier@gmail.com> * Resolve TODOs, update test * Address unused code * Add check after every test for unhandled requests * Refactor (`make_dummy_leaf` and `handle_sent_request`) * Refactor (`make_dummy_topology`) * Minor refactor --------- Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Chris Sosnin <chris125_@live.com> * Fix some clippy lints in tests * Async backing: minor fixes (#6920) * bitfield-distribution test * implicit view tests * Refactor parameters -> params * scheduler: update storage migration (#6963) * update scheduler migration * Adjust weight to account for storage read * Statement Distribution Guide Edits (#7025) * Statement distribution guide edits * Addressed Marcin's comments * Add attested candidate request retry timeouts (#6833) Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Chris Sosnin <chris125_@live.com> Fix async backing statement distribution tests (#6621) Resolve some todos in async backing statement-distribution branch (#6482) Fix clippy errors in statement distribution branch (#6720) * Async backing: add Prospective Parachains impl guide (#6933) Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> * Updates to Provisioner Guide for Async Backing (#7106) * Initial corrections and clarifications * Partial first draft * Finished first draft * Adding back wrongly removed test bit * fmt * Update roadmap/implementers-guide/src/node/utility/provisioner.md Co-authored-by: Marcin S. <marcin@realemail.net> * Addressing comments * Reorganization * fmt --------- Co-authored-by: Marcin S. <marcin@realemail.net> * fmt * Renaming Parathread Mentions (#7287) * Renaming parathreads * Renaming module to pallet * More updates * PVF: Refactor workers into separate crates, remove host dependency (#7253) * PVF: Refactor workers into separate crates, remove host dependency * Fix compile error * Remove some leftover code * Fix compile errors * Update Cargo.lock * Remove worker main.rs files I accidentally copied these from the other PR. This PR isn't intended to introduce standalone workers yet. * Address review comments * cargo fmt * Update a couple of comments * Update log targets * Update quote to 1.0.27 (#7280) Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: parity-processbot <> * pallets: implement `Default` for `GenesisConfig` in `no_std` (#7271) * pallets: implement Default for GenesisConfig in no_std This change is follow-up of: https://github.com/paritytech/substrate/pull/14108 It is a step towards: https://github.com/paritytech/substrate/issues/13334 * Cargo.lock updated * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * cli: enable BEEFY by default on test networks (#7293) We consider BEEFY mature enough to run by default on all nodes for test networks (Rococo/Wococo/Versi). Right now, most nodes are not running it since it's opt-in using --beefy flag. Switch to an opt-out model for test networks. Replace --beefy flag from CLI with --no-beefy and have BEEFY client start by default on test networks. Signed-off-by: acatangiu <adrian@parity.io> * runtime: past session slashing runtime API (#6667) * runtime/vstaging: unapplied_slashes runtime API * runtime/vstaging: key_ownership_proof runtime API * runtime/ParachainHost: submit_report_dispute_lost * fix key_ownership_proof API * runtime: submit_report_dispute_lost runtime API * nits * Update node/subsystem-types/src/messages.rs Co-authored-by: Marcin S. <marcin@bytedude.com> * revert unrelated fmt changes * post merge fixes * fix compilation --------- Co-authored-by: Marcin S. <marcin@bytedude.com> * Correcting git mishap * Document usage of `gum` crate (#7294) * Document usage of gum crate * Small fix * Add some more basic info * Update node/gum/src/lib.rs Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * Update target docs --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * XCM: Fix issue with RequestUnlock (#7278) * XCM: Fix issue with RequestUnlock * Leave API changes for v4 * Fix clippy errors * Fix tests --------- Co-authored-by: parity-processbot <> * Companion for Substrate#14228 (#7295) * Companion for Substrate#14228 https://github.com/paritytech/substrate/pull/14228 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * Companion for #14237: Use latest sp-crates (#7300) * To revert: Update substrate branch to "lexnv/bump_sp_crates" Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert "To revert: Update substrate branch to "lexnv/bump_sp_crates"" This reverts commit 5f1db84eac4a226c37b7f6ce6ee19b49dc7e2008. * Update cargo lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * bounded-collections bump to 0.1.7 (#7305) * bounded-collections bump to 0.1.7 Companion for: paritytech/substrate#14225 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * bump to quote 1.0.28 (#7306) * `RollingSessionWindow` cleanup (#7204) * Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit * Fix tests in import * Fix the rest of the tests * Remove dead code * Fix todos * Simplify session caching * Comments for `SessionInfoProvider` * Separate `SessionInfoProvider` from `State` * `cache_session_info_for_head` becomes freestanding function * Remove unneeded `mut` usage * fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider` * Fix SessionInfo retrieval * Code cleanup * Don't wrap `SessionInfoProvider` in an `Option` * Remove `earliest_session()` * Remove pre-caching -> wip * Fix some tests and code cleanup * Fix all tests * Fixes in tests * Fix comments, variable names and small style changes * Fix a warning * impl From<SessionWindowSize> for NonZeroUsize * Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG * Code review feedback * Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db * Remove `col_session_data` usages * Storage migration clearing columns w/o removing them * Remove session data column usages from `approval-voting` and `dispute-coordinator` tests * Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests * Fix formatting in initialized.rs * Fix a corner case in `SessionInfo` caching for `dispute-coordinator` * Remove `RollingSessionWindow` ;( * Revert "Fix formatting in initialized.rs" This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc. * v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it * Fix `NUM_COLUMNS` in `approval-voting` * Use `columns::v3::NUM_COLUMNS` when opening db * Update node/service/src/parachains_db/upgrade.rs Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3` * Fix `NUM+COLUMNS` in approval_voting * Fix formatting * Fix columns usage * Clarification comments about the different db versions --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * pallet-para-config: Remove remnant WeightInfo functions (#7308) * pallet-para-config: Remove remnant WeightInfo functions Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * set_config_with_weight begone Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::configuration --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: command-bot <> * XCM: PayOverXcm config (#6900) * Move XCM query functionality to trait * Fix tests * Add PayOverXcm implementation * fix the PayOverXcm trait to compile * moved doc comment out of trait implmeentation and to the trait * PayOverXCM documentation * Change documentation a bit * Added empty benchmark methods implementation and changed docs * update PayOverXCM to convert AccountIds to MultiLocations * Implement benchmarking method * Change v3 to latest * Descend origin to an asset sender (#6970) * descend origin to an asset sender * sender as tuple of dest and sender * Add more variants to the QueryResponseStatus enum * Change Beneficiary to Into<[u8; 32]> * update PayOverXcm to return concrete errors and use AccountId as sender * use polkadot-primitives for AccountId * fix dependency to use polkadot-core-primitives * force Unpaid instruction to the top of the instructions list * modify report_outcome to accept interior argument * use new_query directly for building final xcm query, instead of report_outcome * fix usage of new_query to use the XcmQueryHandler * fix usage of new_query to use the XcmQueryHandler * tiny method calling fix * xcm query handler (#7198) * drop redundant query status * rename ReportQueryStatus to OuterQueryStatus * revert rename of QueryResponseStatus * update mapping * Update xcm/xcm-builder/src/pay.rs Co-authored-by: Gavin Wood <gavin@parity.io> * Updates * Docs * Fix benchmarking stuff * Destination can be determined based on asset_kind * Tweaking API to minimise clones * Some repotting and docs --------- Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> * Companion for #14265 (#7307) * Update Cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update Cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: parity-processbot <> * bump serde to 1.0.163 (#7315) * bump serde to 1.0.163 * bump ci * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * fmt * Updated fmt * Removing changes accidentally pulled from master * fix another master pull issue * Another master pull fix * fmt * Fixing implementers guide build * Revert "Merge branch 'rh-async-backing-feature-while-frozen' of https://github.com/paritytech/polkadot into brad-rename-parathread" This reverts commit bebc24af52ab61155e3fe02cb3ce66a592bce49c, reversing changes made to 1b2de662dfb11173679d6da5bd0da9d149c85547. --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: acatangiu <adrian@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Marcin S. <marcin@bytedude.com> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> * fix bitfield distribution test * approval distribution tests * fix bridge tests * update Cargo.lock * [async-backing-branch] Optimize collator-protocol validator-side request fetching (#7457) * Optimize collator-protocol validator-side request fetching * address feedback: replace tuples with structs * feedback: add doc comments * move collation types to subfolder --------- Signed-off-by: alindima <alin@parity.io> * Update collation generation for asynchronous backing (#7405) * break candidate receipt construction and distribution into own function * update implementers' guide to include SubmitCollation * implement SubmitCollation for collation-generation * fmt * fix test compilation & remove unnecessary submodule * add some TODOs for a test suite. * Update roadmap/implementers-guide/src/types/overseer-protocol.md Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * add new test harness and first test * refactor to avoid requiring background sender * ensure collation gets packaged and distributed * tests for the fallback case with no hint * add parent rp-number hint tests * fmt * update uses of CollationGenerationConfig * fix remaining test * address review comments * use subsystemsender for background tasks * fmt * remove ValidationCodeHashHint and related tests --------- Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> * fix some more fallout from merge * fmt * remove staging APIs from Rococo & Westend (#7513) * send network messages on main protocol name (#7515) * misc async backing improvements for allowed ancestry blocks (#7532) * shared: fix acquire_info * backwards-compat test for prospective parachains * same relay parent is allowed * provisioner: request candidate receipt by relay parent (#7527) * return candidates hash from prospective parachains * update provisioner * update tests * guide changes * send a single message to backing * fix test * revert to old `handle_new_activations` logic in some cases (#7514) * revert to old `handle_new_activations` logic * gate sending messages on scheduled cores to max_depth >= 2 * fmt * 2->1 * Omnibus asynchronous backing bugfix PR (#7529) * fix a bug in backing * add some more logs * prospective parachains: take ancestry only up to session bounds * add test * fix zombienet tests (#7614) Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix runtime compilation * make bitfield distribution tests compile * attempt to fix zombienet disputes (#7618) * update metric name * update some metric names * avoid cycles when creating fake candidates * make undying collator more friendly to malformed parents * fix a bug in malus * fmt * clippy * add RUN_IN_CONTAINER to new ZombieNet tests (#7631) * remove duplicated migration happened because of master-merge --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Signed-off-by: acatangiu <adrian@parity.io> Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Signed-off-by: alindima <alin@parity.io> Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Chris Sosnin <chris125_@live.com> Co-authored-by: Parity Bot <admin@parity.io> Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at> Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com> Co-authored-by: Marcin S <marcin@bytedude.com> Co-authored-by: Marcin S <marcin@realemail.net> Co-authored-by: Mattia L.V. Bradascio <28816406+bredamatt@users.noreply.github.com> Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: alexgparity <115470171+alexgparity@users.noreply.github.com> Co-authored-by: BradleyOlson64 <lotrftw9@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> Co-authored-by: Adrian Catangiu <adrian@parity.io> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Sam Johnson <sam@durosoft.com> Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io> Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com> Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com> Co-authored-by: Anthony Alaribe <anthony.alaribe@parity.io> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Alin Dima <alin@parity.io>	2023-08-18 16:11:56 +00:00
gupnik	2802414473	Moves Block to frame_system instead of construct_runtime and removes Header and BlockNumber (#7431) ... * Companion for substrate * Minor update * Formatting * Fixes for cumulus * Fixes tests in polkadot-runtime-parachains * Minor update * Removes unused import * Fixes tests in polkadot-runtime-common * Minor fix * Update roadmap/implementers-guide/src/runtime/configuration.md Co-authored-by: ordian <write@reusable.software> * ".git/.scripts/commands/fmt/fmt.sh" * update lockfile for {"substrate"} --------- Co-authored-by: ordian <write@reusable.software> Co-authored-by: command-bot <>	2023-07-13 12:37:50 +00:00
eskimor	596a9ccd02	Unify code paths of create_inherent and enter (#7137) ... * Remove redundant enter call. * Remove optionality in dispute signature checking * Make enter_inner and create_inherent the same. * Remove redundant metric. * Unification: enter and create_inherent. * Remove `enter_inner` function. * Remove dead code. * Remove redundant import. * Remove dead code in disputes. * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent * Merge fix. * Fix tests. * Remove obsolete comment. * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent * Review remarks, fixes. * Guide updates. * Fmt. * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent --------- Co-authored-by: eskimor <eskimor@no-such-url.com> Co-authored-by: command-bot <>	2023-06-14 10:57:06 +02:00
Gavin Wood	400864c352	Use Message Queue pallet for UMP dispatch (#6271) ... * Add ProcessXcmMessage struct * Migrate away from weights in host config * New well-known key to report UMPQ capacity * Add missing file * Fixes * Remove original UMP files * Docs * Update runtime/parachains/src/inclusion/mod.rs Co-authored-by: asynchronous rob <rphmeier@gmail.com> * Add benchmarking * Benchmarks * Mock example of using the QueueChangeHandler to update the WKK * Use master Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Merge remote-tracking branch 'origin/master' into gav-message-queue * Update Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update remove-weight migration The migration got touched on master; just resolving conflicts here. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add message- to dispatch-origin conversion for XCM processing Just using the `impl Into<MultiLocation>` was a bit inflexible. Like this, the Relaychain can convert `UMP(para)` to a MultiLocation `para`. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * DNM: Temporarily comment code since XCMv3 is not merged yet Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use u64 for queue-wide limits on UmpAcceptanceCheckErr Using u32 here was one audit finding for the queue pallet. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Define one sub-queue per *MP queue Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Harden check_upward_messages Using safe math and casts. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add type-safe well_known_keys Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add message-queue weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Deploy MessageQueue to Polkadot Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update Cargo.toml Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Migrate to parachain config V5 Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update UMP tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cleanup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert messed up merge 🤦 Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update remove-weight migration The migration got touched on master; just resolving conflicts here. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add message- to dispatch-origin conversion for XCM processing Just using the `impl Into<MultiLocation>` was a bit inflexible. Like this, the Relaychain can convert `UMP(para)` to a MultiLocation `para`. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * DNM: Temporarily comment code since XCMv3 is not merged yet Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use u64 for queue-wide limits on UmpAcceptanceCheckErr Using u32 here was one audit finding for the queue pallet. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Define one sub-queue per *MP queue Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Harden check_upward_messages Using safe math and casts. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add type-safe well_known_keys Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add message-queue weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Deploy MessageQueue to Polkadot Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update Cargo.toml Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Migrate to parachain config V5 Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update UMP tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cleanup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Make stuff compile Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Move DMP and HRMP messages to the MessageQueue It currently does not compile in the CIbecause of some local tweaks to Substrate. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Diener for CI Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * diener update cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cleanup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert wrong changes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "DNM: Temporarily comment code since XCMv3 is not merged yet" This reverts commit 820aa235cb21dd1d2621843607f7682bf035434e. * Make compile Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixup runtimes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Define benchmarks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cleanup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use master Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Lockfile Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add AggregateMessageOrigin This enum currently only holds one value, but having it will make it easier in the future to extend. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Forbid UMP for off-boarding paras - Reject candidates with UMP messages for off-boarding paras - Forbid scheduling off-boarding when a para has unprocess UMPs Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Delete stupid test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use BoundedVec for upward messages Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add weights and fix MessageProcessor Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Bound receive_upward_messages and check bound in configuration pallet Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Bound Debug impl Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix test runtime Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix xcm-simulator Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Properly fix xcm-simulator and fuzzer Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * cargo update -p sp-io Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Adapt to upstream Substrate changes Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix ProcesseMessage impls Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Some tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use master Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * cargo update -p sp-io Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use new MQ API Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add UMP while Para offboarding tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use Mocked message processor for benchmarking Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use variables for constants Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add MQ pallet weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use MQ pallet weights Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Configure QueueChangeHandler Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add config test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix MQ serive weight Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cleanup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Cleanup outgoing UMP dispatch queues Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use Master Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update Cargo.lock Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Weight mul is not const Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove merge marker Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update runtime/parachains/src/inclusion/mod.rs Co-authored-by: Gavin Wood <gavin@parity.io> * Update runtime/kusama/src/lib.rs Co-authored-by: Gavin Wood <gavin@parity.io> * Use lowercase UMP Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Clarify comment Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use Weight::from_parts Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix test Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix doc Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Emit event after the fact Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add defensive_proof to receive_upward_messages Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Reapply "Remove original UMP files" Looks like they came back from the dead. Re-apply commit cf6d316f0 Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove old files Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Rename MaxUmpMessageLen -> MaxUmpMessageLenOf Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Test defensive message dropping of receive_upward_messages Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixup imports Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update implementors guide Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove FAIL-CI mark Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Delete unused code Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add another test for MQ change hook Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Imports Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Keep Kusama runtime formatting Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "Delete unused code" This reverts commit dd76bca5025b7e1ef846a9539c3607eed185f16a. * Feature gate mock functions Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Review: Use saturating_add Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Test RelayDispatchQueueSize storage key Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Move migration to own file to avoid merge conflicts Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Migration in own file Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixup migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Rococo: configure MQ pallet Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixup tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "Rococo: configure MQ pallet" Going to do this as follow up, since it needs Substrate changes and i dont want to stall this MR any longer. This reverts commit b9c15e8a8339c4e877d654ee3f09903af4210736. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "Fixup tests" This reverts commit 88f1cbe20774d20e5e1e554e798960ae39437af1. * Fixup migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix CI Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix other migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Bump MAX_CODE_SIZE to 10MiB Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add ForceUpdateUmpLimits migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * clippy Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * imports Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use defensive instead of defensive_proof 'defensive_proof' also prints the 'self', which spams the console too much when running the tests. Just the length is enough. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Rename to ScheduleConfigUpdate Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixup migration checks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add MAX_CODE_SIZE to ScheduleConfigUpdate Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Set MAX_CODE_SIZE to 4MiB Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix benchmark Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix formatting Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "Add MAX_CODE_SIZE to ScheduleConfigUpdate" This reverts commit 7caffb09e83083b57affd548215e45b25c3d64dc. * Revert "Set MAX_CODE_SIZE to 4MiB" This reverts commit 103ffbaf686487d2fbe0082a16826af17cacc1a1. * Revert "Bump MAX_CODE_SIZE to 10MiB" This reverts commit 530734b7b0da5b7680054e0242348fcc79a666fe. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove consistency check from migration Re-addig these checks is blocked on https://github.com/paritytech/polkadot/issues/7108 Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix constants Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Bump MAX_UPWARD_MESSAGE_SIZE_BOUND for Westend Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix migrations Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Use old nightly for fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixes * cargo fmt * Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix tests Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fixes * Add MQ pallet to fuzzer Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix XMC simulator example Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove runtime-benchmarks from fuzzers Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Revert "Remove runtime-benchmarks from fuzzers" This reverts commit e1f2bb01b6dea2dd465539d3658719895b58b557. * Fix example simulator Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Add V6 migration and remove old ones Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Actually make old migrations reusable Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Dont delete old migrations Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Future proof AggregateMessageOrigin and review fixes There are indications that Loopback and Bridged will be needed soon. Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * More cleanup Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fmt Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * fix benchmarks Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix fuzzer build Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Review Co-authored-by: muharem <ismailov.m.h@gmail.com> Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Remove old migration Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Set MQ service weight to 20% Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix tabs in Markdown Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>	2023-05-19 16:14:13 +00:00
Andrei Sandu	bd209c2d70	inherent disputes: remove per block initializer and disputes timeout event (#6937) ... * Limit disputes weight and remove initializer code Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * const Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * remove timeout test Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * review feedback #1 Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Remove the new weight limiting for disputes Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * cargo lock Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Remove dispute_conclusion_by_time_out_period Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Enable migrations Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Update guide Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Fix comment Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * More guide fixes Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Also migrate pending configs Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix build Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix test Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> --------- Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>	2023-03-24 12:48:56 +01:00
Tsvetomir Dimitrov	15ecfb2c02	Add a paragraph about slashing in runtime disputes section from the guide (#6531) ... * Add a paragraph about slashing in runtime disputes section from the guide * Add definition for inconclusive	2023-01-11 12:37:57 +01:00
Tsvetomir Dimitrov	ed9a1a400e	disputes pallet: Remove spam slots (#6345) ... * disputes pallet: Filter disputes with votes less than supermajority threshold * Remove `max_spam_slots` usages * Remove `SpamSlots` * Remove `SpamSlotChange` * Remove `Error<T>::PotentialSpam` and stale comments * `create_disputes_with_no_spam` -> `create_disputes` * Make tests compile - wip commit * Rework `test_dispute_timeout`. Rename `update_spam_slots` to `filter_dispute_set` * Remove `dispute_statement_becoming_onesided_due_to_spamslots_is_accepted` and `filter_correctly_accounts_spam_slots` -> they bring no value with removed spam slots * Fix `test_provide_multi_dispute_success_and_other` * Remove an old comment * Remove spam slots from tests - clean todo comments * Remove test - `test_decrement_spam` * todo comments * Update TODO comments * Extract `test_unconfirmed_are_ignored` as separate test case * Remove dead code * Fix `test_unconfirmed_are_ignored` * Remove dead code in `filter_dispute_data` * Fix weights (related to commit "Remove `SpamSlots`") * Disputes migration - first try * Remove `dispute_max_spam_slots` + storage migration * Fix `HostConfig` migration tests * Deprecate `SpamSlots` * Code review feedback * add weight for storage version update * fix bound for clear() * Fix weights in disputes migration * Revert "Deprecate `SpamSlots`" This reverts commit 8c4d967c7b061abd76ba8b551223918c0b9e6370. * Make mod migration public * Remove `SpamSlots` from disputes pallet and use `storage_alias` in the migration * Fix call to `clear()` for `SpamSlots` in migration * Update migration and add a `try-runtime` test * Add `pre_upgrade` `try-runtime` test * Fix some test names in `HostConfiguration` migration * Link spamslots migration in all runtimes * Add `test_unconfirmed_disputes_cause_block_import_error` * Update guide - Remove `SpamSlots` related information from roadmap/implementers-guide/src/runtime/disputes.md - Add 'Disputes filtering' to Runtime section of the Implementor's guide * Update runtime/parachains/src/configuration/migration.rs Co-authored-by: Marcin S. <marcin@bytedude.com> * Code review feedback - update logs * Code review feedback: fix weights * Update runtime/parachains/src/disputes.rs Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com> * Additional logs in disputes migration * Fix merge conflicts * Add version checks in try-runtime tests * Fix a compilation warning` Co-authored-by: Marcin S. <marcin@bytedude.com> Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>	2023-01-07 14:56:14 +01:00
ordian	063aab236f	guide: remove refences to outdated secondary checkers (#6309) ... * guide: remove refences to outdated secondary checkers * Update roadmap/implementers-guide/src/glossary.md * guide: remove refences to Fisherman * revert changes to roadmap/parachains.md	2022-11-30 13:28:35 +01:00
Bradley Olson	1d7d87856c	First round of implementers guide fixes (#6146)	2022-10-13 17:43:50 +02:00
Sergei Shulepov	72a92eaf9e	configuration: Rename validation_upgrade_{frequency -> cooldown} (#4635) ... This just renames a member of `HostConfiguration` from validation_upgrade_frequency to -//-_cooldown. As was already pointed out in #4460 the existing name is a misnomer, the member actually represents a minimum time period between upgrades, which is neatly expressed by a word cooldown. I've been planning this rename already for some time and the term is already used in paras module: https://github.com/paritytech/polkadot/blob/1394b70d493a3b4bea7a9ae14af094081bc84456/runtime/parachains/src/paras.rs#L1568-L1574	2021-12-30 20:24:47 +00:00
Sergei Shulepov	1394b70d49	pvf-precheck: update implementers' guide (#4612) ... This commit incorporates the changes made to the runtime in the following PRs: - https://github.com/paritytech/polkadot/pull/4408 - https://github.com/paritytech/polkadot/pull/4457 - https://github.com/paritytech/polkadot/pull/4540 - https://github.com/paritytech/polkadot/pull/4542 - https://github.com/paritytech/polkadot/pull/4581 Note that this PR does not include the description of the PVF pre-checker subsystem. This should be addressed within https://github.com/paritytech/polkadot/issues/4611 Co-authored-by: sandreim <54316454+sandreim@users.noreply.github.com>	2021-12-29 14:32:08 +01:00
Sergei Shulepov	3c2fb21b93	pvf-precheck: Strip PastCodeMeta (#4408) ... This PR is a part of https://github.com/paritytech/polkadot/issues/3211. This PR prepares ground for the following runtime changes required for PVF pre-checking. Specifically, we do several changes here: 1. We remove `validation_code_at` and `validation_code_hash_at`. Those functions are not used. They were added in the early days with intent to use it later but turned out that we do not need them. 2. We replace `validation_code_hash_at` with just `current_code_hash` for the case of inclusion and candidate checking. 3. We also replace `last_code_upgrade` with a direct query into `FutureCodeHash` and `UpgradeRestrictionSignal`. Those in conjunction should replace the logic that was used for allowing/disallowing upgrades. This requires special attention of the reviewers. 4. Then we remove the machinery required to support those queries. Specifically the code related to `UseCodeAt`. We do not need it since we do not answer the historical queries. However, we still leave all the data on-chain. At some point we may clean it up, but that would be needed to be done with a dedicated migration which can be done as follow-up. 5. Some now irrelevant tests were removed and/or adapted.	2021-12-08 11:39:44 +00:00
Bernhard Schuster	10961815cb	add additional assurances to create_inherent (#4349) ... * minor: move checks into separate fn * add additional validity checks * simplify shuffling * Closes potential OOB weight * improve docs * fooo * remove obsolete comment * move filtering into the rollback-transaction Technically this is not necessary but avoids future footguns. * move check up and avoid duplicate checks * refactor: make sure backed candidates are sane, even more * doc wording Co-authored-by: Zeke Mostov <z.mostov@gmail.com> * refactor: avoid const generics for sake of wasm size `true` -> `FullCheck::Skip`, `false` -> `FullCheck::Yes`. * chore: unify `CandidateCheckContext` instance names * refactor: introduce `IndexedRetain` for `Vec<T>` * chore: make tests prefix free * doc: re-introduce removed comment * refactor: remove another const generic to save some wasm size Co-authored-by: Zeke Mostov <z.mostov@gmail.com>	2021-11-24 14:11:06 +00:00
Bernhard Schuster	1aa6a4aba4	move paras inherent filtering to runtime (#4028) ... * move things around, add filter methods * validator keys, modify availability bitfields according to disputes * simplify, keep the filter -> sanitize generic for both usecases * minor * assure tests still work, reduce changeset * integration * start entropy passing * fixins * compile, 1 failing test * filter with coverage * fixins * Update runtime/parachains/src/paras_inherent.rs Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * slip of the pen * improve test cases * misc * fix * fixins * test avoid extra into() calls in assert_noop! * chores * ff * test fixup superfluous into call * chore: pfmt * improve apply_block_weight_limit to try to maximize the number of sufficiently backed blocks and add extra bitfields in a round-robin fashion * new code treats the lack of backed candidates as ok * Use vrf based entropy * fixup vrf random * add warn * slip of the pen * fixup * assure ordering * rethink apply_weights * mock * use a closure as predicate check * extract and use DisputedBitfield * chore: simplify * remove stray dbg * chore: fmt * address feedback * fix test, halfway there * stage1 * dbg stuff * make group selection align * fix session index * fix wrongly returned candidates * cleanup * chore fmt * fix ensure check * make good case test work * more tests for bitfields * create sanitize_backed_candidates * fixup tests * update guide * add check referenced in the guide * improve weights code * fmt * fixins * Update roadmap/implementers-guide/src/runtime/inclusion.md Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com> * compiling + address review * add comments * fix weight calc * address review comments and test failure * fix * fix: condition * Fix random_sel function * Fix overlength block check * Zeke + Ladi commit for disputes filtering + integration test builder + runtime benchmarks + integration tests * Add benchmarks for code upgrades * Code upgrade bench; Feature gate TestWeightInfo * Try and make CI happier * Feature gate enter test to not(benchmarks) * Make sure no unused imports/fn * refactor, re-use, the beginning * Fix issue with frame benchmarking dep compilation * More precise feature gating for some derives * integrate piece-wise * foo * fixins * chore fmt * fixins * rename const generic * Update runtime/parachains/src/paras_inherent.rs Co-authored-by: Zeke Mostov <z.mostov@gmail.com> * Fix compilation * limit to test * remove unused spam slots * spellcheck * remove a tick, fix a typo * Add Code upgrade weights * comment improvements + >= Co-authored-by: Zeke Mostov <z.mostov@gmail.com> * remove another tick * Update runtime/parachains/src/paras_inherent/benchmarking.rs Co-authored-by: Zeke Mostov <z.mostov@gmail.com> * saturating fixins + some spaces * fix * benchmarking - preliminary results * Add training wheels * Refactor some early exit logic for enter * Gracefully handle filtering bitfields & candidates (#4280) This updates the logic for sanitize_bitfields and sanitize_backed_candidates to never error when there is an issue, but instead to simply skip the problematic items. * Refactor inherent data weight limiting logic (#4287) * Apply suggestions from code review * Update runtime/parachains/src/builder.rs Co-authored-by: Zeke Mostov <z.mostov@gmail.com> * Update runtime/parachains/src/builder.rs * Update runtime/parachains/src/paras_inherent.rs * final pass * Run cargo +nightly-2021-10-29 fmt * Update implementors guide with `sanitize_*` & `enter` (#4294) * Make spell check happier * Make wasm runtimes compile with benchmarks enabled (#4303) * comment stuff out, use old toml * Seems to be working? * Remove feature gating from builder * Remove commented out stuff * Remove generic from digest * Update weight files for runtime Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com> Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com> Co-authored-by: Zeke Mostov <z.mostov@gmail.com> Co-authored-by: Bastian Köcher <info@kchr.de>	2021-11-16 18:39:39 +00:00
Bernhard Schuster	30bdc8f5d4	collect included disputes from on-chain (#3924) ... * dummy: impl another runtime API * query the on chain disputes, and inform self * make use of the refactor * minro * SPLIT ME * write dispute values * wip * impl for all runtimes * chore: fmt * [] -> get * fixup mock runtime * fixup * fixup discovery for overseer init * chore: fmt * spellcheck * rename imported_on_chain_disputes -> on_chain_votes * reduction * make it mockable * rename and refactor * don't query on chain info if it's not needed * yikes * fmt * fix test * minimal fix for existing tests * attempt to fetch the session info from the rolling window before falling back * moved * comments * comments * test for backing votes * rename * Update runtime/polkadot/src/lib.rs * chore: spellcheck + dict * chore: fmt * fixup cache size * add warning * logging, rationale, less defense * introduce new unchecked, that still checks in debug builds * fix * draft alt approach * fix unused imports * include the session * Update node/core/dispute-coordinator/src/real/mod.rs Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * provide where possible * expand comment * fixin * fixup * ValidityVote <-> ValidityAttestation <-> CompactStatement has a 1:1 representation * mark TODO * Update primitives/src/v1/mod.rs Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * address review comments * update docs Co-authored-by: Robert Habermeier <rphmeier@gmail.com>	2021-10-06 19:16:23 +00:00
Robert Habermeier	a1bf894d79	Free disputed cores before processing bitfields (#4008) ... * guide: extract free_cores in scheduler * scheduler: extract free cores to a separate function * guide: remove disputed cores from scheduler first * free disputed cores in scheduler before processing bitfields * spellcheck is mostly right but sometimes stupid * add comment and fmt	2021-10-04 15:02:31 +00:00
Robert Habermeier	547d166f20	Fix an off-by-one: revert rather than revert-to (#3991) ... * fix off-by-one in disputes reversion code * bump Rococo spec version	2021-10-01 20:24:35 +00:00
Robert Habermeier	b6a87891ac	Drop availability only for candidates that lose disputes (#3973) ... * guide: updates for disputes changes * paras_inherent: drop availability only for candidates that lose disputes * spellcheck * Update runtime/parachains/src/paras_inherent.rs Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Andronik Ordian <write@reusable.software>	2021-09-29 20:33:16 +00:00
Robert Klotzner	135d9ae931	Put all authorities of a session into SessionInfo. (#3813) ... * Make SessionInfo include all authorities of the current session. * Add missing import. * Only take subset for assignment keys. * Fix typo. * Make spellcheck happy. * Really. * Fix test. * More clear documentation. * Update comments in `SessionInfo`.	2021-09-14 13:58:37 +02:00
Sergei Shulepov	83a35874ee	Do not expire HRMP open channel requests (#3543) ... * Do not expire HRMP open channel requests * Fix the build and update the docs * Implement canceling requests and do not remove them automatically * Fix a borked merge * Fix fmt * Please spellchecker * Apply suggestions from code review Co-authored-by: Amar Singh <asinghchrony@protonmail.com> * Use `mutate_exists` for maintaining request counts * Apply `rustfmt` * Move newly introduced entrypoint to end to preserve ordering Co-authored-by: Amar Singh <asinghchrony@protonmail.com>	2021-09-09 14:06:13 +02:00
Sergei Shulepov	72a8bba401	Introduce upgrade go-ahead and upgrade restriction signals (#3371) ... * Introduce upgrade goahead and upgrade restriction signals * Explicit encoding indicies for exposed enums * typo: abscent -> absent * Prune cooldowns as well * Please hunspell	2021-08-03 19:28:37 +02:00
Robert Habermeier	675f71a882	Dispute vote filtering for block authors (#3498) ... * guide: filter_multi_dispute_data * guide: elaborate * Implementation of dispute data filtering * tests for filtering * don't use std, you fool! * use swap_remove * Update runtime/parachains/src/disputes.rs Co-authored-by: Andronik Ordian <write@reusable.software> * use btreeste * address API nit Co-authored-by: Andronik Ordian <write@reusable.software>	2021-07-21 21:48:07 +02:00
Robert Habermeier	ec98120f0e	Disputes runtime (#2947) ... * disputes module skeleton and storage * implement dispute module initialization logic * implement disputes session change logic * provide dispute skeletons * deduplication & ancient check * fix a couple of warnings * begin provide_dispute_data impl * flesh out statement set import somewhat * move ApprovalVote to shared primitives * add a signing-payload API to explicit dispute statements * implement statement signature checking * some bitflags glue for observing changes in disputes * implement dispute vote import logic * flesh out everything except slashing * guide: tweaks * declare and use punishment trait * punish validators for inconclusive disputes * guide: tiny fix * guide: update docs * add disputes getter fn * guide: small change to spam slots handling * improve spam slots handling and fix some bugs * finish API of disputes runtime * define and deposit `RevertTo` log * begin integrating disputes into para_inherent * use precomputed slash_for/against * return candidate hash from process_bitfields * implement inclusion::collect_disputed * finish integration into rest of runtime * add Disputes to initializer * address suggestions * use pallet macro * fix typo * Update runtime/parachains/src/disputes.rs * add test: fix pruning * document specific behavior * deposit events on dispute changes * add an allow(unused) on fn disputes * add a dummy PunishValidators implementation * add disputes module to Rococo * add disputes module to westend runtime * add disputes module to test runtime * add disputes module to kusama runtime * guide: prepare for runtime API for checking frozenness * remove revert digests in favor of state variable * merge reversions * Update runtime/parachains/src/disputes.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update runtime/parachains/src/disputes.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update runtime/parachains/src/disputes.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * add byzantine_threshold and supermajority_threshold utilities to primitives * use primitive helpers * deposit revert event when freezing chain * deposit revert log when freezing chain * test revert event and log are generated when freezing * add trait to decouple disputes handling from paras inherent handling * runtime: fix compilation and setup dispute handler * disputes: add hook for filtering out dispute statements * disputes: add initializer hooks to DisputesHandler * runtime: remove disputes pallet from all runtimes * tag TODOs * don't import any dispute statements just yet... * address grumbles * fix spellcheck, hopefully * maybe now? * last spellcheck round * fix runtime tests * fix test-runtime Co-authored-by: thiolliere <gui.thiolliere@gmail.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: André Silva <andrerfosilva@gmail.com>	2021-07-19 16:49:19 +00:00
Denis Pisarev	fc253e6e4d	WIP: CI: add spellcheck (#3421) ... * CI: add spellcheck * revert me * CI: explicit command for spellchecker * spellcheck: edit misspells * CI: run spellcheck on diff * spellcheck: edits * spellcheck: edit misspells * spellcheck: add rules * spellcheck: mv configs * spellcheck: more edits * spellcheck: chore * spellcheck: one more thing * spellcheck: and another one * spellcheck: seems like it doesn't get to an end * spellcheck: new words after rebase * spellcheck: new words appearing out of nowhere * chore * review edits * more review edits * more edits * wonky behavior * wonky behavior 2 * wonky behavior 3 * change git behavior * spellcheck: another bunch of new edits * spellcheck: new words are koming out of nowhere * CI: finding the master * CI: fetching master implicitly * CI: undebug * new errors * a bunch of new edits * and some more * Update node/core/approval-voting/src/approval_db/v1/mod.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Update xcm/xcm-executor/src/assets.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Apply suggestions from code review Co-authored-by: Andronik Ordian <write@reusable.software> * Suggestions from the code review * CI: scan only changed files Co-authored-by: Andronik Ordian <write@reusable.software>	2021-07-14 19:22:58 +02:00
André Silva	097cadca19	create newtype for ValidationCodeHash (#3212) ... * create newtype for ValidationCodeHash * pvf: fix tests compilation * primitives: fix test compilation	2021-06-12 16:44:07 +00:00
André Silva	8c3bc2b2bb	replace HistoricalValidationCode usages with ValidationCodeByHash (#3210) ... * replace HistoricalValidationCode usages with ValidationCodeByHash * runtime-api: tabify tests file * update implementers guide	2021-06-11 18:00:57 +00:00
Gavin Wood	53de6871e0	Fix teleport accounting and add some events (#3167) ... * Stuff to help inspect the DMP activity * Fix teleport accounting * Fixes * Fixes * Fixes * Fixes	2021-06-03 15:39:08 +02:00
Denis Pisarev	00812ac9bb	Simnet triggers and images (#2714) ... * CI: new image for simnet * CI: chore * CI: separate image for collator * dockerfile: it's bad upgrading in the images * CI: correct links to Dockerfiles * CI: right dotenv usage * CI: workaround GitLab's bug that failed trigger status is not reported and 'job runs forever' * CI: debug: need instead of rules to avoid the bug * dockerfile: fix storage * CI: workaround sending the status when the trigger is failed * CI: fix needed job * CI: edit comments * CI: trigger simnet with API * CI: triggering script * CI: remove real-overseer * CI: trigger real simnet	2021-04-14 11:31:46 +02:00
Guillaume Thiolliere	beca01f118	Ease parachain candidate code fetching (#2593) ... * code stored in para + modify CandidateDescriptor. * WIP: digest + some more impl * validation_code_hash in payload + check in inclusion * check in client + refator * tests * fix encoding indices * remove old todos * fix test * fix test * add test * fetch validation code inside collation-generation from the relay-chain * HashMismatch -> PoVHashMismatch + miscompilation * refactor, store hash when needed * storage rename: more specific but slightly too verbose * do not hash on candidate validation, fetch hash instead * better test * fix test * guide updates * don't panic in runtime Co-authored-by: Robert Habermeier <rphmeier@gmail.com>	2021-04-01 23:55:39 +02:00
Robert Habermeier	ccfabaa0c6	Node-side subsystems for Disputes (#2566) ... * dispute subsystem files * rename * fix linkcheck * flesh out section README * coordinator schema * DisputeCoordinatorMessage * stub & coordinator protocol * dispute coordinator * add some more message fields * move links to bottom * dispute participation * Cleen It Up ! * runtime: store candidate receipts in dispute state yeah, this is a little heavier. why are you reading this? * Revert "runtime: store candidate receipts in dispute state" This reverts commit 51c10bfd4d866e287e6bd88f317ed57ed987eaee. * add dispute availability statement type and prepare for availability * add 'spam slots' to disputes runtmie * return Spam Slots info from runtime * rework `ImportStatement` to `ImportStatements` * some more methods for dispute coordinator * candidates-included runtime API * algo for providing disputes to runtime. * handle signing with coordinator * dispute coordinator chain ops * remove dead file * remove keystore from dispute participation * adjust ApprovedAncestor to return the necssary data * discuss how approved ancestor and determine undisputed chain are used together * add TODO * initiate disputes from approval voting * route statements from candidate backing and approval voting * fix guide build	2021-04-01 03:00:46 +02:00
Robert Habermeier	8b4f46d2df	Configurable maximum validators (#2586) ... * guide: max_validators * guide: new approach * restrict validators based on configurable max * add some tests * fix wasm build (Vec) * clean up warnings * add logging * set validator indices in tests as well * fix common tests * Update runtime/parachains/src/util.rs Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Andronik Ordian <write@reusable.software>	2021-03-09 18:38:55 +00:00
Robert Habermeier	3300b53306	Approval Checking Improvements Omnibus (#2480) ... * add tracing to approval voting * notify if session info is not working * add dispute period to chain specs * propagate genesis session to parachains runtime * use `on_genesis_session` * protect against zero cores in computation * tweak voting rule to be based off of best and add logs * genesis configuration should use VRF slots only * swallow more keystore errors * add some docs * make validation-worker args non-optional and update clap * better tracing for bitfield signing and provisioner * pass amount of bits in bitfields to inclusion instead of recomputing * debug -> warn for some logs * better tracing for availability recovery * a little av-store tracing * bridge: forward availability recovery messages * add missing try_from impl * some more tracing * improve approval distribution tracing * guide: hold onto pending approval messages until NewBlocks * Hold onto pending approval messages until NewBlocks * guide: adjust comment * process all actions for one wakeup at a time * vec * fix network bridge test * replace randomness-collective-flip with Babe * remove PairNotFound	2021-02-23 14:12:28 -06:00
Shawn Tabrizi	a5defa7c7f	Session Delayed Para Changes / Actions Queue (#2406) ... * initial implementation of lifecycles and upgrades * clean up a bit * fix doc comment * more rigid lifecycle checks * include paras which are transitioning, and lifecycle query * format guide * update api * update guide * explicit outgoing state, fix genesis * handle outgoing with transitioning paras * do not include transitioning paras in identifier * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Apply suggestions from code review * Use matches macro * Correct terms * Apply suggestions from code review * actions queue * Revert "actions queue" This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73. * collapse onboarding state * starting actions queue * consolidate actions queue * schedule para initialize result * more actions queue for upgrade/downgrade * clean up with fully implemented actions queue * fix tests * fix scheduler tests * fix hrmp tests * fix test * doc fixes * fix hrmp test w/ valid para * Update paras.md * fix paras registrar * Update propose_parachain.rs * fix merge * Introduce "shared" module * fix rococo build * fix up and use shared * guide updates * add shared config to common tests * add shared to test-runtime * remove println * fix note Co-authored-by: Gavin Wood <gavin@parity.io>	2021-02-18 21:20:18 -06:00
Robert Habermeier	a8d3aca13d	Disputes High-level rewrite & Disputes runtime (#2424) ... * REVERT: comment out graphviz * rewrite most of protocol-disputes * write about conclusion and chain selection * tie back in overview * basic disputes module * guide: InclusionInherent -> ParaInherent * language * add ParaInherentData type * plug parainherentdata into provisioner * provide_multi_dispute * tweak * inclusion pipeline logic for disputes * be clearer about signature checking * reject backing of disputed blocks * some type rejigging * known-disputes runtime API * wire up inclusion * Revert "REVERT: comment out graphviz" This reverts commit 66203e362f7872cb413d258f74634a0aad70302b. * timeouts * include in initialization order * address grumbles	2021-02-16 18:50:14 +00:00
Shawn Tabrizi	eb0159f288	Initializer + Paras Clean Up Messages When Offboarding (#2413) ... * initial hack in * finish up * use notification to pass outgoing paras * move outgoing paras from notifications * missing comma * update guides * clean up	2021-02-15 17:48:25 +00:00
Shawn Tabrizi	2b533ddc77	Explicit Para Lifecycle w/ Upgrades and Downgrades (#2354) ... * initial implementation of lifecycles and upgrades * clean up a bit * fix doc comment * more rigid lifecycle checks * include paras which are transitioning, and lifecycle query * format guide * update api * update guide * explicit outgoing state, fix genesis * handle outgoing with transitioning paras * do not include transitioning paras in identifier * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Apply suggestions from code review * Use matches macro * Correct terms * Apply suggestions from code review * actions queue * Revert "actions queue" This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73. * collapse onboarding state Co-authored-by: Gavin Wood <gavin@parity.io>	2021-02-04 11:05:44 -06:00
Robert Habermeier	956bc300c1	Approval voting guide changes (#2347) ... * grab guide updates from approval-voting branch * focus on control flow and behavior, not functions * add approval voting preamble info * tabs -> spaces	2021-01-29 17:06:16 +00:00
Robert Habermeier	3465c18b71	scheduler: handle re-scheduling around finalization correctly (#2257) ... * scheduler: handle re-scheduling around finalization correctly * also make sure parathreads get cleaned * run scheduling in finalization * Remove stray println! * Update the schedule call site in inclusion inherent * Clarify subtlety around SessionStartBlock * Remove double semi-colon * reschedule prior to `availability_cores` and in on-initialize * improve docs * fix line * more doc reformat * remove unneeded call * avoid unnecessary scheduling on initialize * split `clear` and `schedule * Update runtime/parachains/src/scheduler.rs Co-authored-by: Sergei Shulepov <sergei@parity.io> Co-authored-by: Sergei Shulepov <sergei@parity.io>	2021-01-13 22:07:09 +00:00
Sergei Shulepov	a864eaa093	Add relay storage root to persisted validation data (#2161) ... * Cont.: Implement the state root obtaining during inclusion During inclusion now we obtain the storage root by passing it through the inclusion_inherent. * Fix tests * Bump rococo spec version * Reorder the parent header into the end of the inclusion inherent. When the parent header is in the beginning, it shifts the other two fields, so that a previous version won't be able to decode that. If we put the parent header in the end, the other two fields will stay at their positions, thus make it possible to decode with the previous version. That allows us to perform upgrade of rococo runtime without needing of simultanuous upgrade of nodes and runtime, or restart of the network. * Squash a stray tab	2021-01-04 11:58:20 -05:00
Robert Habermeier	2557181ecf	Alter behavior of max_validators_per_core (#2143) ... * guide: ensure max-per-core leads to creation of extra, semi-useless cores * alter behavior of max_validators_per_core * guide fixes	2020-12-24 13:55:34 -05:00
Sergei Shulepov	4405f52e2d	Do not use rely on the block initialization when calling runtime APIs (#2123) ... * Don't initialize block when calling runtime APIs * Adapt check_validation_outputs We split the code path for the inclusion and for the commitments checking. * Slap #[skip_initialize_block] on safe runtime APIs That is, those that should not be affected by this attribute * Make `Scheduled` not ephemeral So that it is persisted in the storage and ready to be inspected by the runtime APIs. This is in contrast to what was before, where we would remove the storage entry and then rely on the scheduling performed by `on_initialize` again. * Add a big fat comment * Typos Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Move session change to the end of the current block Previously, it was the beginning of the next block. This allows us to put #[skip_initialize_block] * Update tests * Fix a test in paras registrar Also refactor it a bit so the next time there are more chances this kind of issue is diagnosed quicker. * Add for_runtime_api to inclusion's check_validation_outputs Co-authored-by: Robert Habermeier <rphmeier@gmail.com>	2020-12-21 18:02:22 -05:00
Robert Habermeier	38276b08a4	Add candidate info to OccupiedCore (#2134) ... * guide: add candidate information to OccupiedCore * add descriptor and hash to occupied core type * guide: add candidate hash to inclusion * runtime: return candidate info in core state * bitfield signing: stop querying runtime as much * minimize going to runtime in availability distribution * fix availability distribution tests * guide: remove para ID from Occupied core * get all crates compiling	2020-12-18 11:20:37 +03:00
Robert Habermeier	f4e930529b	Reward validators for participating in parachains (#2089) ... * plumbing for rewarding backers * give validators reward points for participating * fix tests * add bitfield rewarding * add mocks for backing rewards * add testing for backing & availability rewards * implement RewardValidators on top of staking * add to test-runtime and rococo * add to test-runtime & rococo * point to source on rewards values * fix common tests * do not reward availability anymore	2020-12-16 12:28:46 -05:00
Robert Habermeier	15c253117d	Add assignment keys to session keys, no separate approvals key (#2092) ... * guide: merge backing and approval keys * bump substrate master & update primitives * use new SessionInfo struct in session_info * session keys upgrade for Polkadot * kusama & westend runtimes * bump westend, kusama, and polkadot versions * add session key to rococo & test-runtime * update prepare-test-net to latest subkey * update chain specs to support new para_assignment session key * get cargo.lock from master * formatting * update kill_storage based on substrate master * fix test-service * assgn -> asgn * use session info module for assignment session key	2020-12-10 20:30:27 -06:00
Robert Habermeier	0f4395fc44	parachains runtime: configurable maximum validators per core (#2043) ... * parachains runtime: configurable maximum validators per core * update guide and add test	2020-11-30 20:18:02 +00:00
Andronik Ordian	39a12b68f6	past-session validator discovery APIs (#2009) ... * guide: fix formatting for SessionInfo module * primitives: SessionInfo type * punt on approval keys * ah, revert the type alias * session info runtime module skeleton * update the guide * runtime/configuration: sync with the guide * runtime/configuration: setters for newly added fields * runtime/configuration: set codec indexes * runtime/configuration: update test * primitives: fix SessionInfo definition * runtime/session_info: initial impl * runtime/session_info: use initializer for session handling (wip) * runtime/session_info: mock authority discovery trait * guide: update the initializer's order * runtime/session_info: tests skeleton * runtime/session_info: store n_delay_tranches in Configuration * runtime/session_info: punt on approval keys * runtime/session_info: add some basic tests * Update primitives/src/v1.rs * small fixes * remove codec index annotation on structs * fix off-by-one error * validator_discovery: accept a session index * runtime: replace validator_discovery api with session_info * Update runtime/parachains/src/session_info.rs Co-authored-by: Sergei Shulepov <sergei@parity.io> * runtime/session_info: add a comment about missing entries * runtime/session_info: define the keys * util: expose connect_to_past_session_validators * util: allow session_info requests for jobs * runtime-api: add mock test for session_info * collator-protocol: add session_index to test state * util: fix error message for runtime error * fix compilation * fix tests after merge with master Co-authored-by: Sergei Shulepov <sergei@parity.io>	2020-11-26 11:02:50 +00:00