* don't modify inherent data on heavy block
* write up current thinking on block weight detection
* extract inherent inclusion check into its own function
* put heavy block check into runtime
* the `inclusion` inherent call is Operational, not Mandatory
This resolves a lot of the trickiness about this issue, because
we no longer need to override or supplant any existing proposer
logic; the existing logic should exhibit these behaviors:
- the `inclusion` inherent is prioritized over standard transactions
- but if it's too heavy, i.e. in case of runtime upgrade, it'll be
dropped in favor of that.
It is my belief that allowing the proposer to just not include
this data won't have any adverse effects: it's equivalent to replacing
them with empty versions of themselves, which the `ProvideInherent`
impl already does.
* Revert "the `inclusion` inherent call is Operational, not Mandatory"
This reverts commit e58858d109b18b84e7af3ac47981c6900b2d9a3e.
* Revert "write up current thinking on block weight detection"
This reverts commit fd587b80c46761b2a2b62448193348237863f99f.
* Revert "don't modify inherent data on heavy block"
This reverts commit 38299d3c23e9efb5a354d8cfa658e62a5c8c7ddf.
* add backed candidate block weight assumption to configuration
* Limit backed candidates according to a candidate weight heuristic.
This approach replaces making the inclusion inherent non-mandatory.
It's still not ideal in that we have to configure a heuristic for
how much each backed candidate 'weighs', instead of directly
measuring it somehow.
This approach also never truncates the signed bitfields. The
rationale for that depends on some assumptions:
- processing the signed bitfields is cheap compared to the
backed candidates
- it is beneficial to the progress of the relay chain
to update the signed bitfields even if not all backed candidates
are updated
* simplify limit_backed_candidates and weight assumption
* don't trust the provisioner to fairly distribute candidates
* use saturating subtraction
* empty commit to restart ci
* use new mechanism for getting max block weight
* apply weight refunds to the inclusion inherent
This makes some assumptions about fundamental weights, which are
encapsulated as constants. From there, it lets Substrate know
what the actual computed weight of the inherent is.
* use a correct fixed weight for the inclusion inherent
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* use dynamic inclusion weight so we reduce calculated weight when excluding candidates
* don't double-count this intrinsic's weight in the block weight
* add unit tests of fn limit_backed_candidates
* add tests that the inclusion inherent's weight correctly updates
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Cont.: Implement the state root obtaining during inclusion
During inclusion now we obtain the storage root by passing it through
the inclusion_inherent.
* Fix tests
* Bump rococo spec version
* Reorder the parent header into the end
of the inclusion inherent.
When the parent header is in the beginning, it shifts the other two
fields, so that a previous version won't be able to decode that. If
we put the parent header in the end, the other two fields will stay
at their positions, thus make it possible to decode with the previous
version.
That allows us to perform upgrade of rococo runtime without needing of
simultanuous upgrade of nodes and runtime, or restart of the network.
* Squash a stray tab
* Guide: Split router module in guide.
Now we have: DMP, UMP and Router module.
* Add a glossary entry for what used to be called Router
* Extract DMP
* Extract UMP
* Extract HRMP
* Switch over to new modules
* Router: goodbye sweet prince
* Link to messaging overview for details.
* Update missed rococo and test runtimes.
* Commit destroyed by rebase changes
* Don't deprecate Router but rather make it a meta-project
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Fix typos suggestion
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Fix repetition in the impl guide
* Clarify that processed_downward_messages has the u32 type
* Remove the router subdir.
* Deabbreviate DMP,UMP,HRMP
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
If a core is freed, we need to make sure it is inserted as free into
availability cores, because we can not be sure that the core is occupied
directly again.
* UMP: Update the impl guide
* UMP: Incorporate XCM related changes into the guide
* UMP: Data structures and configuration
* UMP: Initial plumbing
* UMP: Data layout
* UMP: Acceptance criteria & enactment
* UMP: Fix dispatcher bug and add the test for it
* UMP: Constrain the maximum size of an UMP message
This commit addresses the UMP part of https://github.com/paritytech/polkadot/issues/1869
* Fix failing test due to misconfiguration
* Make the type of RelayDispatchQueueSize be more apparent in the guide
* Revert renaming `max_upward_queue_capacity` to `max_upward_queue_count`
* convert spaces to tabs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Update runtime/parachains/src/router/ump.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* create a v1 primitives module
* Improve guide on availability types
* punctuate
* new parachains runtime uses new primitives
* tests of new runtime now use new primitives
* add ErasureChunk to guide
* export erasure chunk from v1 primitives
* subsystem crate uses v1 primitives
* node-primitives uses new v1 primitives
* port overseer to new primitives
* new-proposer uses v1 primitives (no ParachainHost anymore)
* fix no-std compilation for primitives
* service-new uses v1 primitives
* network-bridge uses new primitives
* statement distribution uses v1 primitives
* PoV distribution uses v1 primitives; add PoV::hash fn
* move parachain to v0
* remove inclusion_inherent module and place into v1
* remove everything from primitives crate root
* remove some unused old types from v0 primitives
* point everything else at primitives::v0
* squanch some warns up
* add RuntimeDebug import to no-std as well
* port over statement-table and validation
* fix final errors in validation and node-primitives
* add dummy Ord impl to committed candidate receipt
* guide: update CandidateValidationMessage
* add primitive for validationoutputs
* expand CandidateValidationMessage further
* bikeshed
* add some impls to omitted-validation-data and available-data
* expand CandidateValidationMessage
* make erasure-coding generic over v1/v0
* update usages of erasure-coding
* implement commitments.hash()
* use Arc<Pov> for CandidateValidation
* improve new erasure-coding method names
* fix up candidate backing
* update docs a bit
* fix most tests and add short-circuiting to make_pov_available
* fix remainder of candidate backing tests
* squanching warns
* squanch it up
* some fallout
* overseer fallout
* free from polkadot-test-service hell
* impl ProvideInherent for InclusionInherent
* reduce import churn; correct expect message
* move inclusion inherent identifier into primitives
It's not clear precisely why this is desired, but it's a pattern
I've seen in several places, so I'm going this to be on the
safe side. Worst case, we can revert this commit pretty easily.
* bump kusama spec_version to placate CI
* add license header
* empty commit; maybe github will notice the one with changes
* add sanity check to only include valid inherents
* add availability bitfield types to primitives
* begin inclusion module
* use GitHub issue link for limitation
* fix some compiler errors
* integrate validators into initializer
* add generic signing context
* make signing-context more generic
* fix issues with inclusion module
* add TODO
* guide: add validators and session index to inclusion
* guide: add session index to change notification
* implement session change logic
* add BackedCandidate type
* guide: refine inclusion pipeline
* guide: rename group_on to group_validators
* guide: add check about collator for parathread
* guide: add last_code_upgrade to paras and use in inclusion
* implement Paras::last_code_upgrade
* implement most checks in process_candidates
* make candidate receipt structs more generic
* make BackedCandidate struct more generic
* use hash param, not block number
* check that candidate is in context of the parent block
* include inclusion module in initializer
* implement enact-candidate
* check that only occupied cores have bits set
* finish implementing bitfield processing
* restructure consistency checks on candidates
* make some more primitives generic
* signature checking logic for backed candidates
* finish implementing process_candidates
* implement collect_pending
* add some trait implementations to primitives
* implement InclusionInherent and squash warnings
* test bitfield signing checks
* rename parachain head to para_head
* fix note_new_head bug in paras
* test bitfield enactment in inclusion
* helpers for candidate checks
* add test for most candidate checks
* add test for backing setting storage
* test session change logic
* remove extraneous type parameter
* remove some allow(unused)s
* extract threshold computation to const fn
* remove some more allow(unused)s
* improve doc
* add debug assertion
* fix primitive test compilation
* tag unanimous variant as unused
Peter Goodspeed-Niklaus