Cumulus test-parachain node and test runtime were still using relay
chain consensus and 12s blocktimes. With async backing around the corner
on the major chains we should switch our tests too.
Also needed to nicely test the changes coming to collators in #3168.
### Changes Overview
- Followed the [migration
guide](https://wiki.polkadot.network/docs/maintain-guides-async-backing)
for async backing for the cumulus-test-runtime
- Adjusted the cumulus-test-service to use the correct import-queue,
lookahead collator etc.
- The block validation function now uses the Aura Ext Executor so that
the seal of the block is validated
- Previous point requires that we seal block before calling into
`validate_block`, I introduced a helper function for that
- Test client adjusted to provide a slot to the relay chain proof and
the aura pre-digest
* Update substrate & polkadot
* min changes to make async backing compile
* (async backing) parachain-system: track limitations for unincluded blocks (#2438)
* unincluded segment draft
* read para head from storage proof
* read_para_head -> read_included_para_head
* Provide pub interface
* add errors
* fix unincluded segment update
* BlockTracker -> Ancestor
* add a dmp limit
* Read para head depending on the storage switch
* doc comments
* storage items docs
* add a sanity check on block initialize
* Check watermark
* append to the segment on block finalize
* Move segment update into set_validation_data
* Resolve para head todo
* option watermark
* fix comment
* Drop dmq check
* fix weight
* doc-comments on inherent invariant
* Remove TODO
* add todo
* primitives tests
* pallet tests
* doc comments
* refactor unincluded segment length into a ConsensusHook (#2501)
* refactor unincluded segment length into a ConsensusHook
* add docs
* refactor bandwidth_out calculation
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
* test for limits from impl
* fmt
* make tests compile
* update comment
* uncomment test
* fix collator test by adding parent to state proof
* patch HRMP watermark rules for unincluded segment
* get consensus-common tests to pass, using unincluded segment
* fix unincluded segment tests
* get all tests passing
* fmt
* rustdoc CI
* aura-ext: limit the number of authored blocks per slot (#2551)
* aura_ext consensus hook
* reverse dependency
* include weight into hook
* fix tests
* remove stray println
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
* fix test warning
* fix doc link
---------
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
* parachain-system: ignore go ahead signal once upgrade is processed (#2594)
* handle goahead signal for unincluded segment
* doc comment
* add test
* parachain-system: drop processed messages from inherent data (#2590)
* implement `drop_processed_messages`
* drop messages based on relay parent number
* adjust tests
* drop changes to mqc
* fix comment
* drop test
* drop more dead code
* clippy
* aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658)
* aura-ext: check slot in consensus hook
* convert relay chain slot
* Make relay chain slot duration generic
* use fixed velocity hook for pallets with aura
* purge timestamp inherent
* fix warning
* adjust runtime tests
* fix slots in tests
* Make `xcm-emulator` test pass for new consensus hook (#2722)
* add pallets on_initialize
* tests pass
* add AuraExt on_init
* ".git/.scripts/commands/fmt/fmt.sh"
---------
Co-authored-by: command-bot <>
---------
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
* update polkadot git refs
* CollationGenerationConfig closure is now optional (#2772)
* CollationGenerationConfig closure is now optional
* fix test
* propagate network-protocol-staging feature (#2899)
* Feature Flagging Consensus Hook Type Parameter (#2911)
* First pass
* fmt
* Added as default feature in tomls
* Changed to direct dependency feature
* Dealing with clippy error
* Update pallets/parachain-system/src/lib.rs
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
---------
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
* fmt
* bump deps and remove warning
* parachain-system: update RelevantMessagingState according to the unincluded segment (#2948)
* mostly address 2471 with a bug introduced
* adjust relevant messaging state after computing total
* fmt
* max -> min
* fix test implementation of xcmp source
* add test
* fix test message sending logic
* fix + test
* add more to unincluded segment test
* fmt
---------
Co-authored-by: Chris Sosnin <chris125_@live.com>
* Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864)
* add a comment
* refactor client/service utilities
* deprecate start_collator
* update parachain-template
* update test-service in the same way
* update polkadot-parachain crate
* fmt
* wire up new SubmitCollation message
* some runtime utilities for implementing unincluded segment runtime APIs
* allow parachains to configure their level of sybil-resistance when starting the network
* make aura-ext compile
* update to specify sybil resistance levels
* fmt
* specify relay chain slot duration in milliseconds
* update Aura to explicitly produce Send futures
also, make relay_chain_slot_duration a Duration
* add authoring duration to basic collator and document params
* integrate new basic collator into parachain-template
* remove assert_send used for testing
* basic-aura: only author when parent included
* update polkadot-parachain-bin
* fmt
* some fixes
* fixes
* add a RelayNumberMonotonicallyIncreases
* add a utility function for initializing subsystems
* some logging for timestamp adjustment
* fmt
* some fixes for lookahead collator
* add a log
* update `find_potential_parents` to account for sessions
* bound the loop
* restore & deprecate old start_collator and start_full_node functions.
* remove unnecessary await calls
* fix warning
* clippy
* more clippy
* remove unneeded logic
* ci
* update comment
Co-authored-by: Marcin S. <marcin@bytedude.com>
* (async backing) restore `CheckInherents` for backwards-compatibility (#2977)
* bring back timestamp
* Restore CheckInherents
* revert to empty CheckInherents
* make CheckInherents optional
* attempt
* properly end system blocks
* add some more comments
* ignore failing system parachain tests
* update refs after main feature branch merge
* comment out the offending tests because CI runs ignored tests
* fix warnings
* fmt
* revert to polkadot master
* cargo update -p polkadot-primitives -p sp-io
---------
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: Andronik <write@reusable.software>
* rough draft of potential parent search
* get things compiling
* fmt
* add new function to all RelayChainInterface implementations
* fix compilation
* set slot and timestamp based on relay parent, prepare for find-parent
* skeleton of new aura logic
* fmt
* introduce a collator module in the Aura crate
* extract different implementations into own modules
* make interface more convenient
* docs and todos for lookahead
* refactor basic collator to use new collator utility
* some more refactoring
* finish most of the control flow for new aura
* introduce backend as parameter
* fix compilation
* fix a couple more TODOs
* add an `announce_block` function to collator service
* announce with barrier
* rename block announcement validator to be more specific
* fmt
* clean up unused import errors
* update references to BlockAnnounceValidator
* rename unstable_reimpl
* add AuraUnincludedSegmentApi
* finish rename
* integrate AuraUnincludedSegmentApi
* add a new block announcement validator for backwards compatibility
* add some naive equivocation defenses
* rustfmt
* clean up remaining TODO [now]s
* fmt
* try to fix inprocess-interface
* actually fix compilation
* ignored -> rejected rephrase
* fix test compilation
* fmt
* clippy
* implement a proposer utility for consensus
* tidy up deps of new proposer crate
* implement a collator-service crate
* rewrite cumulus-collator to use new service struct
* implement a module for relay-chain-driven collators
* adapt start_collator to use the new relay_chain_driven module
* move collator-service to a public submodule
* create an interface trait for the proposer
* begin aura reimplementation
* address review comments
* update substrrate git ref
* update polkadot-primitives refs
* rough draft of aura collation using standalone fns
* add a ServiceInterface
* port aura reimpl to use new service trait
* add an import queue utility crate
* remove import queue crate in favor of module in common
* implement new verification queue for aura
* implement remaining behaviors
* split 'collate' into smaller functions that could be pub
* add telemetry
* fix doc job?
* Specify async-trait patch version
Co-authored-by: Bastian Köcher <git@kchr.de>
* remove 'fn@' in doc string.
Co-authored-by: Bastian Köcher <git@kchr.de>
* update variable names to be more readable
* refactor proposer errors to anyhow/thiserror
* remove manual span instrumentation
Co-authored-by: Bastian Köcher <git@kchr.de>
* make slot_claim private
* fix unused import
* fmt
* fmt
* make clippy happy
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
* BlockId removal: refactor of runtime API
It changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`
* fmt
* cargo update -p polkadot-primitives
* Revert "cargo update -p polkadot-primitives"
This reverts commit 58506ecd31e1e5e42b78c2db3982acbc4d8edc70.
* update lockfile for {"substrate", "polkadot"}
---------
Co-authored-by: parity-processbot <>
* Use primitives reexported from `polkadot_primitives` crate root
* restart CI
* Fixes after merge
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: parity-processbot <>
* BlockId removal: refactor: BlockBackend::block|block_status
It changes the arguments of:
- `BlockBackend::block`
- `BlockBackend::block_status`
method from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: parity-processbot <>
* BlockId removal: refactor: HeaderBackend::header
It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`
methods from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: parity-processbot <>
* Return compressed PoV from collation
This enables us to also print the size of the compressed PoV.
* FMT
* Update Polkadot & Substrate
* Fix compilation
* Fixes
* FMT
* state-update4 branch
* new ref
* Update to latest.
* update deps
* switch to host state version
* update
* fmt
* up
* remove trie patch
* remove patch
* fmt
* update
* set state_versions in runtimes
* state version from storage
* state version from storage
* seedling compat
* restore lock
* update lockfile for substrate
* update lockfile for polkadot
Co-authored-by: parity-processbot <>
* Adds support for checking the timestamp inherent while validating a block
This adds support for checking the timestamp inherent while validating a
block. This will use the relay chain slot number * relay chain slot
duration to calculate a timestamp. This timestamp is used to check the
timestamp in the timestamp inherent.
* Update polkadot-parachains/rococo-runtime/src/lib.rs
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Update polkadot-parachains/statemine-runtime/src/lib.rs
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Update primitives/timestamp/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix warnings
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
This is basically a hack that prevents the inclusion of `:code` in the
storage proof. A proper fix requires some changes to the trie crate to
not include nodes that aren't read.
* Introduce `CollectCollationInfo` runtime api
Instead of using well known keys to communicate information about a
collation between the runtime and the collator, we now use a runtime api
for this.
* Fixes bug
* Apply suggestions from code review
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Doc update
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Remove pov hash comparison when waiting on a block to announce
Internally we get notified on the channel for our candidate anyway.
Besides that polkadot will compress the pov which leads to a different
pov hash and thus, would lead to a failing check on the `pov_hash`.
* Fix
* Fix tests
* Add a command to purge the relay chain only
* WIP
* Update rococo-parachains/src/cli.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Move cli stuff to its own crate
* Copyright dates
* Test not working for some reason...
* WIP
* Revert "WIP"
This reverts commit f97cd63742c7df822e4a6e52a29db5e0f56b7bfa.
* Fix test to use provided relay chain
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Add hint about which database could not be purged
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Move consensus to consensus-common crate
* Move the parachain consensus out of the collator
* Add first relay chain consensus stuff
* Remove some warnings
* Fix more stuff
* Fix collator test
* Change `ParachainConsensus` to take a mutable self
* Make everything compile
* Feedback
This renames and moves the `SystemInherentData` into its own crate.
The struct is now called `ParachainInherentData`. Besides moving the
struct, this also moves the code for creating this struct into this crate.
* Move pallets to pallets folder and rename them
* Move genesis file to service
* Rename primitives to primitives-core
* Delete cumulus-runtime
* Move stuff to client folder and rename
Sebastian Kunert