With Coretime enabled we can no longer assume there is a static 1:1
mapping between core index and para id. This mapping should be obtained
from the scheduler/claimqueue on block by block basis.
This PR modifies `para_id()` (from `CoreState`) to return the scheduled
`ParaId` for occupied cores and removes its usages in the code.
Closes https://github.com/paritytech/polkadot-sdk/issues/3948
---------
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
This works only for collators that implement the `collator_fn` allowing
`collation-generation` subsystem to pull collations triggered on new
heads.
Also enables
`request_v2::CollationFetchingResponse::CollationWithParentHeadData` for
test adder/undying collators.
TODO:
- [x] fix tests
- [x] new tests
- [x] PR doc
---------
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Fixing:
```
Verification failed for block 0x07bbf1e04121d70a4bdb21cc055132b53ac2390fa95c4d05497fc91b1e8bf7f5 received from (12D3KooWJzLd8skcAgA24EcJey7aJAhYctfUxWGjSP5Usk9wbpPZ): "Header 0x07bbf1e04121d70a4bdb21cc055132b53ac2390fa95c4d05497fc91b1e8bf7f5 rejected: too far in the future"
```
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
This introduces a check to ensure that the parachain code matches the
validation code stored in the relay chain state. If not, it will print a
warning. This should be mainly useful for parachain builders to make
sure they have setup everything correctly.
Given how the block production is driven for Parachains right now, with
the enabling of async backing we would produce two blocks per slot.
Until we have a proper collator implementation, the "hack" is to prevent
the production of multiple blocks per slot.
Closes: https://github.com/paritytech/polkadot-sdk/issues/3282
It's a follow-up of #2949. It enables the lookahead collator to
dynamically adjust the aura slot size, which may change during the
runtime upgrade. It also addressed a couple of issues with time
constants I missed in the original PR.
Good news: it works. The parachain successfully switches from sync
backing with 12s slots to async backing with 6s slots.
Bad news: during the transitional period of a single block in which the
actual runtime upgrade is performed, it still gets the old slot duration
of 12s (as it gets it from the best block), resulting in a runtime panic
(logs follow). That doesn't affect the following block production of the
parachain. Ideas on how to improve the situation are appreciated.
<details>
```
2024-02-05 12:59:36.373 INFO tokio-runtime-worker sc_basic_authorship::basic_authorship: [Parachain] 🙌 Starting consensus session on top of parent 0x6fd2d8f904f12c22531bfabf77b16dc84a6a29e45d9ae358aa6547fbf3f0438b
2024-02-05 12:59:36.373 ERROR tokio-runtime-worker runtime: [Parachain] panicked at /home/s0me0ne/wrk/parity/polkadot-sdk/cumulus/pallets/aura-ext/src/consensus_hook.rs:69:9:
assertion `left == right` failed: slot number mismatch
left: Slot(142261198)
right: Slot(284522396)
2024-02-05 12:59:36.373 WARN tokio-runtime-worker sp_state_machine::overlayed_changes::changeset: [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back.
2024-02-05 12:59:36.373 WARN tokio-runtime-worker sp_state_machine::overlayed_changes::changeset: [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back.
2024-02-05 12:59:36.373 WARN tokio-runtime-worker basic-authorship: [Parachain] ❗ Inherent extrinsic returned unexpected error: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
WASM backtrace:
error while executing at wasm backtrace:
0: 0x4e4a3b - <unknown>!rust_begin_unwind
1: 0x46cf57 - <unknown>!core::panicking::panic_fmt::h3c280dba88683724
2: 0x46d238 - <unknown>!core::panicking::assert_failed_inner::hebac5970933beb4d
3: 0x3d00fc - <unknown>!core::panicking::assert_failed::h640a47e2fb5dfb4b
4: 0xd0db3 - <unknown>!frame_support::storage::transactional::with_transaction::hcbc31515f81b2ee1
5: 0x34d654 - <unknown>!<cumulus_pallet_parachain_system::pallet::Call<T> as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::{{closure}}::hb7c2c9a11fa88301
6: 0x3547db - <unknown>!environmental::local_key::LocalKey<T>::with::h783f2605ae27d6d3
7: 0x7f454 - <unknown>!<asset_hub_rococo_runtime::RuntimeCall as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::h5e11a01ab97c06c7
8: 0x7f237 - <unknown>!<asset_hub_rococo_runtime::RuntimeCall as sp_runtime::traits::Dispatchable>::dispatch::h7f8ae4a8fede71ca
9: 0x26a0f3 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::apply_extrinsic::h75e524ff34738391
10: 0x282211 - <unknown>!BlockBuilder_apply_extrinsic. Dropping.
2024-02-05 12:59:36.374 ERROR tokio-runtime-worker runtime: [Parachain] panicked at /home/s0me0ne/wrk/parity/polkadot-sdk/substrate/frame/aura/src/lib.rs:416:9:
assertion `left == right` failed: Timestamp slot must match `CurrentSlot`
left: Slot(142261198)
right: Slot(284522396)
2024-02-05 12:59:36.374 WARN tokio-runtime-worker sp_state_machine::overlayed_changes::changeset: [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back.
2024-02-05 12:59:36.374 WARN tokio-runtime-worker sp_state_machine::overlayed_changes::changeset: [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back.
2024-02-05 12:59:36.374 WARN tokio-runtime-worker basic-authorship: [Parachain] ❗ Inherent extrinsic returned unexpected error: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
WASM backtrace:
error while executing at wasm backtrace:
0: 0x4e4a3b - <unknown>!rust_begin_unwind
1: 0x46cf57 - <unknown>!core::panicking::panic_fmt::h3c280dba88683724
2: 0x46d238 - <unknown>!core::panicking::assert_failed_inner::hebac5970933beb4d
3: 0x3d00fc - <unknown>!core::panicking::assert_failed::h640a47e2fb5dfb4b
4: 0x9ece6 - <unknown>!frame_support::storage::transactional::with_transaction::h26f75cb9f9462088
5: 0x356d7e - <unknown>!environmental::local_key::LocalKey<T>::with::hbcf2d4e17b48fdb5
6: 0x7f507 - <unknown>!<asset_hub_rococo_runtime::RuntimeCall as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::h5e11a01ab97c06c7
7: 0x7f237 - <unknown>!<asset_hub_rococo_runtime::RuntimeCall as sp_runtime::traits::Dispatchable>::dispatch::h7f8ae4a8fede71ca
8: 0x26a0f3 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::apply_extrinsic::h75e524ff34738391
9: 0x282211 - <unknown>!BlockBuilder_apply_extrinsic. Dropping.
2024-02-05 12:59:36.374 DEBUG tokio-runtime-worker runtime::xcmp-queue-migration: [Parachain] Lazy migration finished: item gone
2024-02-05 12:59:36.374 ERROR tokio-runtime-worker runtime: [Parachain] panicked at /home/s0me0ne/wrk/parity/polkadot-sdk/cumulus/pallets/parachain-system/src/lib.rs:265:18:
set_validation_data inherent needs to be present in every block!
2024-02-05 12:59:36.374 ERROR tokio-runtime-worker aura::cumulus: [Parachain] err=Error { inner: Proposing
Caused by:
0: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
WASM backtrace:
error while executing at wasm backtrace:
0: 0x4e4a3b - <unknown>!rust_begin_unwind
1: 0x46cf57 - <unknown>!core::panicking::panic_fmt::h3c280dba88683724
2: 0x46da8b - <unknown>!core::option::expect_failed::hdf18d99c3adabca7
3: 0x2134cb - <unknown>!<cumulus_pallet_parachain_system::pallet::Pallet<T> as frame_support::traits::hooks::OnFinalize<<<<T as frame_system::pallet::Config>::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::on_finalize::hf98aac39802896ba
4: 0x26a9d6 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::h32775c0df0749d92
5: 0x26ad9f - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::finalize_block::h15e5a1a6b9ca8032
6: 0x2822bd - <unknown>!BlockBuilder_finalize_block
1: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
WASM backtrace:
error while executing at wasm backtrace:
0: 0x4e4a3b - <unknown>!rust_begin_unwind
1: 0x46cf57 - <unknown>!core::panicking::panic_fmt::h3c280dba88683724
2: 0x46da8b - <unknown>!core::option::expect_failed::hdf18d99c3adabca7
3: 0x2134cb - <unknown>!<cumulus_pallet_parachain_system::pallet::Pallet<T> as frame_support::traits::hooks::OnFinalize<<<<T as frame_system::pallet::Config>::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::on_finalize::hf98aac39802896ba
4: 0x26a9d6 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::h32775c0df0749d92
5: 0x26ad9f - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::finalize_block::h15e5a1a6b9ca8032
6: 0x2822bd - <unknown>!BlockBuilder_finalize_block }
```
</details>
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
This splits `cumulus-primitives-parachain-inherent` into two crates, the
previous `cumulus-primitives-parachain-inherent` and a new
`cumulus-client-parachain-inherent`. The idea behind this is to move the
`create_at` logic into the client crate. This removes quite a lot of
unrelated dependencies from the runtime std build and thus, makes the
compilation faster. On my Laptop the compilation is goes down by one
minute for `asset-hub-rococo-runtime`. I also assume that the full build
of the entire workspace probably can be speed-up a little bit, because
more stuff can be compiled in parallel.
---------
Co-authored-by: command-bot <>
This opens up the proposer to only optionally create blocks. Nodes may
only make blocks when there are transactions or the chain is scheduled.
---------
Co-authored-by: command-bot <>
* Update substrate & polkadot
* min changes to make async backing compile
* (async backing) parachain-system: track limitations for unincluded blocks (#2438)
* unincluded segment draft
* read para head from storage proof
* read_para_head -> read_included_para_head
* Provide pub interface
* add errors
* fix unincluded segment update
* BlockTracker -> Ancestor
* add a dmp limit
* Read para head depending on the storage switch
* doc comments
* storage items docs
* add a sanity check on block initialize
* Check watermark
* append to the segment on block finalize
* Move segment update into set_validation_data
* Resolve para head todo
* option watermark
* fix comment
* Drop dmq check
* fix weight
* doc-comments on inherent invariant
* Remove TODO
* add todo
* primitives tests
* pallet tests
* doc comments
* refactor unincluded segment length into a ConsensusHook (#2501)
* refactor unincluded segment length into a ConsensusHook
* add docs
* refactor bandwidth_out calculation
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
* test for limits from impl
* fmt
* make tests compile
* update comment
* uncomment test
* fix collator test by adding parent to state proof
* patch HRMP watermark rules for unincluded segment
* get consensus-common tests to pass, using unincluded segment
* fix unincluded segment tests
* get all tests passing
* fmt
* rustdoc CI
* aura-ext: limit the number of authored blocks per slot (#2551)
* aura_ext consensus hook
* reverse dependency
* include weight into hook
* fix tests
* remove stray println
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
* fix test warning
* fix doc link
---------
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
* parachain-system: ignore go ahead signal once upgrade is processed (#2594)
* handle goahead signal for unincluded segment
* doc comment
* add test
* parachain-system: drop processed messages from inherent data (#2590)
* implement `drop_processed_messages`
* drop messages based on relay parent number
* adjust tests
* drop changes to mqc
* fix comment
* drop test
* drop more dead code
* clippy
* aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658)
* aura-ext: check slot in consensus hook
* convert relay chain slot
* Make relay chain slot duration generic
* use fixed velocity hook for pallets with aura
* purge timestamp inherent
* fix warning
* adjust runtime tests
* fix slots in tests
* Make `xcm-emulator` test pass for new consensus hook (#2722)
* add pallets on_initialize
* tests pass
* add AuraExt on_init
* ".git/.scripts/commands/fmt/fmt.sh"
---------
Co-authored-by: command-bot <>
---------
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
* update polkadot git refs
* CollationGenerationConfig closure is now optional (#2772)
* CollationGenerationConfig closure is now optional
* fix test
* propagate network-protocol-staging feature (#2899)
* Feature Flagging Consensus Hook Type Parameter (#2911)
* First pass
* fmt
* Added as default feature in tomls
* Changed to direct dependency feature
* Dealing with clippy error
* Update pallets/parachain-system/src/lib.rs
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
---------
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
* fmt
* bump deps and remove warning
* parachain-system: update RelevantMessagingState according to the unincluded segment (#2948)
* mostly address 2471 with a bug introduced
* adjust relevant messaging state after computing total
* fmt
* max -> min
* fix test implementation of xcmp source
* add test
* fix test message sending logic
* fix + test
* add more to unincluded segment test
* fmt
---------
Co-authored-by: Chris Sosnin <chris125_@live.com>
* Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864)
* add a comment
* refactor client/service utilities
* deprecate start_collator
* update parachain-template
* update test-service in the same way
* update polkadot-parachain crate
* fmt
* wire up new SubmitCollation message
* some runtime utilities for implementing unincluded segment runtime APIs
* allow parachains to configure their level of sybil-resistance when starting the network
* make aura-ext compile
* update to specify sybil resistance levels
* fmt
* specify relay chain slot duration in milliseconds
* update Aura to explicitly produce Send futures
also, make relay_chain_slot_duration a Duration
* add authoring duration to basic collator and document params
* integrate new basic collator into parachain-template
* remove assert_send used for testing
* basic-aura: only author when parent included
* update polkadot-parachain-bin
* fmt
* some fixes
* fixes
* add a RelayNumberMonotonicallyIncreases
* add a utility function for initializing subsystems
* some logging for timestamp adjustment
* fmt
* some fixes for lookahead collator
* add a log
* update `find_potential_parents` to account for sessions
* bound the loop
* restore & deprecate old start_collator and start_full_node functions.
* remove unnecessary await calls
* fix warning
* clippy
* more clippy
* remove unneeded logic
* ci
* update comment
Co-authored-by: Marcin S. <marcin@bytedude.com>
* (async backing) restore `CheckInherents` for backwards-compatibility (#2977)
* bring back timestamp
* Restore CheckInherents
* revert to empty CheckInherents
* make CheckInherents optional
* attempt
* properly end system blocks
* add some more comments
* ignore failing system parachain tests
* update refs after main feature branch merge
* comment out the offending tests because CI runs ignored tests
* fix warnings
* fmt
* revert to polkadot master
* cargo update -p polkadot-primitives -p sp-io
---------
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: Andronik <write@reusable.software>
* rough draft of potential parent search
* get things compiling
* fmt
* add new function to all RelayChainInterface implementations
* fix compilation
* set slot and timestamp based on relay parent, prepare for find-parent
* skeleton of new aura logic
* fmt
* introduce a collator module in the Aura crate
* extract different implementations into own modules
* make interface more convenient
* docs and todos for lookahead
* refactor basic collator to use new collator utility
* some more refactoring
* finish most of the control flow for new aura
* introduce backend as parameter
* fix compilation
* fix a couple more TODOs
* add an `announce_block` function to collator service
* announce with barrier
* rename block announcement validator to be more specific
* fmt
* clean up unused import errors
* update references to BlockAnnounceValidator
* rename unstable_reimpl
* add AuraUnincludedSegmentApi
* finish rename
* integrate AuraUnincludedSegmentApi
* add a new block announcement validator for backwards compatibility
* add some naive equivocation defenses
* rustfmt
* clean up remaining TODO [now]s
* fmt
* try to fix inprocess-interface
* actually fix compilation
* ignored -> rejected rephrase
* fix test compilation
* fmt
* clippy
* implement a proposer utility for consensus
* tidy up deps of new proposer crate
* implement a collator-service crate
* rewrite cumulus-collator to use new service struct
* implement a module for relay-chain-driven collators
* adapt start_collator to use the new relay_chain_driven module
* move collator-service to a public submodule
* create an interface trait for the proposer
* begin aura reimplementation
* address review comments
* update substrrate git ref
* update polkadot-primitives refs
* rough draft of aura collation using standalone fns
* add a ServiceInterface
* port aura reimpl to use new service trait
* add an import queue utility crate
* remove import queue crate in favor of module in common
* implement new verification queue for aura
* implement remaining behaviors
* split 'collate' into smaller functions that could be pub
* add telemetry
* fix doc job?
* Specify async-trait patch version
Co-authored-by: Bastian Köcher <git@kchr.de>
* remove 'fn@' in doc string.
Co-authored-by: Bastian Köcher <git@kchr.de>
* update variable names to be more readable
* refactor proposer errors to anyhow/thiserror
* remove manual span instrumentation
Co-authored-by: Bastian Köcher <git@kchr.de>
* make slot_claim private
* fix unused import
* fmt
* fmt
* make clippy happy
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
* Align to Substrate keystore overhaul
* 'sync_keystore' renamed to 'keystore'
* Further fixes after removal of async keystore trait
* update lockfile for {"polkadot", "substrate"}
---------
Co-authored-by: parity-processbot <>
* Use primitives reexported from `polkadot_primitives` crate root
* restart CI
* Fixes after merge
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: parity-processbot <>
Prevents the StateDbError::TooManySiblingBlocks error from being triggered by eagerly removing
stale blocks from the backend on block import and before the error condition is met.
Introduces a just in time block recovery mechanism for blocks that were wrongly removed
via an explicit pov-recovery method
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Companion for aura CompatibilityMode
As no parachain was launched with the broken authority set change handling we don't need to expose
the compatibility mode.
* FMT
* update lockfile for {"substrate", "polkadot"}
Co-authored-by: parity-processbot <>
* Update polkadot
* Migrate all uses of MQC heads to merkle proofs
* Mass rename `relay_parent_storage_root`
* Restore parachain-system tests
* Update polkadot and libp2p swarm for testing
* Collapse match into an if let
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Start with something
* Update Substrate & Polkadot
* Start to make it compile
* Make it compile
* Begin with something
* Yep
* I'm a hacker
* Bring back the builder
* Make it work in some way
* Compile
* Parachains use their own "slot"
* Adds cumulus-pallet-aura
* Wrap AuRa import queue to disable equivocation checking by default
* Pass slot duration
* Check the seal when validating a block
* Adds missing file
* Try to make the seal working
* Fix it
* Some fixes
* Bring in the latest features to cleanup the code
* Update and make it compile
* Improve the import
* Start fixing
* More work
* Fix fix fix
* Make everything compile
* Small cleanups
* Rename and more docs
* Docs
* Fixes fixes fixes
* Update rococo-parachains/src/chain_spec.rs
* Update client/consensus/aura/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/aura/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update primitives/parachain-inherent/Cargo.toml
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update primitives/parachain-inherent/Cargo.toml
* Update primitives/parachain-inherent/Cargo.toml
* Update primitives/parachain-inherent/Cargo.toml
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Bastian Köcher