mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-12 10:01:17 +00:00
Multi-Block-Migrations, poll hook and new System callbacks (#1781)
This MR is the merge of https://github.com/paritytech/substrate/pull/14414 and https://github.com/paritytech/substrate/pull/14275. It implements [RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes https://github.com/paritytech/polkadot-sdk/issues/198. ----- This Merge request introduces three major topicals: 1. Multi-Block-Migrations 1. New pallet `poll` hook for periodic service work 1. Replacement hooks for `on_initialize` and `on_finalize` in cases where `poll` cannot be used and some more general changes to FRAME. The changes for each topical span over multiple crates. They are listed in topical order below. # 1.) Multi-Block-Migrations Multi-Block-Migrations are facilitated by creating `pallet_migrations` and configuring `System::Config::MultiBlockMigrator` to point to it. Executive picks this up and triggers one step of the migrations pallet per block. The chain is in lockdown mode for as long as an MBM is ongoing. Executive does this by polling `MultiBlockMigrator::ongoing` and not allowing any transaction in a block, if true. A MBM is defined through trait `SteppedMigration`. A condensed version looks like this: ```rust /// A migration that can proceed in multiple steps. pub trait SteppedMigration { type Cursor: FullCodec + MaxEncodedLen; type Identifier: FullCodec + MaxEncodedLen; fn id() -> Self::Identifier; fn max_steps() -> Option<u32>; fn step( cursor: Option<Self::Cursor>, meter: &mut WeightMeter, ) -> Result<Option<Self::Cursor>, SteppedMigrationError>; } ``` `pallet_migrations` can be configured with an aggregated tuple of these migrations. It then starts to migrate them one-by-one on the next runtime upgrade. Two things are important here: - 1. Doing another runtime upgrade while MBMs are ongoing is not a good idea and can lead to messed up state. - 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`, otherwise it is not used.** The pallet supports an `UpgradeStatusHandler` that can be used to notify external logic of upgrade start/finish (for example to pause XCM dispatch). Error recovery is very limited in the case that a migration errors or times out (exceeds its `max_steps`). Currently the runtime dev can decide in `FailedMigrationHandler::failed` how to handle this. One follow-up would be to pair this with the `SafeMode` pallet and enact safe mode when an upgrade fails, to allow governance to rescue the chain. This is currently not possible, since governance is not `Mandatory`. ## Runtime API - `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to inform the Block Author whether they can push transactions. ### Integration Add it to your runtime implementation of `Core` and `BlockBuilder`: ```patch diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs @@ impl_runtime_apis! { impl sp_block_builder::Core<Block> for Runtime { - fn initialize_block(header: &<Block as BlockT>::Header) { + fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode { Executive::initialize_block(header) } ... } ``` # 2.) `poll` hook A new pallet hook is introduced: `poll`. `Poll` is intended to replace mostly all usage of `on_initialize`. The reason for this is that any code that can be called from `on_initialize` cannot be migrated through an MBM. Currently there is no way to statically check this; the implication is to use `on_initialize` as rarely as possible. Failing to do so can result in broken storage invariants. The implementation of the poll hook depends on the `Runtime API` changes that are explained above. # 3.) Hard-Deadline callbacks Three new callbacks are introduced and configured on `System::Config`: `PreInherents`, `PostInherents` and `PostTransactions`. These hooks are meant as replacement for `on_initialize` and `on_finalize` in cases where the code that runs cannot be moved to `poll`. The reason for this is to make the usage of HD-code (hard deadline) more explicit - again to prevent broken invariants by MBMs. # 4.) FRAME (general changes) ## `frame_system` pallet A new memorize storage item `InherentsApplied` is added. It is used by executive to track whether inherents have already been applied. Executive and can then execute the MBMs directly between inherents and transactions. The `Config` gets five new items: - `SingleBlockMigrations` this is the new way of configuring migrations that run in a single block. Previously they were defined as last generic argument of `Executive`. This shift is brings all central configuration about migrations closer into view of the developer (migrations that are configured in `Executive` will still work for now but is deprecated). - `MultiBlockMigrator` this can be configured to an engine that drives MBMs. One example would be the `pallet_migrations`. Note that this is only the engine; the exact MBMs are injected into the engine. - `PreInherents` a callback that executes after `on_initialize` but before inherents. - `PostInherents` a callback that executes after all inherents ran (including MBMs and `poll`). - `PostTransactions` in symmetry to `PreInherents`, this one is called before `on_finalize` but after all transactions. A sane default is to set all of these to `()`. Example diff suitable for any chain: ```patch @@ impl frame_system::Config for Test { type MaxConsumers = ConstU32<16>; + type SingleBlockMigrations = (); + type MultiBlockMigrator = (); + type PreInherents = (); + type PostInherents = (); + type PostTransactions = (); } ``` An overview of how the block execution now looks like is here. The same graph is also in the rust doc. <details><summary>Block Execution Flow</summary> <p>  </p> </details> ## Inherent Order Moved to https://github.com/paritytech/polkadot-sdk/pull/2154 --------------- ## TODO - [ ] Check that `try-runtime` still works - [ ] Ensure backwards compatibility with old Runtime APIs - [x] Consume weight correctly - [x] Cleanup --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: Juan Girini <juangirini@gmail.com> Co-authored-by: command-bot <> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de>
This commit is contained in:
Oliver Tale-Yazdi
committed by
GitHub
GitHub
parent
576681b867
commit
eefd5fe449
@@ -130,11 +130,13 @@ use frame_support::{
|
||||
DispatchResult, DispatchResultWithPostInfo, PerDispatchClass, PostDispatchInfo,
|
||||
},
|
||||
ensure, impl_ensure_origin_with_arg_ignoring_arg,
|
||||
migrations::MultiStepMigrator,
|
||||
pallet_prelude::Pays,
|
||||
storage::{self, StorageStreamIter},
|
||||
traits::{
|
||||
ConstU32, Contains, EnsureOrigin, EnsureOriginWithArg, Get, HandleLifetime,
|
||||
OnKilledAccount, OnNewAccount, OriginTrait, PalletInfo, SortedMembers, StoredMap, TypedGet,
|
||||
OnKilledAccount, OnNewAccount, OnRuntimeUpgrade, OriginTrait, PalletInfo, SortedMembers,
|
||||
StoredMap, TypedGet,
|
||||
},
|
||||
Parameter,
|
||||
};
|
||||
@@ -169,6 +171,7 @@ pub use extensions::{
|
||||
// Backward compatible re-export.
|
||||
pub use extensions::check_mortality::CheckMortality as CheckEra;
|
||||
pub use frame_support::dispatch::RawOrigin;
|
||||
use frame_support::traits::{PostInherents, PostTransactions, PreInherents};
|
||||
pub use weights::WeightInfo;
|
||||
|
||||
const LOG_TARGET: &str = "runtime::system";
|
||||
@@ -299,6 +302,11 @@ pub mod pallet {
|
||||
type BaseCallFilter = frame_support::traits::Everything;
|
||||
type BlockHashCount = frame_support::traits::ConstU64<10>;
|
||||
type OnSetCode = ();
|
||||
type SingleBlockMigrations = ();
|
||||
type MultiBlockMigrator = ();
|
||||
type PreInherents = ();
|
||||
type PostInherents = ();
|
||||
type PostTransactions = ();
|
||||
}
|
||||
|
||||
/// Default configurations of this pallet in a solo-chain environment.
|
||||
@@ -393,6 +401,11 @@ pub mod pallet {
|
||||
|
||||
/// The set code logic, just the default since we're not a parachain.
|
||||
type OnSetCode = ();
|
||||
type SingleBlockMigrations = ();
|
||||
type MultiBlockMigrator = ();
|
||||
type PreInherents = ();
|
||||
type PostInherents = ();
|
||||
type PostTransactions = ();
|
||||
}
|
||||
|
||||
/// Default configurations of this pallet in a relay-chain environment.
|
||||
@@ -572,6 +585,35 @@ pub mod pallet {
|
||||
|
||||
/// The maximum number of consumers allowed on a single account.
|
||||
type MaxConsumers: ConsumerLimits;
|
||||
|
||||
/// All migrations that should run in the next runtime upgrade.
|
||||
///
|
||||
/// These used to be formerly configured in `Executive`. Parachains need to ensure that
|
||||
/// running all these migrations in one block will not overflow the weight limit of a block.
|
||||
/// The migrations are run *before* the pallet `on_runtime_upgrade` hooks, just like the
|
||||
/// `OnRuntimeUpgrade` migrations.
|
||||
type SingleBlockMigrations: OnRuntimeUpgrade;
|
||||
|
||||
/// The migrator that is used to run Multi-Block-Migrations.
|
||||
///
|
||||
/// Can be set to [`pallet-migrations`] or an alternative implementation of the interface.
|
||||
/// The diagram in `frame_executive::block_flowchart` explains when it runs.
|
||||
type MultiBlockMigrator: MultiStepMigrator;
|
||||
|
||||
/// A callback that executes in *every block* directly before all inherents were applied.
|
||||
///
|
||||
/// See `frame_executive::block_flowchart` for a in-depth explanation when it runs.
|
||||
type PreInherents: PreInherents;
|
||||
|
||||
/// A callback that executes in *every block* directly after all inherents were applied.
|
||||
///
|
||||
/// See `frame_executive::block_flowchart` for a in-depth explanation when it runs.
|
||||
type PostInherents: PostInherents;
|
||||
|
||||
/// A callback that executes in *every block* directly after all transactions were applied.
|
||||
///
|
||||
/// See `frame_executive::block_flowchart` for a in-depth explanation when it runs.
|
||||
type PostTransactions: PostTransactions;
|
||||
}
|
||||
|
||||
#[pallet::pallet]
|
||||
@@ -619,6 +661,9 @@ pub mod pallet {
|
||||
}
|
||||
|
||||
/// Set the new runtime code without doing any checks of the given `code`.
|
||||
///
|
||||
/// Note that runtime upgrades will not run if this is called with a not-increasing spec
|
||||
/// version!
|
||||
#[pallet::call_index(3)]
|
||||
#[pallet::weight((T::SystemWeightInfo::set_code(), DispatchClass::Operational))]
|
||||
pub fn set_code_without_checks(
|
||||
@@ -814,6 +859,8 @@ pub mod pallet {
|
||||
NonZeroRefCount,
|
||||
/// The origin filter prevent the call to be dispatched.
|
||||
CallFiltered,
|
||||
/// A multi-block migration is ongoing and prevents the current code from being replaced.
|
||||
MultiBlockMigrationsOngoing,
|
||||
#[cfg(feature = "experimental")]
|
||||
/// The specified [`Task`] is not valid.
|
||||
InvalidTask,
|
||||
@@ -845,6 +892,10 @@ pub mod pallet {
|
||||
#[pallet::storage]
|
||||
pub(super) type ExtrinsicCount<T: Config> = StorageValue<_, u32>;
|
||||
|
||||
/// Whether all inherents have been applied.
|
||||
#[pallet::storage]
|
||||
pub type InherentsApplied<T: Config> = StorageValue<_, bool, ValueQuery>;
|
||||
|
||||
/// The current weight for the block.
|
||||
#[pallet::storage]
|
||||
#[pallet::whitelist_storage]
|
||||
@@ -1373,6 +1424,19 @@ impl<T: Config> Pallet<T> {
|
||||
Self::deposit_event(Event::CodeUpdated);
|
||||
}
|
||||
|
||||
/// Whether all inherents have been applied.
|
||||
pub fn inherents_applied() -> bool {
|
||||
InherentsApplied::<T>::get()
|
||||
}
|
||||
|
||||
/// Note that all inherents have been applied.
|
||||
///
|
||||
/// Should be called immediately after all inherents have been applied. Must be called at least
|
||||
/// once per block.
|
||||
pub fn note_inherents_applied() {
|
||||
InherentsApplied::<T>::put(true);
|
||||
}
|
||||
|
||||
/// Increment the reference counter on an account.
|
||||
#[deprecated = "Use `inc_consumers` instead"]
|
||||
pub fn inc_ref(who: &T::AccountId) {
|
||||
@@ -1692,6 +1756,7 @@ impl<T: Config> Pallet<T> {
|
||||
<Digest<T>>::put(digest);
|
||||
<ParentHash<T>>::put(parent_hash);
|
||||
<BlockHash<T>>::insert(*number - One::one(), parent_hash);
|
||||
<InherentsApplied<T>>::kill();
|
||||
|
||||
// Remove previous block data from storage
|
||||
BlockWeight::<T>::kill();
|
||||
@@ -1738,6 +1803,7 @@ impl<T: Config> Pallet<T> {
|
||||
ExecutionPhase::<T>::kill();
|
||||
AllExtrinsicsLen::<T>::kill();
|
||||
storage::unhashed::kill(well_known_keys::INTRABLOCK_ENTROPY);
|
||||
InherentsApplied::<T>::kill();
|
||||
|
||||
// The following fields
|
||||
//
|
||||
@@ -1981,10 +2047,14 @@ impl<T: Config> Pallet<T> {
|
||||
|
||||
/// Determine whether or not it is possible to update the code.
|
||||
///
|
||||
/// Checks the given code if it is a valid runtime wasm blob by instantianting
|
||||
/// Checks the given code if it is a valid runtime wasm blob by instantiating
|
||||
/// it and extracting the runtime version of it. It checks that the runtime version
|
||||
/// of the old and new runtime has the same spec name and that the spec version is increasing.
|
||||
pub fn can_set_code(code: &[u8]) -> Result<(), sp_runtime::DispatchError> {
|
||||
if T::MultiBlockMigrator::ongoing() {
|
||||
return Err(Error::<T>::MultiBlockMigrationsOngoing.into())
|
||||
}
|
||||
|
||||
let current_version = T::Version::get();
|
||||
let new_version = sp_io::misc::runtime_version(code)
|
||||
.and_then(|v| RuntimeVersion::decode(&mut &v[..]).ok())
|
||||
|
||||
@@ -86,6 +86,22 @@ impl Config for Test {
|
||||
type Version = Version;
|
||||
type AccountData = u32;
|
||||
type OnKilledAccount = RecordKilled;
|
||||
type MultiBlockMigrator = MockedMigrator;
|
||||
}
|
||||
|
||||
parameter_types! {
|
||||
pub static Ongoing: bool = false;
|
||||
}
|
||||
|
||||
pub struct MockedMigrator;
|
||||
impl frame_support::migrations::MultiStepMigrator for MockedMigrator {
|
||||
fn ongoing() -> bool {
|
||||
Ongoing::get()
|
||||
}
|
||||
|
||||
fn step() -> Weight {
|
||||
Weight::zero()
|
||||
}
|
||||
}
|
||||
|
||||
pub type SysEvent = frame_system::Event<Test>;
|
||||
|
||||
@@ -675,6 +675,28 @@ fn set_code_with_real_wasm_blob() {
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_code_rejects_during_mbm() {
|
||||
Ongoing::set(true);
|
||||
|
||||
let executor = substrate_test_runtime_client::new_native_or_wasm_executor();
|
||||
let mut ext = new_test_ext();
|
||||
ext.register_extension(sp_core::traits::ReadRuntimeVersionExt::new(executor));
|
||||
ext.execute_with(|| {
|
||||
System::set_block_number(1);
|
||||
let res = System::set_code(
|
||||
RawOrigin::Root.into(),
|
||||
substrate_test_runtime_client::runtime::wasm_binary_unwrap().to_vec(),
|
||||
);
|
||||
assert_eq!(
|
||||
res,
|
||||
Err(DispatchErrorWithPostInfo::from(Error::<Test>::MultiBlockMigrationsOngoing))
|
||||
);
|
||||
|
||||
assert!(System::events().is_empty());
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_code_via_authorization_works() {
|
||||
let executor = substrate_test_runtime_client::new_native_or_wasm_executor();
|
||||
|
||||
Reference in New Issue
Block a user