// This file is part of Bizinikiwi. // Copyright (C) Parity Technologies (UK) Ltd. and Dijital Kurdistan Tech Institute // SPDX-License-Identifier: Apache-2.0 // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #![deny(missing_docs)] #![deny(rustdoc::broken_intra_doc_links)] //! # `pezpallet-migrations` //! //! Provides multi block migrations for FRAME runtimes. //! //! ## Overview //! //! The pezpallet takes care of executing a batch of multi-step migrations over multiple blocks. The //! process starts on each runtime upgrade. Normal and operational transactions are paused while //! migrations are on-going. //! //! ### Example //! //! This example demonstrates a simple mocked walk through of a basic success scenario. The //! pezpallet is configured with two migrations: one succeeding after just one step, and the second //! one succeeding after two steps. A runtime upgrade is then enacted and the block number is //! advanced until all migrations finish executing. Afterwards, the recorded historic migrations are //! checked and events are asserted. #![doc = docify::embed!("src/tests.rs", simple_works)] //! //! ## Pezpallet API //! //! See the [`pezpallet`] module for more information about the interfaces this pezpallet exposes, //! including its configuration trait, dispatchables, storage items, events and errors. //! //! Otherwise noteworthy API of this pezpallet include its implementation of the //! [`MultiStepMigrator`] trait. This must be plugged into //! [`pezframe_system::Config::MultiBlockMigrator`] for proper function. //! //! The API contains some calls for emergency management. They are all prefixed with `force_` and //! should normally not be needed. Pay special attention prior to using them. //! //! ### Design Goals //! //! 1. Must automatically execute migrations over multiple blocks. //! 2. Must expose information about whether migrations are ongoing. //! 3. Must respect pessimistic weight bounds of migrations. //! 4. Must execute migrations in order. Skipping is not allowed; migrations are run on a //! all-or-nothing basis. //! 5. Must prevent re-execution of past migrations. //! 6. Must provide transactional storage semantics for migrations. //! 7. Must guarantee progress. //! //! ### Design //! //! Migrations are provided to the pezpallet through the associated type [`Config::Migrations`] of //! type [`SteppedMigrations`]. This allows multiple migrations to be aggregated through a tuple. It //! simplifies the trait bounds since all associated types of the trait must be provided by the //! pezpallet. The actual progress of the pezpallet is stored in the [`Cursor`] storage item. This //! can either be [`MigrationCursor::Active`] or [`MigrationCursor::Stuck`]. In the active case it //! points to the currently active migration and stores its inner cursor. The inner cursor can then //! be used by the migration to store its inner state and advance. Each time when the migration //! returns `Some(cursor)`, it signals the pezpallet that it is not done yet. //! //! The cursor is reset on each runtime upgrade. This ensures that it starts to execute at the //! first migration in the vector. The pallets cursor is only ever incremented or set to `Stuck` //! once it encounters an error (Goal 4). Once in the stuck state, the pezpallet will stay stuck //! until it is fixed through manual governance intervention. //! //! As soon as the cursor of the pezpallet becomes `Some(_)`; [`MultiStepMigrator::ongoing`] returns //! `true` (Goal 2). This can be used by upstream code to possibly pause transactions. //! In `on_initialize` the pezpallet will load the current migration and check whether it was //! already executed in the past by checking for membership of its ID in the [`Historic`] set. //! Historic migrations are skipped without causing an error. Each successfully executed migration //! is added to this set (Goal 5). //! //! This proceeds until no more migrations remain. At that point, the event `UpgradeCompleted` is //! emitted (Goal 1). //! //! The execution of each migration happens by calling [`SteppedMigration::transactional_step`]. //! This function wraps the inner `step` function into a transactional layer to allow rollback in //! the error case (Goal 6). //! //! Weight limits must be checked by the migration itself. The pezpallet provides a [`WeightMeter`] //! for that purpose. The pezpallet may return [`SteppedMigrationError::InsufficientWeight`] at any //! point. In that scenario, one of two things will happen: if that migration was exclusively //! executed in this block, and therefore required more than the maximum amount of weight possible, //! the process becomes `Stuck`. Otherwise, one re-attempt is executed with the same logic in the //! next block (Goal 3). Progress through the migrations is guaranteed by providing a timeout for //! each migration via [`SteppedMigration::max_steps`]. The pezpallet **ONLY** guarantees progress //! if this is set to sensible limits (Goal 7). //! //! ### Scenario: Governance cleanup //! //! Every now and then, governance can make use of the [`clear_historic`][Pezpallet::clear_historic] //! call. This ensures that no old migrations pile up in the [`Historic`] set. This can be done very //! rarely, since the storage should not grow quickly and the lookup weight does not suffer much. //! Another possibility would be to have a synchronous single-block migration perpetually deployed //! that cleans them up before the MBMs start. //! //! ### Scenario: Successful upgrade //! //! The standard procedure for a successful runtime upgrade can look like this: //! 1. Migrations are configured in the `Migrations` config item. All migrations expose //! [`max_steps`][SteppedMigration::max_steps], are error tolerant, check their weight bounds and //! have a unique identifier. //! 2. The runtime upgrade is enacted. An `UpgradeStarted` event is //! followed by lots of `MigrationAdvanced` and `MigrationCompleted` events. Finally //! `UpgradeCompleted` is emitted. //! 3. Cleanup as described in the governance scenario be executed at any time after the migrations //! completed. //! //! ### Advice: Failed upgrades //! //! Failed upgrades cannot be recovered from automatically and require governance intervention. Set //! up monitoring for `UpgradeFailed` events to be made aware of any failures. The hook //! [`FailedMigrationHandler::failed`] should be setup in a way that it allows governance to act, //! but still prevent other transactions from interacting with the inconsistent storage state. Note //! that this is paramount, since the inconsistent state might contain a faulty balance amount or //! similar that could cause great harm if user transactions don't remain suspended. One way to //! implement this would be to use the `SafeMode` or `TxPause` pallets that can prevent most user //! interactions but still allow a whitelisted set of governance calls. //! //! ### Remark: Failed migrations //! //! Failed migrations are not added to the `Historic` set. This means that an erroneous //! migration must be removed and fixed manually. This already applies, even before considering the //! historic set. //! //! ### Remark: Transactional processing //! //! You can see the transactional semantics for migration steps as mostly useless, since in the //! stuck case the state is already messed up. This just prevents it from becoming even more messed //! up, but doesn't prevent it in the first place. #![cfg_attr(not(feature = "std"), no_std)] mod benchmarking; pub mod migrations; mod mock; pub mod mock_helpers; mod tests; pub mod weights; extern crate alloc; pub use pezpallet::*; pub use weights::WeightInfo; use alloc::vec::Vec; use codec::{Decode, DecodeWithMemTracking, Encode, MaxEncodedLen}; use core::ops::ControlFlow; use pezframe_support::{ defensive, defensive_assert, migrations::*, pezpallet_prelude::*, traits::Get, weights::{Weight, WeightMeter}, BoundedVec, }; use pezframe_system::{ pezpallet_prelude::{BlockNumberFor, *}, Pezpallet as System, }; use pezsp_runtime::Saturating; /// Points to the next migration to execute. #[derive( Debug, Clone, Eq, PartialEq, Encode, Decode, DecodeWithMemTracking, scale_info::TypeInfo, MaxEncodedLen, )] pub enum MigrationCursor { /// Points to the currently active migration and its inner cursor. Active(ActiveCursor), /// Migration got stuck and cannot proceed. This is bad. Stuck, } impl MigrationCursor { /// Try to return self as an [`ActiveCursor`]. pub fn as_active(&self) -> Option<&ActiveCursor> { match self { MigrationCursor::Active(active) => Some(active), MigrationCursor::Stuck => None, } } } impl From> for MigrationCursor { fn from(active: ActiveCursor) -> Self { MigrationCursor::Active(active) } } /// Points to the currently active migration and its inner cursor. #[derive( Debug, Clone, Eq, PartialEq, Encode, Decode, DecodeWithMemTracking, scale_info::TypeInfo, MaxEncodedLen, )] pub struct ActiveCursor { /// The index of the migration in the MBM tuple. pub index: u32, /// The cursor of the migration that is referenced by `index`. pub inner_cursor: Option, /// The block number that the migration started at. /// /// This is used to calculate how many blocks it took. pub started_at: BlockNumber, } impl ActiveCursor { /// Advance the overarching cursor to the next migration. pub(crate) fn goto_next_migration(&mut self, current_block: BlockNumber) { self.index.saturating_inc(); self.inner_cursor = None; self.started_at = current_block; } } /// How to clear the records of historic migrations. #[derive( Debug, Clone, Eq, PartialEq, Encode, Decode, DecodeWithMemTracking, scale_info::TypeInfo, )] pub enum HistoricCleanupSelector { /// Clear exactly these entries. /// /// This is the advised way of doing it. Specific(Vec), /// Clear up to this many entries Wildcard { /// How many should be cleared in this call at most. limit: Option, /// The cursor that was emitted from any previous `HistoricCleared`. /// /// Does not need to be passed when clearing the first batch. previous_cursor: Option>, }, } /// The default number of entries that should be cleared by a `HistoricCleanupSelector::Wildcard`. /// /// The caller can explicitly specify a higher amount. Benchmarks are run with twice this value. const DEFAULT_HISTORIC_BATCH_CLEAR_SIZE: u32 = 128; impl HistoricCleanupSelector { /// The maximal number of entries that this will remove. /// /// Needed for weight calculation. pub fn limit(&self) -> u32 { match self { Self::Specific(ids) => ids.len() as u32, Self::Wildcard { limit, .. } => limit.unwrap_or(DEFAULT_HISTORIC_BATCH_CLEAR_SIZE), } } } /// Convenience alias for [`MigrationCursor`]. pub type CursorOf = MigrationCursor, BlockNumberFor>; /// Convenience alias for the raw inner cursor of a migration. pub type RawCursorOf = BoundedVec::CursorMaxLen>; /// Convenience alias for the identifier of a migration. pub type IdentifierOf = BoundedVec::IdentifierMaxLen>; /// Convenience alias for [`ActiveCursor`]. pub type ActiveCursorOf = ActiveCursor, BlockNumberFor>; /// Trait for a tuple of No-OP migrations with one element. #[impl_trait_for_tuples::impl_for_tuples(30)] pub trait MockedMigrations: SteppedMigrations { /// The migration should fail after `n` steps. fn set_fail_after(n: u32); /// The migration should succeed after `n` steps. fn set_success_after(n: u32); } #[cfg(feature = "try-runtime")] /// Wrapper for pre-upgrade bytes, allowing us to impl MEL on it. /// /// For `try-runtime` testing only. #[derive(Debug, Clone, Eq, PartialEq, Encode, Decode, scale_info::TypeInfo, Default)] struct PreUpgradeBytesWrapper(pub Vec); /// Data stored by the pre-upgrade hook of the MBMs. Only used for `try-runtime` testing. /// /// Define this outside of the pezpallet so it is not confused with actual storage. #[cfg(feature = "try-runtime")] #[pezframe_support::storage_alias] type PreUpgradeBytes = StorageMap, Twox64Concat, IdentifierOf, PreUpgradeBytesWrapper, ValueQuery>; #[pezframe_support::pezpallet] pub mod pezpallet { use super::*; /// The in-code storage version. const STORAGE_VERSION: StorageVersion = StorageVersion::new(0); #[pezpallet::pezpallet] #[pezpallet::storage_version(STORAGE_VERSION)] pub struct Pezpallet(_); #[pezpallet::config(with_default)] pub trait Config: pezframe_system::Config { /// The overarching event type of the runtime. #[pezpallet::no_default_bounds] #[allow(deprecated)] type RuntimeEvent: From> + IsType<::RuntimeEvent>; /// All the multi-block migrations to run. /// /// Should only be updated in a runtime-upgrade once all the old migrations have completed. /// (Check that [`Cursor`] is `None`). #[cfg(not(feature = "runtime-benchmarks"))] #[pezpallet::no_default] type Migrations: SteppedMigrations; /// Mocked migrations for benchmarking only. /// /// Should be configured to [`crate::mock_helpers::MockedMigrations`] in benchmarks. #[cfg(feature = "runtime-benchmarks")] #[pezpallet::no_default] type Migrations: MockedMigrations; /// The maximal length of an encoded cursor. /// /// A good default needs to selected such that no migration will ever have a cursor with MEL /// above this limit. This is statically checked in `integrity_test`. #[pezpallet::constant] type CursorMaxLen: Get; /// The maximal length of an encoded identifier. /// /// A good default needs to selected such that no migration will ever have an identifier /// with MEL above this limit. This is statically checked in `integrity_test`. #[pezpallet::constant] type IdentifierMaxLen: Get; /// Notifications for status updates of a runtime upgrade. /// /// Could be used to pause XCM etc. type MigrationStatusHandler: MigrationStatusHandler; /// Handler for failed migrations. type FailedMigrationHandler: FailedMigrationHandler; /// The maximum weight to spend each block to execute migrations. type MaxServiceWeight: Get; /// Weight information for the calls and functions of this pezpallet. type WeightInfo: WeightInfo; } /// Default implementations of [`DefaultConfig`], which can be used to implement [`Config`]. pub mod config_preludes { use super::{inject_runtime_type, DefaultConfig}; use pezframe_support::{ derive_impl, migrations::FreezeChainOnFailedMigration, pezpallet_prelude::{ConstU32, *}, }; use pezframe_system::limits::BlockWeights; /// Provides a viable default config that can be used with /// [`derive_impl`](`pezframe_support::derive_impl`) to derive a testing pezpallet config /// based on this one. /// /// See `Test` in the `default-config` example pezpallet's `test.rs` for an example of /// a downstream user of this particular `TestDefaultConfig` pub struct TestDefaultConfig; pezframe_support::parameter_types! { /// Maximal weight per block that can be spent on migrations in tests. pub TestMaxServiceWeight: Weight = <::BlockWeights as Get>::get().max_block.div(2); } #[derive_impl(pezframe_system::config_preludes::TestDefaultConfig, no_aggregated_types)] impl pezframe_system::DefaultConfig for TestDefaultConfig {} #[pezframe_support::register_default_impl(TestDefaultConfig)] impl DefaultConfig for TestDefaultConfig { #[inject_runtime_type] type RuntimeEvent = (); type CursorMaxLen = ConstU32<{ 1 << 16 }>; type IdentifierMaxLen = ConstU32<{ 256 }>; type MigrationStatusHandler = (); type FailedMigrationHandler = FreezeChainOnFailedMigration; type MaxServiceWeight = TestMaxServiceWeight; type WeightInfo = (); } } /// The currently active migration to run and its cursor. /// /// `None` indicates that no migration is running. #[pezpallet::storage] pub type Cursor = StorageValue<_, CursorOf, OptionQuery>; /// Set of all successfully executed migrations. /// /// This is used as blacklist, to not re-execute migrations that have not been removed from the /// codebase yet. Governance can regularly clear this out via `clear_historic`. #[pezpallet::storage] pub type Historic = StorageMap<_, Twox64Concat, IdentifierOf, (), OptionQuery>; #[pezpallet::event] #[pezpallet::generate_deposit(pub(super) fn deposit_event)] pub enum Event { /// A Runtime upgrade started. /// /// Its end is indicated by `UpgradeCompleted` or `UpgradeFailed`. UpgradeStarted { /// The number of migrations that this upgrade contains. /// /// This can be used to design a progress indicator in combination with counting the /// `MigrationCompleted` and `MigrationSkipped` events. migrations: u32, }, /// The current runtime upgrade completed. /// /// This implies that all of its migrations completed successfully as well. UpgradeCompleted, /// Runtime upgrade failed. /// /// This is very bad and will require governance intervention. UpgradeFailed, /// A migration was skipped since it was already executed in the past. MigrationSkipped { /// The index of the skipped migration within the [`Config::Migrations`] list. index: u32, }, /// A migration progressed. MigrationAdvanced { /// The index of the migration within the [`Config::Migrations`] list. index: u32, /// The number of blocks that this migration took so far. took: BlockNumberFor, }, /// A Migration completed. MigrationCompleted { /// The index of the migration within the [`Config::Migrations`] list. index: u32, /// The number of blocks that this migration took so far. took: BlockNumberFor, }, /// A Migration failed. /// /// This implies that the whole upgrade failed and governance intervention is required. MigrationFailed { /// The index of the migration within the [`Config::Migrations`] list. index: u32, /// The number of blocks that this migration took so far. took: BlockNumberFor, }, /// The set of historical migrations has been cleared. HistoricCleared { /// Should be passed to `clear_historic` in a successive call. next_cursor: Option>, }, } #[pezpallet::error] pub enum Error { /// The operation cannot complete since some MBMs are ongoing. Ongoing, } #[pezpallet::hooks] impl Hooks> for Pezpallet { fn on_runtime_upgrade() -> Weight { Self::onboard_new_mbms() } #[cfg(feature = "std")] fn integrity_test() { // Check that the migrations tuple is legit. pezframe_support::assert_ok!(T::Migrations::integrity_test()); // Very important! Ensure that the pezpallet is configured in `System::Config`. { assert!(!Cursor::::exists(), "Externalities storage should be clean"); assert!(!::MultiBlockMigrator::ongoing()); Cursor::::put(MigrationCursor::Stuck); assert!(::MultiBlockMigrator::ongoing()); Cursor::::kill(); } // The per-block service weight is sane. { let want = T::MaxServiceWeight::get(); let max = ::BlockWeights::get().max_block; assert!(want.all_lte(max), "Service weight is larger than a block: {want} > {max}"); } // Cursor MEL { let mel = T::Migrations::cursor_max_encoded_len(); let max_mel = T::CursorMaxLen::get() as usize; assert!( mel <= max_mel, "A Cursor is not guaranteed to fit into the storage: {mel} > {max_mel}", ); } // Identifier MEL { let mel = T::Migrations::identifier_max_encoded_len(); let max_mel = T::IdentifierMaxLen::get() as usize; assert!( mel <= max_mel, "An Identifier is not guaranteed to fit into the storage: {mel} > {max_mel}", ); } } } #[pezpallet::call(weight = T::WeightInfo)] impl Pezpallet { /// Allows root to set a cursor to forcefully start, stop or forward the migration process. /// /// Should normally not be needed and is only in place as emergency measure. Note that /// restarting the migration process in this manner will not call the /// [`MigrationStatusHandler::started`] hook or emit an `UpgradeStarted` event. #[pezpallet::call_index(0)] pub fn force_set_cursor( origin: OriginFor, cursor: Option>, ) -> DispatchResult { ensure_root(origin)?; Cursor::::set(cursor); Ok(()) } /// Allows root to set an active cursor to forcefully start/forward the migration process. /// /// This is an edge-case version of [`Self::force_set_cursor`] that allows to set the /// `started_at` value to the next block number. Otherwise this would not be possible, since /// `force_set_cursor` takes an absolute block number. Setting `started_at` to `None` /// indicates that the current block number plus one should be used. #[pezpallet::call_index(1)] pub fn force_set_active_cursor( origin: OriginFor, index: u32, inner_cursor: Option>, started_at: Option>, ) -> DispatchResult { ensure_root(origin)?; let started_at = started_at.unwrap_or( System::::block_number().saturating_add(pezsp_runtime::traits::One::one()), ); Cursor::::put(MigrationCursor::Active(ActiveCursor { index, inner_cursor, started_at, })); Ok(()) } /// Forces the onboarding of the migrations. /// /// This process happens automatically on a runtime upgrade. It is in place as an emergency /// measurement. The cursor needs to be `None` for this to succeed. #[pezpallet::call_index(2)] pub fn force_onboard_mbms(origin: OriginFor) -> DispatchResult { ensure_root(origin)?; ensure!(!Cursor::::exists(), Error::::Ongoing); Self::onboard_new_mbms(); Ok(()) } /// Clears the `Historic` set. /// /// `map_cursor` must be set to the last value that was returned by the /// `HistoricCleared` event. The first time `None` can be used. `limit` must be chosen in a /// way that will result in a sensible weight. #[pezpallet::call_index(3)] #[pezpallet::weight(T::WeightInfo::clear_historic(selector.limit()))] pub fn clear_historic( origin: OriginFor, selector: HistoricCleanupSelector>, ) -> DispatchResult { ensure_root(origin)?; match &selector { HistoricCleanupSelector::Specific(ids) => { for id in ids { Historic::::remove(id); } Self::deposit_event(Event::HistoricCleared { next_cursor: None }); }, HistoricCleanupSelector::Wildcard { previous_cursor, .. } => { let next = Historic::::clear(selector.limit(), previous_cursor.as_deref()); Self::deposit_event(Event::HistoricCleared { next_cursor: next.maybe_cursor }); }, } Ok(()) } } } impl Pezpallet { /// Onboard all new Multi-Block-Migrations and start the process of executing them. /// /// Should only be called once all previous migrations completed. fn onboard_new_mbms() -> Weight { if let Some(cursor) = Cursor::::get() { log::error!("Ongoing migrations interrupted - chain stuck"); let maybe_index = cursor.as_active().map(|c| c.index); Self::upgrade_failed(maybe_index); return T::WeightInfo::onboard_new_mbms(); } let migrations = T::Migrations::len(); log::debug!("Onboarding {migrations} new MBM migrations"); if migrations > 0 { // Set the cursor to the first migration: Cursor::::set(Some( ActiveCursor { index: 0, inner_cursor: None, started_at: System::::block_number(), } .into(), )); Self::deposit_event(Event::UpgradeStarted { migrations }); T::MigrationStatusHandler::started(); } T::WeightInfo::onboard_new_mbms() } /// Tries to make progress on the Multi-Block-Migrations process. fn progress_mbms(n: BlockNumberFor) -> Weight { let mut meter = WeightMeter::with_limit(T::MaxServiceWeight::get()); meter.consume(T::WeightInfo::progress_mbms_none()); let mut cursor = match Cursor::::get() { None => { log::trace!("[Block {n:?}] Waiting for cursor to become `Some`."); return meter.consumed(); }, Some(MigrationCursor::Active(cursor)) => { log::debug!("Progressing MBM #{}", cursor.index); cursor }, Some(MigrationCursor::Stuck) => { log::error!("Migration stuck. Governance intervention required."); return meter.consumed(); }, }; debug_assert!(Self::ongoing()); // The limit here is a defensive measure to prevent an infinite loop. It expresses that we // allow no more than 8 MBMs to finish in a single block. This should be harmless, since we // generally expect *Multi*-Block-Migrations to take *multiple* blocks. for i in 0..8 { match Self::exec_migration(cursor, i == 0, &mut meter) { None => return meter.consumed(), Some(ControlFlow::Continue(next_cursor)) => { cursor = next_cursor; }, Some(ControlFlow::Break(last_cursor)) => { cursor = last_cursor; break; }, } } Cursor::::set(Some(cursor.into())); meter.consumed() } /// Try to make progress on the current migration. /// /// Returns whether processing should continue or break for this block. The return value means: /// - `None`: The migration process is completely finished. /// - `ControlFlow::Break`: Continue in the *next* block with the given cursor. /// - `ControlFlow::Continue`: Continue in the *current* block with the given cursor. fn exec_migration( mut cursor: ActiveCursorOf, is_first: bool, meter: &mut WeightMeter, ) -> Option, ActiveCursorOf>> { // The differences between the single branches' weights is not that big. And since we do // only one step per block, we can just use the maximum instead of more precise accounting. if meter.try_consume(Self::exec_migration_max_weight()).is_err() { defensive_assert!(!is_first, "There should be enough weight to do this at least once"); return Some(ControlFlow::Break(cursor)); } if cursor.index >= T::Migrations::len() { // No more migrations in the tuple - we are done. defensive_assert!(cursor.index == T::Migrations::len(), "Inconsistent MBMs tuple"); Self::deposit_event(Event::UpgradeCompleted); Cursor::::kill(); T::MigrationStatusHandler::completed(); return None; }; let id = T::Migrations::nth_id(cursor.index).map(TryInto::try_into); let Some(Ok(bounded_id)): Option, _>> = id else { defensive!("integrity_test ensures that all identifiers are present and bounde; qed."); Self::upgrade_failed(Some(cursor.index)); return None; }; if Historic::::contains_key(&bounded_id) { Self::deposit_event(Event::MigrationSkipped { index: cursor.index }); cursor.goto_next_migration(System::::block_number()); return Some(ControlFlow::Continue(cursor)); } let max_steps = T::Migrations::nth_max_steps(cursor.index); // If this is the first time running this migration, exec the pre-upgrade hook. #[cfg(feature = "try-runtime")] if !PreUpgradeBytes::::contains_key(&bounded_id) { let bytes = T::Migrations::nth_pre_upgrade(cursor.index) .expect("Invalid cursor.index") .expect("Pre-upgrade failed"); PreUpgradeBytes::::insert(&bounded_id, PreUpgradeBytesWrapper(bytes)); } let next_cursor = T::Migrations::nth_transactional_step( cursor.index, cursor.inner_cursor.clone().map(|c| c.into_inner()), meter, ); let Some((max_steps, next_cursor)) = max_steps.zip(next_cursor) else { defensive!("integrity_test ensures that the tuple is valid; qed"); Self::upgrade_failed(Some(cursor.index)); return None; }; let took = System::::block_number().saturating_sub(cursor.started_at); match next_cursor { Ok(Some(next_cursor)) => { let Ok(bound_next_cursor) = next_cursor.try_into() else { defensive!("The integrity check ensures that all cursors' MEL bound fits into CursorMaxLen; qed"); Self::upgrade_failed(Some(cursor.index)); return None; }; Self::deposit_event(Event::MigrationAdvanced { index: cursor.index, took }); cursor.inner_cursor = Some(bound_next_cursor); if max_steps.is_some_and(|max| took > max.into()) { Self::deposit_event(Event::MigrationFailed { index: cursor.index, took }); Self::upgrade_failed(Some(cursor.index)); None } else { // A migration cannot progress more than one step per block, we therefore break. Some(ControlFlow::Break(cursor)) } }, Ok(None) => { // A migration is done when it returns cursor `None`. // Run post-upgrade checks. #[cfg(feature = "try-runtime")] T::Migrations::nth_post_upgrade( cursor.index, PreUpgradeBytes::::get(&bounded_id).0, ) .expect("Invalid cursor.index.") .expect("Post-upgrade failed."); Self::deposit_event(Event::MigrationCompleted { index: cursor.index, took }); Historic::::insert(&bounded_id, ()); cursor.goto_next_migration(System::::block_number()); Some(ControlFlow::Continue(cursor)) }, Err(SteppedMigrationError::InsufficientWeight { required }) => { if is_first || required.any_gt(meter.limit()) { Self::deposit_event(Event::MigrationFailed { index: cursor.index, took }); Self::upgrade_failed(Some(cursor.index)); None } else { // Retry and hope that there is more weight in the next block. Some(ControlFlow::Break(cursor)) } }, Err(SteppedMigrationError::InvalidCursor | SteppedMigrationError::Failed) => { Self::deposit_event(Event::MigrationFailed { index: cursor.index, took }); Self::upgrade_failed(Some(cursor.index)); None }, } } /// Fail the current runtime upgrade, caused by `migration`. /// /// When the `try-runtime` feature is enabled, this function will panic. // Allow unreachable code so it can compile without warnings when `try-runtime` is enabled. fn upgrade_failed(migration: Option) { use FailedMigrationHandling::*; Self::deposit_event(Event::UpgradeFailed); if cfg!(feature = "try-runtime") { panic!("Migration with index {migration:?} failed."); } else { match T::FailedMigrationHandler::failed(migration) { KeepStuck => Cursor::::set(Some(MigrationCursor::Stuck)), ForceUnstuck => Cursor::::kill(), Ignore => {}, } } } /// The maximal weight of calling the private `Self::exec_migration` function. pub fn exec_migration_max_weight() -> Weight { T::WeightInfo::exec_migration_complete() .max(T::WeightInfo::exec_migration_completed()) .max(T::WeightInfo::exec_migration_skipped_historic()) .max(T::WeightInfo::exec_migration_advance()) .max(T::WeightInfo::exec_migration_fail()) } } impl MultiStepMigrator for Pezpallet { fn ongoing() -> bool { Cursor::::exists() } fn step() -> Weight { Self::progress_mbms(System::::block_number()) } }