mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-24 09:01:08 +00:00
eefd5fe449
This MR is the merge of https://github.com/paritytech/substrate/pull/14414 and https://github.com/paritytech/substrate/pull/14275. It implements [RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes https://github.com/paritytech/polkadot-sdk/issues/198. ----- This Merge request introduces three major topicals: 1. Multi-Block-Migrations 1. New pallet `poll` hook for periodic service work 1. Replacement hooks for `on_initialize` and `on_finalize` in cases where `poll` cannot be used and some more general changes to FRAME. The changes for each topical span over multiple crates. They are listed in topical order below. # 1.) Multi-Block-Migrations Multi-Block-Migrations are facilitated by creating `pallet_migrations` and configuring `System::Config::MultiBlockMigrator` to point to it. Executive picks this up and triggers one step of the migrations pallet per block. The chain is in lockdown mode for as long as an MBM is ongoing. Executive does this by polling `MultiBlockMigrator::ongoing` and not allowing any transaction in a block, if true. A MBM is defined through trait `SteppedMigration`. A condensed version looks like this: ```rust /// A migration that can proceed in multiple steps. pub trait SteppedMigration { type Cursor: FullCodec + MaxEncodedLen; type Identifier: FullCodec + MaxEncodedLen; fn id() -> Self::Identifier; fn max_steps() -> Option<u32>; fn step( cursor: Option<Self::Cursor>, meter: &mut WeightMeter, ) -> Result<Option<Self::Cursor>, SteppedMigrationError>; } ``` `pallet_migrations` can be configured with an aggregated tuple of these migrations. It then starts to migrate them one-by-one on the next runtime upgrade. Two things are important here: - 1. Doing another runtime upgrade while MBMs are ongoing is not a good idea and can lead to messed up state. - 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`, otherwise it is not used.** The pallet supports an `UpgradeStatusHandler` that can be used to notify external logic of upgrade start/finish (for example to pause XCM dispatch). Error recovery is very limited in the case that a migration errors or times out (exceeds its `max_steps`). Currently the runtime dev can decide in `FailedMigrationHandler::failed` how to handle this. One follow-up would be to pair this with the `SafeMode` pallet and enact safe mode when an upgrade fails, to allow governance to rescue the chain. This is currently not possible, since governance is not `Mandatory`. ## Runtime API - `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to inform the Block Author whether they can push transactions. ### Integration Add it to your runtime implementation of `Core` and `BlockBuilder`: ```patch diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs @@ impl_runtime_apis! { impl sp_block_builder::Core<Block> for Runtime { - fn initialize_block(header: &<Block as BlockT>::Header) { + fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode { Executive::initialize_block(header) } ... } ``` # 2.) `poll` hook A new pallet hook is introduced: `poll`. `Poll` is intended to replace mostly all usage of `on_initialize`. The reason for this is that any code that can be called from `on_initialize` cannot be migrated through an MBM. Currently there is no way to statically check this; the implication is to use `on_initialize` as rarely as possible. Failing to do so can result in broken storage invariants. The implementation of the poll hook depends on the `Runtime API` changes that are explained above. # 3.) Hard-Deadline callbacks Three new callbacks are introduced and configured on `System::Config`: `PreInherents`, `PostInherents` and `PostTransactions`. These hooks are meant as replacement for `on_initialize` and `on_finalize` in cases where the code that runs cannot be moved to `poll`. The reason for this is to make the usage of HD-code (hard deadline) more explicit - again to prevent broken invariants by MBMs. # 4.) FRAME (general changes) ## `frame_system` pallet A new memorize storage item `InherentsApplied` is added. It is used by executive to track whether inherents have already been applied. Executive and can then execute the MBMs directly between inherents and transactions. The `Config` gets five new items: - `SingleBlockMigrations` this is the new way of configuring migrations that run in a single block. Previously they were defined as last generic argument of `Executive`. This shift is brings all central configuration about migrations closer into view of the developer (migrations that are configured in `Executive` will still work for now but is deprecated). - `MultiBlockMigrator` this can be configured to an engine that drives MBMs. One example would be the `pallet_migrations`. Note that this is only the engine; the exact MBMs are injected into the engine. - `PreInherents` a callback that executes after `on_initialize` but before inherents. - `PostInherents` a callback that executes after all inherents ran (including MBMs and `poll`). - `PostTransactions` in symmetry to `PreInherents`, this one is called before `on_finalize` but after all transactions. A sane default is to set all of these to `()`. Example diff suitable for any chain: ```patch @@ impl frame_system::Config for Test { type MaxConsumers = ConstU32<16>; + type SingleBlockMigrations = (); + type MultiBlockMigrator = (); + type PreInherents = (); + type PostInherents = (); + type PostTransactions = (); } ``` An overview of how the block execution now looks like is here. The same graph is also in the rust doc. <details><summary>Block Execution Flow</summary> <p>  </p> </details> ## Inherent Order Moved to https://github.com/paritytech/polkadot-sdk/pull/2154 --------------- ## TODO - [ ] Check that `try-runtime` still works - [ ] Ensure backwards compatibility with old Runtime APIs - [x] Consume weight correctly - [x] Cleanup --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: Juan Girini <juangirini@gmail.com> Co-authored-by: command-bot <> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de>
320 lines
9.6 KiB
Rust
320 lines
9.6 KiB
Rust
// This file is part of Substrate.
|
|
|
|
// Copyright (C) Parity Technologies (UK) Ltd.
|
|
// SPDX-License-Identifier: Apache-2.0
|
|
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
//! Provides functionality around the transaction storage.
|
|
//!
|
|
//! Transactional storage provides functionality to run an entire code block
|
|
//! in a storage transaction. This means that either the entire changes to the
|
|
//! storage are committed or everything is thrown away. This simplifies the
|
|
//! writing of functionality that may bail at any point of operation. Otherwise
|
|
//! you would need to first verify all storage accesses and then do the storage
|
|
//! modifications.
|
|
//!
|
|
//! [`with_transaction`] provides a way to run a given closure in a transactional context.
|
|
|
|
use sp_io::storage::{commit_transaction, rollback_transaction, start_transaction};
|
|
use sp_runtime::{DispatchError, TransactionOutcome, TransactionalError};
|
|
|
|
/// The type that is being used to store the current number of active layers.
|
|
pub type Layer = u32;
|
|
/// The key that is holds the current number of active layers.
|
|
///
|
|
/// Encodes to `0x3a7472616e73616374696f6e5f6c6576656c3a`.
|
|
pub const TRANSACTION_LEVEL_KEY: &[u8] = b":transaction_level:";
|
|
/// The maximum number of nested layers.
|
|
pub const TRANSACTIONAL_LIMIT: Layer = 255;
|
|
|
|
/// Returns the current number of nested transactional layers.
|
|
fn get_transaction_level() -> Layer {
|
|
crate::storage::unhashed::get_or_default::<Layer>(TRANSACTION_LEVEL_KEY)
|
|
}
|
|
|
|
/// Set the current number of nested transactional layers.
|
|
fn set_transaction_level(level: Layer) {
|
|
crate::storage::unhashed::put::<Layer>(TRANSACTION_LEVEL_KEY, &level);
|
|
}
|
|
|
|
/// Kill the transactional layers storage.
|
|
fn kill_transaction_level() {
|
|
crate::storage::unhashed::kill(TRANSACTION_LEVEL_KEY);
|
|
}
|
|
|
|
/// Increments the transaction level. Returns an error if levels go past the limit.
|
|
///
|
|
/// Returns a guard that when dropped decrements the transaction level automatically.
|
|
fn inc_transaction_level() -> Result<StorageLayerGuard, ()> {
|
|
let existing_levels = get_transaction_level();
|
|
if existing_levels >= TRANSACTIONAL_LIMIT {
|
|
return Err(())
|
|
}
|
|
// Cannot overflow because of check above.
|
|
set_transaction_level(existing_levels + 1);
|
|
Ok(StorageLayerGuard)
|
|
}
|
|
|
|
fn dec_transaction_level() {
|
|
let existing_levels = get_transaction_level();
|
|
if existing_levels == 0 {
|
|
log::warn!(
|
|
"We are underflowing with calculating transactional levels. Not great, but let's not panic...",
|
|
);
|
|
} else if existing_levels == 1 {
|
|
// Don't leave any trace of this storage item.
|
|
kill_transaction_level();
|
|
} else {
|
|
// Cannot underflow because of checks above.
|
|
set_transaction_level(existing_levels - 1);
|
|
}
|
|
}
|
|
|
|
struct StorageLayerGuard;
|
|
|
|
impl Drop for StorageLayerGuard {
|
|
fn drop(&mut self) {
|
|
dec_transaction_level()
|
|
}
|
|
}
|
|
|
|
/// Check if the current call is within a transactional layer.
|
|
pub fn is_transactional() -> bool {
|
|
get_transaction_level() > 0
|
|
}
|
|
|
|
/// Execute the supplied function in a new storage transaction.
|
|
///
|
|
/// All changes to storage performed by the supplied function are discarded if the returned
|
|
/// outcome is `TransactionOutcome::Rollback`.
|
|
///
|
|
/// Transactions can be nested up to `TRANSACTIONAL_LIMIT` times; more than that will result in an
|
|
/// error.
|
|
///
|
|
/// Commits happen to the parent transaction.
|
|
pub fn with_transaction<T, E, F>(f: F) -> Result<T, E>
|
|
where
|
|
E: From<DispatchError>,
|
|
F: FnOnce() -> TransactionOutcome<Result<T, E>>,
|
|
{
|
|
// This needs to happen before `start_transaction` below.
|
|
// Otherwise we may rollback the increase, then decrease as the guard goes out of scope
|
|
// and then end in some bad state.
|
|
let _guard = inc_transaction_level().map_err(|()| TransactionalError::LimitReached.into())?;
|
|
|
|
start_transaction();
|
|
|
|
match f() {
|
|
TransactionOutcome::Commit(res) => {
|
|
commit_transaction();
|
|
res
|
|
},
|
|
TransactionOutcome::Rollback(res) => {
|
|
rollback_transaction();
|
|
res
|
|
},
|
|
}
|
|
}
|
|
|
|
/// Same as [`with_transaction`] but casts any internal error to `()`.
|
|
///
|
|
/// This rids `E` of the `From<DispatchError>` bound that is required by `with_transaction`.
|
|
pub fn with_transaction_opaque_err<T, E, F>(f: F) -> Result<Result<T, E>, ()>
|
|
where
|
|
F: FnOnce() -> TransactionOutcome<Result<T, E>>,
|
|
{
|
|
with_transaction(move || -> TransactionOutcome<Result<Result<T, E>, DispatchError>> {
|
|
match f() {
|
|
TransactionOutcome::Commit(res) => TransactionOutcome::Commit(Ok(res)),
|
|
TransactionOutcome::Rollback(res) => TransactionOutcome::Rollback(Ok(res)),
|
|
}
|
|
})
|
|
.map_err(|_| ())
|
|
}
|
|
|
|
/// Same as [`with_transaction`] but without a limit check on nested transactional layers.
|
|
///
|
|
/// This is mostly for backwards compatibility before there was a transactional layer limit.
|
|
/// It is recommended to only use [`with_transaction`] to avoid users from generating too many
|
|
/// transactional layers.
|
|
pub fn with_transaction_unchecked<R, F>(f: F) -> R
|
|
where
|
|
F: FnOnce() -> TransactionOutcome<R>,
|
|
{
|
|
// This needs to happen before `start_transaction` below.
|
|
// Otherwise we may rollback the increase, then decrease as the guard goes out of scope
|
|
// and then end in some bad state.
|
|
let maybe_guard = inc_transaction_level();
|
|
|
|
if maybe_guard.is_err() {
|
|
log::warn!(
|
|
"The transactional layer limit has been reached, and new transactional layers are being
|
|
spawned with `with_transaction_unchecked`. This could be caused by someone trying to
|
|
attack your chain, and you should investigate usage of `with_transaction_unchecked` and
|
|
potentially migrate to `with_transaction`, which enforces a transactional limit.",
|
|
);
|
|
}
|
|
|
|
start_transaction();
|
|
|
|
match f() {
|
|
TransactionOutcome::Commit(res) => {
|
|
commit_transaction();
|
|
res
|
|
},
|
|
TransactionOutcome::Rollback(res) => {
|
|
rollback_transaction();
|
|
res
|
|
},
|
|
}
|
|
}
|
|
|
|
/// Execute the supplied function, adding a new storage layer.
|
|
///
|
|
/// This is the same as `with_transaction`, but assuming that any function returning an `Err` should
|
|
/// rollback, and any function returning `Ok` should commit. This provides a cleaner API to the
|
|
/// developer who wants this behavior.
|
|
pub fn with_storage_layer<T, E, F>(f: F) -> Result<T, E>
|
|
where
|
|
E: From<DispatchError>,
|
|
F: FnOnce() -> Result<T, E>,
|
|
{
|
|
with_transaction(|| {
|
|
let r = f();
|
|
if r.is_ok() {
|
|
TransactionOutcome::Commit(r)
|
|
} else {
|
|
TransactionOutcome::Rollback(r)
|
|
}
|
|
})
|
|
}
|
|
|
|
/// Execute the supplied function, ensuring we are at least in one storage layer.
|
|
///
|
|
/// If we are already in a storage layer, we just execute the provided closure.
|
|
/// If we are not, we execute the closure within a [`with_storage_layer`].
|
|
pub fn in_storage_layer<T, E, F>(f: F) -> Result<T, E>
|
|
where
|
|
E: From<DispatchError>,
|
|
F: FnOnce() -> Result<T, E>,
|
|
{
|
|
if is_transactional() {
|
|
f()
|
|
} else {
|
|
with_storage_layer(f)
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use crate::{assert_noop, assert_ok};
|
|
use sp_io::TestExternalities;
|
|
use sp_runtime::DispatchResult;
|
|
|
|
#[test]
|
|
fn is_transactional_should_return_false() {
|
|
TestExternalities::default().execute_with(|| {
|
|
assert!(!is_transactional());
|
|
});
|
|
}
|
|
|
|
#[test]
|
|
fn is_transactional_should_not_error_in_with_transaction() {
|
|
TestExternalities::default().execute_with(|| {
|
|
assert_ok!(with_transaction(|| -> TransactionOutcome<DispatchResult> {
|
|
assert!(is_transactional());
|
|
TransactionOutcome::Commit(Ok(()))
|
|
}));
|
|
|
|
assert_noop!(
|
|
with_transaction(|| -> TransactionOutcome<DispatchResult> {
|
|
assert!(is_transactional());
|
|
TransactionOutcome::Rollback(Err("revert".into()))
|
|
}),
|
|
"revert"
|
|
);
|
|
});
|
|
}
|
|
|
|
fn recursive_transactional(num: u32) -> DispatchResult {
|
|
if num == 0 {
|
|
return Ok(())
|
|
}
|
|
|
|
with_transaction(|| -> TransactionOutcome<DispatchResult> {
|
|
let res = recursive_transactional(num - 1);
|
|
TransactionOutcome::Commit(res)
|
|
})
|
|
}
|
|
|
|
#[test]
|
|
fn transaction_limit_should_work() {
|
|
TestExternalities::default().execute_with(|| {
|
|
assert_eq!(get_transaction_level(), 0);
|
|
|
|
assert_ok!(with_transaction(|| -> TransactionOutcome<DispatchResult> {
|
|
assert_eq!(get_transaction_level(), 1);
|
|
TransactionOutcome::Commit(Ok(()))
|
|
}));
|
|
|
|
assert_ok!(with_transaction(|| -> TransactionOutcome<DispatchResult> {
|
|
assert_eq!(get_transaction_level(), 1);
|
|
let res = with_transaction(|| -> TransactionOutcome<DispatchResult> {
|
|
assert_eq!(get_transaction_level(), 2);
|
|
TransactionOutcome::Commit(Ok(()))
|
|
});
|
|
TransactionOutcome::Commit(res)
|
|
}));
|
|
|
|
assert_ok!(recursive_transactional(255));
|
|
assert_noop!(
|
|
recursive_transactional(256),
|
|
sp_runtime::TransactionalError::LimitReached
|
|
);
|
|
|
|
assert_eq!(get_transaction_level(), 0);
|
|
});
|
|
}
|
|
|
|
#[test]
|
|
fn in_storage_layer_works() {
|
|
TestExternalities::default().execute_with(|| {
|
|
assert_eq!(get_transaction_level(), 0);
|
|
|
|
let res = in_storage_layer(|| -> DispatchResult {
|
|
assert_eq!(get_transaction_level(), 1);
|
|
in_storage_layer(|| -> DispatchResult {
|
|
// We are still in the same layer :)
|
|
assert_eq!(get_transaction_level(), 1);
|
|
Ok(())
|
|
})
|
|
});
|
|
|
|
assert_ok!(res);
|
|
|
|
let res = in_storage_layer(|| -> DispatchResult {
|
|
assert_eq!(get_transaction_level(), 1);
|
|
in_storage_layer(|| -> DispatchResult {
|
|
// We are still in the same layer :)
|
|
assert_eq!(get_transaction_level(), 1);
|
|
Err("epic fail".into())
|
|
})
|
|
});
|
|
|
|
assert_noop!(res, "epic fail");
|
|
});
|
|
}
|
|
}
|