mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-04-26 07:37:57 +00:00
82f3c3e2e81d3ea4b9bc35a14432d3ab93fb5b52
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oliver Tale-Yazdi
|
eefd5fe449 |
Multi-Block-Migrations, poll hook and new System callbacks (#1781)
This MR is the merge of https://github.com/paritytech/substrate/pull/14414 and https://github.com/paritytech/substrate/pull/14275. It implements [RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes https://github.com/paritytech/polkadot-sdk/issues/198. ----- This Merge request introduces three major topicals: 1. Multi-Block-Migrations 1. New pallet `poll` hook for periodic service work 1. Replacement hooks for `on_initialize` and `on_finalize` in cases where `poll` cannot be used and some more general changes to FRAME. The changes for each topical span over multiple crates. They are listed in topical order below. # 1.) Multi-Block-Migrations Multi-Block-Migrations are facilitated by creating `pallet_migrations` and configuring `System::Config::MultiBlockMigrator` to point to it. Executive picks this up and triggers one step of the migrations pallet per block. The chain is in lockdown mode for as long as an MBM is ongoing. Executive does this by polling `MultiBlockMigrator::ongoing` and not allowing any transaction in a block, if true. A MBM is defined through trait `SteppedMigration`. A condensed version looks like this: ```rust /// A migration that can proceed in multiple steps. pub trait SteppedMigration { type Cursor: FullCodec + MaxEncodedLen; type Identifier: FullCodec + MaxEncodedLen; fn id() -> Self::Identifier; fn max_steps() -> Option<u32>; fn step( cursor: Option<Self::Cursor>, meter: &mut WeightMeter, ) -> Result<Option<Self::Cursor>, SteppedMigrationError>; } ``` `pallet_migrations` can be configured with an aggregated tuple of these migrations. It then starts to migrate them one-by-one on the next runtime upgrade. Two things are important here: - 1. Doing another runtime upgrade while MBMs are ongoing is not a good idea and can lead to messed up state. - 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`, otherwise it is not used.** The pallet supports an `UpgradeStatusHandler` that can be used to notify external logic of upgrade start/finish (for example to pause XCM dispatch). Error recovery is very limited in the case that a migration errors or times out (exceeds its `max_steps`). Currently the runtime dev can decide in `FailedMigrationHandler::failed` how to handle this. One follow-up would be to pair this with the `SafeMode` pallet and enact safe mode when an upgrade fails, to allow governance to rescue the chain. This is currently not possible, since governance is not `Mandatory`. ## Runtime API - `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to inform the Block Author whether they can push transactions. ### Integration Add it to your runtime implementation of `Core` and `BlockBuilder`: ```patch diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs @@ impl_runtime_apis! { impl sp_block_builder::Core<Block> for Runtime { - fn initialize_block(header: &<Block as BlockT>::Header) { + fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode { Executive::initialize_block(header) } ... } ``` # 2.) `poll` hook A new pallet hook is introduced: `poll`. `Poll` is intended to replace mostly all usage of `on_initialize`. The reason for this is that any code that can be called from `on_initialize` cannot be migrated through an MBM. Currently there is no way to statically check this; the implication is to use `on_initialize` as rarely as possible. Failing to do so can result in broken storage invariants. The implementation of the poll hook depends on the `Runtime API` changes that are explained above. # 3.) Hard-Deadline callbacks Three new callbacks are introduced and configured on `System::Config`: `PreInherents`, `PostInherents` and `PostTransactions`. These hooks are meant as replacement for `on_initialize` and `on_finalize` in cases where the code that runs cannot be moved to `poll`. The reason for this is to make the usage of HD-code (hard deadline) more explicit - again to prevent broken invariants by MBMs. # 4.) FRAME (general changes) ## `frame_system` pallet A new memorize storage item `InherentsApplied` is added. It is used by executive to track whether inherents have already been applied. Executive and can then execute the MBMs directly between inherents and transactions. The `Config` gets five new items: - `SingleBlockMigrations` this is the new way of configuring migrations that run in a single block. Previously they were defined as last generic argument of `Executive`. This shift is brings all central configuration about migrations closer into view of the developer (migrations that are configured in `Executive` will still work for now but is deprecated). - `MultiBlockMigrator` this can be configured to an engine that drives MBMs. One example would be the `pallet_migrations`. Note that this is only the engine; the exact MBMs are injected into the engine. - `PreInherents` a callback that executes after `on_initialize` but before inherents. - `PostInherents` a callback that executes after all inherents ran (including MBMs and `poll`). - `PostTransactions` in symmetry to `PreInherents`, this one is called before `on_finalize` but after all transactions. A sane default is to set all of these to `()`. Example diff suitable for any chain: ```patch @@ impl frame_system::Config for Test { type MaxConsumers = ConstU32<16>; + type SingleBlockMigrations = (); + type MultiBlockMigrator = (); + type PreInherents = (); + type PostInherents = (); + type PostTransactions = (); } ``` An overview of how the block execution now looks like is here. The same graph is also in the rust doc. <details><summary>Block Execution Flow</summary> <p>  </p> </details> ## Inherent Order Moved to https://github.com/paritytech/polkadot-sdk/pull/2154 --------------- ## TODO - [ ] Check that `try-runtime` still works - [ ] Ensure backwards compatibility with old Runtime APIs - [x] Consume weight correctly - [x] Cleanup --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Liam Aharon <liam.aharon@hotmail.com> Co-authored-by: Juan Girini <juangirini@gmail.com> Co-authored-by: command-bot <> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> |
||
|
gupnik
|
cd8741c8b5 |
Moves all test runtimes to use derive_impl (#2409)
Step in https://github.com/paritytech/polkadot-sdk/issues/171 This PR adds `derive_impl` on all `frame_system` config impls for mock runtimes. The overridden configs are maintained as of now to ensure minimal changes. --------- Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> |
||
|
Juan
|
878c562cd4 |
Restructure frame_support macro related exports (#14745)
* make reexports private * make reexports private 2 * make reexports private for runtime-benchmarking * make reexports private for try-runtime * fix for try-runtime * make reexports private for tests * fmt * make reexports private for tests * make reexports private for experimental * fix beefy * fix ui test * fix ui test * fix benches * ".git/.scripts/commands/fmt/fmt.sh" * fix contracts use * wip * wip * do not reexport sp_api::metadata_ir * fix CI checks * fix support tests * ".git/.scripts/commands/fmt/fmt.sh" * Update frame/support/src/lib.rs Co-authored-by: Bastian Köcher <git@kchr.de> * import codec directly * fmt * fix node-cli tests --------- Co-authored-by: command-bot <> Co-authored-by: Bastian Köcher <git@kchr.de> |
||
|
Juan
|
6a29a70a92 |
Replace system config Index for Nonce (#14290)
* replace Index by Nonce * replace Index by Nonce * replace Index by Nonce * replace Index by Nonce * replace Index by Nonce * wip * remove index in lieu of nonce * wip * remove accountnonce in lieu of nonce * add minor improvement * rebase and merge conflicts |
||
|
gupnik
|
5e7b27e98c |
Moves Block to frame_system instead of construct_runtime and removes Header and BlockNumber (#14437)
* Initial setup * Adds node block * Uses UncheckedExtrinsic and removes Where section * Updates frame_system to use Block * Adds deprecation warning * Fixes pallet-timestamp * Removes Header and BlockNumber * Addresses review comments * Addresses review comments * Adds comment about compiler bug * Removes where clause * Refactors code * Fixes errors in cargo check * Fixes errors in cargo check * Fixes warnings in cargo check * Formatting * Fixes construct_runtime tests * Uses import instead of full path for BlockNumber * Uses import instead of full path for Header * Formatting * Fixes construct_runtime tests * Fixes imports in benchmarks * Formatting * Fixes construct_runtime tests * Formatting * Minor updates * Fixes construct_runtime ui tests * Fixes construct_runtime ui tests with 1.70 * Fixes docs * Fixes docs * Adds u128 mock block type * Fixes split example * fixes for cumulus * ".git/.scripts/commands/fmt/fmt.sh" * Updates new tests * Fixes fully-qualified path in few places * Formatting * Update frame/examples/default-config/src/lib.rs Co-authored-by: Juan <juangirini@gmail.com> * Update frame/support/procedural/src/construct_runtime/mod.rs Co-authored-by: Juan <juangirini@gmail.com> * ".git/.scripts/commands/fmt/fmt.sh" * Addresses some review comments * Fixes build * ".git/.scripts/commands/fmt/fmt.sh" * Update frame/democracy/src/lib.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/democracy/src/lib.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/support/procedural/src/construct_runtime/mod.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Update frame/support/procedural/src/construct_runtime/mod.rs Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Addresses review comments * Updates trait bounds * Minor fix * ".git/.scripts/commands/fmt/fmt.sh" * Removes unnecessary bound * ".git/.scripts/commands/fmt/fmt.sh" * Updates test * Fixes build * Adds a bound for header * ".git/.scripts/commands/fmt/fmt.sh" * Removes where block * Minor fix * Minor fix * Fixes tests * ".git/.scripts/commands/update-ui/update-ui.sh" 1.70 * Updates test * Update primitives/runtime/src/traits.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/runtime/src/traits.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Updates doc * Updates doc --------- Co-authored-by: command-bot <> Co-authored-by: Juan <juangirini@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> |
||
|
Bastian Köcher
|
e2547f5064 |
Improve handling of unset StorageVersion (#13417)
* Improve handling of unset `StorageVersion` When a user is forgetting to set the storage version in a pallet and calls `current_storage_version` to compare it against the `on_chain_storage_version` it will now fail to compile the code. Before the pallet macro just returned `StorageVersion::default()` for `current_storage_version` leading to potential issues with migrations. Besides that it also checks in `post_upgrade` that the pallet storage version was upgraded and thus, no migration was missed. * Use correct `Cargo.lock` * Fixes * Fix test * Update frame/support/test/tests/pallet.rs * Ensure we don't set a storage version when the pallet is missing the attribute * Fix merge conflict * Update frame/support/procedural/src/pallet/expand/hooks.rs Co-authored-by: Roman Useinov <roman.useinov@gmail.com> * Update frame/support/procedural/src/pallet/expand/hooks.rs Co-authored-by: Roman Useinov <roman.useinov@gmail.com> * Fix compilation * Do not run everything with `try-runtime` * Fix test * Apply suggestions from code review Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> * Fix `no-metadata-docs` --------- Co-authored-by: Roman Useinov <roman.useinov@gmail.com> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: parity-processbot <> |
||
|
Alexandru Vasile
|
29ea9e0ad9 |
Metadata V15: Add Runtime API metadata (#13302)
* impl_runtime_apis: Generate getters for `metadata_at` functions Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime: Implement new `Metadata` runtime trait Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime: Move `metadata_at` functions to construct_runtime macro Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * contruct_runtime: Use `OpaqueMetadata` from hidden imports Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Add tests for the new API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Helper to extract documentation literals Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Helper to filter all `cfg` attributes Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Generate documentation getters for metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Avoid trait collision with snake case methods Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * proc-macro/tests: Check doc getters Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Generate metadata for runtime methods Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/api: Export scale-info and frame-metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Generate metadata for runtime traits Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/runtime: Expose metadata v15 internally Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * test: Use metadata v15 from `lexnv/md_v15_test` branch Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Generate crate access one module up Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame: Implement `runtime_metadata` for mocks and tests Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Fix warnings Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Add no-docs flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame: Adjust more tests Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Check runtime metadata correctness Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/benchmarking: Adjust benchmarks Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/benchmarks: Adjust more benchmarks Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/api: Fix clippy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/proc-macro: Generate runtime metadata on the `decl_runtime_apis` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame: Abuse Deref to resolve `runtime_metadata` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert "frame: Implement `runtime_metadata` for mocks and tests" This reverts commit b7f41aa189218589392a6e713ea9488e93c4db45. Revert "frame: Adjust more tests" This reverts commit 3cba5982c7f45552e76335e96c430aecbc42d8c6. Revert "frame/benchmarking: Adjust benchmarks" This reverts commit 60b382ada486c791ffceeb65da587e949b90ec5d. Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Revert "frame/benchmarks: Adjust more benchmarks" This reverts commit eb75c477179b1a27347a5554c5732ef26a00d7e8. * primitives/proc-macro: Remove unused imports and function Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Adjust runtime metadata test Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/tests: Remove doc getter test Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/support: Enable `no-metadata-docs` feature from `sp-api` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/tests: Add `TypeInfo` for test::extrinsic Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/api: Expose scale-info and frame-metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update frame-metadata to include v15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Fix merge conflicts Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata_ir: Add IR for runtime API metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/metadata_ir: Convert IR to V15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/api: Collect IR metadata for runtime API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/api: Move `metadata_ir` from frame/support Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame/tests: Adjust `metadata_versions` test Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/runtime_metadata: Exclude default type parameters from methods Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/src/metadata_ir/types.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/src/metadata_ir/mod.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/utils.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * primitives: Fix build Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives/metadata-ir: Move IR to dedicated crate Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives: Reexport metadata-ir and frame-metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * frame: Use apis field instead of runtime Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Better documentation for the `Deref` abstraction Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * ui-tests: Check empty `impl_runtime_apis` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives: Remove unneeded bounds on generic params Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives: Rename `collect_where_bounds` to `get_argument_type_param` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives: Generate crate access per fn call Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert "primitives: Remove unneeded bounds on generic params" This reverts commit 5178e38cf21cfb481156eefd628d62989201d59a. * metadata-ir: Add no-std Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * primitives: Adjust where bounds Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Change `frame-metadata` branch to "origin/main" Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update to `main` from origin Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update frame-metadata to crates.io v15.1 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert "ui-tests: Check empty `impl_runtime_apis`" This reverts commit cf78a7190ad9cba3c3bb2e78dc3d0dc382b2fea9. * Move ui test to primitives/ui Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update frame/support/test/tests/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Update primitives/api/proc-macro/src/runtime_metadata.rs Co-authored-by: Bastian Köcher <git@kchr.de> * Test already covered by `empty_impl_runtime_apis_call.stderr` This reverts commit 3bafb294cbe9745569bf5e5a1a2e6b4a4c1aadc5. * Retriger CI Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Import `TokenStream` as `TokenStream2` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: parity-processbot <> Co-authored-by: Bastian Köcher <git@kchr.de> |