substrate-test-runtime migrated to "pure" frame runtime (#13737)

* substrate-test-runtime migrated to pure-frame based

* test block builder: helpers added

* simple renaming

* basic_authorship test adjusted

* block_building storage_proof test adjusted

* babe: tests: should_panic expected added

* babe: tests adjusted

ConsensusLog::NextEpochData is now added by pallet_babe as
pallet_babe::SameAuthoritiesForever trigger is used in runtime config.

* beefy: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digests stored during block execution are
checked against header digests:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(MmrRoot / AuthoritiesChange) is used.

* grandpa: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digest logs stored during block execution are
checked against header digest logs:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(ScheduledChange / ForcedChange and DigestItem::Other) is used.

* network:bitswap: test adjusted

The size of unchecked extrinsic was increased. The pattern used in test will
be placed at the end of scale-encoded buffer.

* runtime apis versions adjusted

* storage keys used in runtime adjusted

* wasm vs native tests removed

* rpc tests: adjusted

Transfer transaction processing was slightly improved, test was
adjusted.

* tests: sizes adjusted

Runtime extrinsic size was increased. Size of data read during block
execution was also increased due to usage of new pallets in runtime.

Sizes were adjusted in tests.

* cargo.lock update

cargo update -p substrate-test-runtime -p substrate-test-runtime-client

* warnings fixed

* builders cleanup: includes / std

* extrinsic validation cleanup

* txpool: benches performance fixed

* fmt

* spelling

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Apply code review suggestions

* Apply code review suggestions

* get rid of 1063 const

* renaming: UncheckedExtrinsic -> Extrinsic

* test-utils-runtime: further step to pure-frame

* basic-authorship: tests OK

* CheckSubstrateCall added + tests fixes

* test::Transfer call removed

* priority / propagate / no sudo+root-testing

* fixing warnings + format

* cleanup: build2/nonce + format

* final tests fixes

all tests are passing

* logs/comments removal

* should_not_accept_old_signatures test removed

* make txpool benches work again

* Cargo.lock reset

* format

* sudo hack removed

* txpool benches fix+cleanup

* .gitignore reverted

* rebase fixing + unsigned cleanup

* Cargo.toml/Cargo.lock cleanup

* force-debug feature removed

* mmr tests fixed

* make cargo-clippy happy

* network sync test uses unsigned extrinsic

* cleanup

* ".git/.scripts/commands/fmt/fmt.sh"

* push_storage_change signed call remove

* GenesisConfig cleanup

* fix

* fix

* GenesisConfig simplified

* storage_keys_works: reworked

* storage_keys_works: expected keys in vec

* storage keys list moved to substrate-test-runtime

* substrate-test: some sanity tests + GenesisConfigBuilder rework

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

* Review suggestions

* fix

* fix

* beefy: generate_blocks_and_sync block_num sync with actaul value

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* cargo update -p sc-rpc -p sc-transaction-pool

* Review suggestions

* fix

* doc added

* slot_duration adjusted for Babe::slot_duration

* small doc fixes

* array_bytes::hex used instead of hex

* tiny -> medium name fix

* Apply suggestions from code review

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* TransferData::try_from_unchecked_extrinsic -> try_from

* Update Cargo.lock

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
This commit is contained in:
Michal Kucharczyk
2023-05-04 18:20:22 +02:00
committed by GitHub
parent 3a90728de0
commit 6a295e7c28
34 changed files with 1943 additions and 2356 deletions
@@ -0,0 +1,207 @@
// This file is part of Substrate.
// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Provides utils for building the `Extrinsic` instances used with `substrate-test-runtime`.
use crate::{
substrate_test_pallet::pallet::Call as PalletCall, AccountId, Balance, BalancesCall,
CheckSubstrateCall, Extrinsic, Index, Pair, RuntimeCall, SignedPayload, TransferData,
};
use codec::Encode;
use frame_system::{CheckNonce, CheckWeight};
use sp_core::crypto::Pair as TraitPair;
use sp_keyring::AccountKeyring;
use sp_runtime::{transaction_validity::TransactionPriority, Perbill};
use sp_std::prelude::*;
/// Transfer used in test substrate pallet. Extrinsic is created and signed using this data.
#[derive(Clone)]
pub struct Transfer {
/// Transfer sender and signer of created extrinsic
pub from: Pair,
/// The recipient of the transfer
pub to: AccountId,
/// Amount of transfer
pub amount: Balance,
/// Sender's account nonce at which transfer is valid
pub nonce: u64,
}
impl Transfer {
/// Convert into a signed unchecked extrinsic.
pub fn into_unchecked_extrinsic(self) -> Extrinsic {
ExtrinsicBuilder::new_transfer(self).build()
}
}
impl Default for TransferData {
fn default() -> Self {
Self {
from: AccountKeyring::Alice.into(),
to: AccountKeyring::Bob.into(),
amount: 0,
nonce: 0,
}
}
}
/// If feasible converts given `Extrinsic` to `TransferData`
impl TryFrom<&Extrinsic> for TransferData {
type Error = ();
fn try_from(uxt: &Extrinsic) -> Result<Self, Self::Error> {
match uxt {
Extrinsic {
function: RuntimeCall::Balances(BalancesCall::transfer_allow_death { dest, value }),
signature: Some((from, _, (CheckNonce(nonce), ..))),
} => Ok(TransferData { from: *from, to: *dest, amount: *value, nonce: *nonce }),
Extrinsic {
function: RuntimeCall::SubstrateTest(PalletCall::bench_call { transfer }),
signature: None,
} => Ok(transfer.clone()),
_ => Err(()),
}
}
}
/// Generates `Extrinsic`
pub struct ExtrinsicBuilder {
function: RuntimeCall,
signer: Option<Pair>,
nonce: Option<Index>,
}
impl ExtrinsicBuilder {
/// Create builder for given `RuntimeCall`. By default `Extrinsic` will be signed by `Alice`.
pub fn new(function: impl Into<RuntimeCall>) -> Self {
Self { function: function.into(), signer: Some(AccountKeyring::Alice.pair()), nonce: None }
}
/// Create builder for given `RuntimeCall`. `Extrinsic` will be unsigned.
pub fn new_unsigned(function: impl Into<RuntimeCall>) -> Self {
Self { function: function.into(), signer: None, nonce: None }
}
/// Create builder for `pallet_call::bench_transfer` from given `TransferData`.
pub fn new_bench_call(transfer: TransferData) -> Self {
Self::new_unsigned(PalletCall::bench_call { transfer })
}
/// Create builder for given `Transfer`. Transfer `nonce` will be used as `Extrinsic` nonce.
/// Transfer `from` will be used as Extrinsic signer.
pub fn new_transfer(transfer: Transfer) -> Self {
Self {
nonce: Some(transfer.nonce),
signer: Some(transfer.from.clone()),
..Self::new(BalancesCall::transfer_allow_death {
dest: transfer.to,
value: transfer.amount,
})
}
}
/// Create builder for `PalletCall::include_data` call using given parameters
pub fn new_include_data(data: Vec<u8>) -> Self {
Self::new(PalletCall::include_data { data })
}
/// Create builder for `PalletCall::storage_change` call using given parameters. Will
/// create unsigned Extrinsic.
pub fn new_storage_change(key: Vec<u8>, value: Option<Vec<u8>>) -> Self {
Self::new_unsigned(PalletCall::storage_change { key, value })
}
/// Create builder for `PalletCall::offchain_index_set` call using given parameters
pub fn new_offchain_index_set(key: Vec<u8>, value: Vec<u8>) -> Self {
Self::new(PalletCall::offchain_index_set { key, value })
}
/// Create builder for `PalletCall::offchain_index_clear` call using given parameters
pub fn new_offchain_index_clear(key: Vec<u8>) -> Self {
Self::new(PalletCall::offchain_index_clear { key })
}
/// Create builder for `PalletCall::indexed_call` call using given parameters
pub fn new_indexed_call(data: Vec<u8>) -> Self {
Self::new(PalletCall::indexed_call { data })
}
/// Create builder for `PalletCall::new_deposit_log_digest_item` call using given `log`
pub fn new_deposit_log_digest_item(log: sp_runtime::generic::DigestItem) -> Self {
Self::new_unsigned(PalletCall::deposit_log_digest_item { log })
}
/// Create builder for `PalletCall::Call::new_deposit_log_digest_item`
pub fn new_fill_block(ratio: Perbill) -> Self {
Self::new(PalletCall::fill_block { ratio })
}
/// Create builder for `PalletCall::call_do_not_propagate` call using given parameters
pub fn new_call_do_not_propagate() -> Self {
Self::new(PalletCall::call_do_not_propagate {})
}
/// Create builder for `PalletCall::call_with_priority` call using given parameters
pub fn new_call_with_priority(priority: TransactionPriority) -> Self {
Self::new(PalletCall::call_with_priority { priority })
}
/// Create builder for `PalletCall::read` call using given parameters
pub fn new_read(count: u32) -> Self {
Self::new_unsigned(PalletCall::read { count })
}
/// Create builder for `PalletCall::read` call using given parameters
pub fn new_read_and_panic(count: u32) -> Self {
Self::new_unsigned(PalletCall::read_and_panic { count })
}
/// Unsigned `Extrinsic` will be created
pub fn unsigned(mut self) -> Self {
self.signer = None;
self
}
/// Given `nonce` will be set in `Extrinsic`
pub fn nonce(mut self, nonce: Index) -> Self {
self.nonce = Some(nonce);
self
}
/// Extrinsic will be signed by `signer`
pub fn signer(mut self, signer: Pair) -> Self {
self.signer = Some(signer);
self
}
/// Build `Extrinsic` using embedded parameters
pub fn build(self) -> Extrinsic {
if let Some(signer) = self.signer {
let extra = (
CheckNonce::from(self.nonce.unwrap_or(0)),
CheckWeight::new(),
CheckSubstrateCall {},
);
let raw_payload =
SignedPayload::from_raw(self.function.clone(), extra.clone(), ((), (), ()));
let signature = raw_payload.using_encoded(|e| signer.sign(e));
Extrinsic::new_signed(self.function, signer.public(), signature, extra)
} else {
Extrinsic::new_unsigned(self.function)
}
}
}
+92 -54
View File
@@ -17,75 +17,119 @@
//! Tool for creating the genesis block.
use super::{system, wasm_binary_unwrap, AccountId, AuthorityId, Runtime};
use codec::{Encode, Joiner, KeyedVec};
use frame_support::traits::GenesisBuild;
use super::{
currency, substrate_test_pallet, wasm_binary_unwrap, AccountId, AuthorityId, Balance,
GenesisConfig,
};
use codec::Encode;
use sc_service::construct_genesis_block;
use sp_core::{
map,
sr25519,
storage::{well_known_keys, StateVersion, Storage},
Pair,
};
use sp_keyring::{AccountKeyring, Sr25519Keyring};
use sp_runtime::{
traits::{Block as BlockT, Hash as HashT, Header as HeaderT},
BuildStorage,
};
use sp_io::hashing::{blake2_256, twox_128};
use sp_runtime::traits::{Block as BlockT, Hash as HashT, Header as HeaderT};
use std::collections::BTreeMap;
/// Configuration of a general Substrate test genesis block.
pub struct GenesisConfig {
/// Builder for generating storage from substrate-test-runtime genesis config. Default storage can
/// be extended with additional key-value pairs.
pub struct GenesisStorageBuilder {
authorities: Vec<AuthorityId>,
balances: Vec<(AccountId, u64)>,
heap_pages_override: Option<u64>,
/// Additional storage key pairs that will be added to the genesis map.
extra_storage: Storage,
wasm_code: Option<Vec<u8>>,
}
impl GenesisConfig {
impl Default for GenesisStorageBuilder {
/// Creates a builder with default settings for `substrate_test_runtime`.
fn default() -> Self {
Self::new(
vec![
sr25519::Public::from(Sr25519Keyring::Alice).into(),
sr25519::Public::from(Sr25519Keyring::Bob).into(),
sr25519::Public::from(Sr25519Keyring::Charlie).into(),
],
(0..16_usize)
.into_iter()
.map(|i| AccountKeyring::numeric(i).public())
.chain(vec![
AccountKeyring::Alice.into(),
AccountKeyring::Bob.into(),
AccountKeyring::Charlie.into(),
])
.collect(),
1000 * currency::DOLLARS,
)
}
}
impl GenesisStorageBuilder {
/// Creates a storage builder for genesis config. `substrage test runtime` `GenesisConfig` is
/// initialized with provided `authorities`, `endowed_accounts` with given balance. Key-pairs
/// from `extra_storage` will be injected into built storage. `HEAP_PAGES` key and value will
/// also be placed into storage.
pub fn new(
authorities: Vec<AuthorityId>,
endowed_accounts: Vec<AccountId>,
balance: u64,
heap_pages_override: Option<u64>,
extra_storage: Storage,
balance: Balance,
) -> Self {
GenesisConfig {
GenesisStorageBuilder {
authorities,
balances: endowed_accounts.into_iter().map(|a| (a, balance)).collect(),
heap_pages_override,
extra_storage,
heap_pages_override: None,
extra_storage: Default::default(),
wasm_code: None,
}
}
pub fn genesis_map(&self) -> Storage {
let wasm_runtime = wasm_binary_unwrap().to_vec();
let mut map: BTreeMap<Vec<u8>, Vec<u8>> = self
.balances
.iter()
.map(|&(ref account, balance)| {
(account.to_keyed_vec(b"balance:"), vec![].and(&balance))
})
.map(|(k, v)| (blake2_256(&k[..])[..].to_vec(), v.to_vec()))
.chain(
vec![
(well_known_keys::CODE.into(), wasm_runtime),
(
well_known_keys::HEAP_PAGES.into(),
vec![].and(&(self.heap_pages_override.unwrap_or(16_u64))),
),
]
.into_iter(),
)
.collect();
map.insert(twox_128(&b"sys:auth"[..])[..].to_vec(), self.authorities.encode());
// Add the extra storage entries.
map.extend(self.extra_storage.top.clone().into_iter());
/// Override default wasm code to be placed into GenesisConfig.
pub fn with_wasm_code(mut self, wasm_code: &Option<Vec<u8>>) -> Self {
self.wasm_code = wasm_code.clone();
self
}
// Assimilate the system genesis config.
let mut storage =
Storage { top: map, children_default: self.extra_storage.children_default.clone() };
<system::GenesisConfig as GenesisBuild<Runtime>>::assimilate_storage(
&system::GenesisConfig { authorities: self.authorities.clone() },
&mut storage,
)
.expect("Adding `system::GensisConfig` to the genesis");
pub fn with_heap_pages(mut self, heap_pages_override: Option<u64>) -> Self {
self.heap_pages_override = heap_pages_override;
self
}
pub fn with_extra_storage(mut self, storage: Storage) -> Self {
self.extra_storage = storage;
self
}
/// Builds the `GenesisConfig` and returns its storage.
pub fn build_storage(&mut self) -> Storage {
let genesis_config = GenesisConfig {
system: frame_system::GenesisConfig {
code: self.wasm_code.clone().unwrap_or(wasm_binary_unwrap().to_vec()),
},
babe: pallet_babe::GenesisConfig {
authorities: self.authorities.clone().into_iter().map(|x| (x, 1)).collect(),
epoch_config: Some(crate::TEST_RUNTIME_BABE_EPOCH_CONFIGURATION),
},
substrate_test: substrate_test_pallet::GenesisConfig {
authorities: self.authorities.clone(),
},
balances: pallet_balances::GenesisConfig { balances: self.balances.clone() },
};
let mut storage = genesis_config
.build_storage()
.expect("Build storage from substrate-test-runtime GenesisConfig");
storage.top.insert(
well_known_keys::HEAP_PAGES.into(),
self.heap_pages_override.unwrap_or(16_u64).encode(),
);
storage.top.extend(self.extra_storage.top.clone());
storage.children_default.extend(self.extra_storage.children_default.clone());
storage
}
@@ -108,12 +152,6 @@ pub fn insert_genesis_block(storage: &mut Storage) -> sp_core::hash::H256 {
);
let block: crate::Block = construct_genesis_block(state_root, StateVersion::V1);
let genesis_hash = block.header.hash();
storage.top.extend(additional_storage_with_genesis(&block));
genesis_hash
}
pub fn additional_storage_with_genesis(genesis_block: &crate::Block) -> BTreeMap<Vec<u8>, Vec<u8>> {
map![
twox_128(&b"latest"[..]).to_vec() => genesis_block.hash().as_fixed_bytes().to_vec()
]
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,244 @@
// This file is part of Substrate.
// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! # substrate-test pallet
//!
//! Provides functionality used in unit-tests of numerous modules across substrate that require
//! functioning runtime. Some calls are allowed to be submitted as unsigned extrinsics, however most
//! of them requires signing. Refer to `pallet::Call` for further details.
use crate::AuthorityId;
use frame_support::{pallet_prelude::*, storage};
use sp_runtime::transaction_validity::{
InvalidTransaction, TransactionSource, TransactionValidity, ValidTransaction,
};
use sp_std::prelude::*;
pub use self::pallet::*;
const LOG_TARGET: &str = "substrate_test_pallet";
#[frame_support::pallet(dev_mode)]
pub mod pallet {
use super::*;
use crate::TransferData;
use frame_system::pallet_prelude::*;
use sp_core::storage::well_known_keys;
use sp_runtime::{transaction_validity::TransactionPriority, Perbill};
#[pallet::pallet]
#[pallet::without_storage_info]
pub struct Pallet<T>(PhantomData<T>);
#[pallet::config]
pub trait Config: frame_system::Config {}
#[pallet::storage]
#[pallet::getter(fn authorities)]
pub type Authorities<T> = StorageValue<_, Vec<AuthorityId>, ValueQuery>;
#[pallet::genesis_config]
#[cfg_attr(feature = "std", derive(Default))]
pub struct GenesisConfig {
pub authorities: Vec<AuthorityId>,
}
#[pallet::genesis_build]
impl<T: Config> GenesisBuild<T> for GenesisConfig {
fn build(&self) {
<Authorities<T>>::put(self.authorities.clone());
}
}
#[pallet::call]
impl<T: Config> Pallet<T> {
/// Legacy call used in transaction pool benchmarks.
#[pallet::call_index(0)]
#[pallet::weight(100)]
pub fn bench_call(_origin: OriginFor<T>, _transfer: TransferData) -> DispatchResult {
Ok(())
}
/// Implicitly fill a block body with some data.
#[pallet::call_index(1)]
#[pallet::weight(100)]
pub fn include_data(origin: OriginFor<T>, _data: Vec<u8>) -> DispatchResult {
frame_system::ensure_signed(origin)?;
Ok(())
}
/// Put/delete some data from storage. Intended to use as an unsigned extrinsic.
#[pallet::call_index(2)]
#[pallet::weight(100)]
pub fn storage_change(
_origin: OriginFor<T>,
key: Vec<u8>,
value: Option<Vec<u8>>,
) -> DispatchResult {
match value {
Some(value) => storage::unhashed::put_raw(&key, &value),
None => storage::unhashed::kill(&key),
}
Ok(())
}
/// Write a key value pair to the offchain database.
#[pallet::call_index(3)]
#[pallet::weight(100)]
pub fn offchain_index_set(
origin: OriginFor<T>,
key: Vec<u8>,
value: Vec<u8>,
) -> DispatchResult {
frame_system::ensure_signed(origin)?;
sp_io::offchain_index::set(&key, &value);
Ok(())
}
/// Remove a key and an associated value from the offchain database.
#[pallet::call_index(4)]
#[pallet::weight(100)]
pub fn offchain_index_clear(origin: OriginFor<T>, key: Vec<u8>) -> DispatchResult {
frame_system::ensure_signed(origin)?;
sp_io::offchain_index::clear(&key);
Ok(())
}
/// Create an index for this call.
#[pallet::call_index(5)]
#[pallet::weight(100)]
pub fn indexed_call(origin: OriginFor<T>, data: Vec<u8>) -> DispatchResult {
frame_system::ensure_signed(origin)?;
let content_hash = sp_io::hashing::blake2_256(&data);
let extrinsic_index: u32 =
storage::unhashed::get(well_known_keys::EXTRINSIC_INDEX).unwrap();
sp_io::transaction_index::index(extrinsic_index, data.len() as u32, content_hash);
Ok(())
}
/// Deposit given digest items into the system storage. They will be included in a header
/// during finalization.
#[pallet::call_index(6)]
#[pallet::weight(100)]
pub fn deposit_log_digest_item(
_origin: OriginFor<T>,
log: sp_runtime::generic::DigestItem,
) -> DispatchResult {
<frame_system::Pallet<T>>::deposit_log(log);
Ok(())
}
/// This call is validated as `ValidTransaction` with given priority.
#[pallet::call_index(7)]
#[pallet::weight(100)]
pub fn call_with_priority(
_origin: OriginFor<T>,
_priority: TransactionPriority,
) -> DispatchResult {
Ok(())
}
/// This call is validated as non-propagable `ValidTransaction`.
#[pallet::call_index(8)]
#[pallet::weight(100)]
pub fn call_do_not_propagate(_origin: OriginFor<T>) -> DispatchResult {
Ok(())
}
/// Fill the block weight up to the given ratio.
#[pallet::call_index(9)]
#[pallet::weight(*_ratio * T::BlockWeights::get().max_block)]
pub fn fill_block(origin: OriginFor<T>, _ratio: Perbill) -> DispatchResult {
ensure_signed(origin)?;
Ok(())
}
/// Read X times from the state some data.
///
/// Panics if it can not read `X` times.
#[pallet::call_index(10)]
#[pallet::weight(100)]
pub fn read(_origin: OriginFor<T>, count: u32) -> DispatchResult {
Self::execute_read(count, false)
}
/// Read X times from the state some data and then panic!
///
/// Returns `Ok` if it didn't read anything.
#[pallet::call_index(11)]
#[pallet::weight(100)]
pub fn read_and_panic(_origin: OriginFor<T>, count: u32) -> DispatchResult {
Self::execute_read(count, true)
}
}
impl<T: Config> Pallet<T> {
fn execute_read(read: u32, panic_at_end: bool) -> DispatchResult {
let mut next_key = vec![];
for _ in 0..(read as usize) {
if let Some(next) = sp_io::storage::next_key(&next_key) {
// Read the value
sp_io::storage::get(&next);
next_key = next;
} else {
if panic_at_end {
return Ok(())
} else {
panic!("Could not read {read} times from the state");
}
}
}
if panic_at_end {
panic!("BYE")
} else {
Ok(())
}
}
}
#[pallet::validate_unsigned]
impl<T: Config> ValidateUnsigned for Pallet<T> {
type Call = Call<T>;
fn validate_unsigned(_source: TransactionSource, call: &Self::Call) -> TransactionValidity {
log::trace!(target: LOG_TARGET, "validate_unsigned {call:?}");
match call {
// Some tests do not need to be complicated with signer and nonce, some need
// reproducible block hash (call signature can't be there).
// Offchain testing requires storage_change.
Call::deposit_log_digest_item { .. } |
Call::storage_change { .. } |
Call::read { .. } |
Call::read_and_panic { .. } => Ok(Default::default()),
_ => Err(TransactionValidityError::Invalid(InvalidTransaction::Call)),
}
}
}
}
pub fn validate_runtime_call<T: pallet::Config>(call: &pallet::Call<T>) -> TransactionValidity {
log::trace!(target: LOG_TARGET, "validate_runtime_call {call:?}");
match call {
Call::call_do_not_propagate {} =>
Ok(ValidTransaction { propagate: false, ..Default::default() }),
Call::call_with_priority { priority } =>
Ok(ValidTransaction { priority: *priority, ..Default::default() }),
_ => Ok(Default::default()),
}
}
-587
View File
@@ -1,587 +0,0 @@
// This file is part of Substrate.
// Copyright (C) Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! System manager: Handles all of the top-level stuff; executing block/transaction, setting code
//! and depositing logs.
use crate::{
AccountId, AuthorityId, Block, BlockNumber, Digest, Extrinsic, Header, Runtime, Transfer,
H256 as Hash,
};
use codec::{Decode, Encode, KeyedVec};
use frame_support::storage;
use sp_core::storage::well_known_keys;
use sp_io::{hashing::blake2_256, storage::root as storage_root, trie};
use sp_runtime::{
generic,
traits::Header as _,
transaction_validity::{
InvalidTransaction, TransactionValidity, TransactionValidityError, ValidTransaction,
},
ApplyExtrinsicResult,
};
use sp_std::prelude::*;
const NONCE_OF: &[u8] = b"nonce:";
const BALANCE_OF: &[u8] = b"balance:";
pub use self::pallet::*;
#[frame_support::pallet]
mod pallet {
use super::*;
use frame_support::pallet_prelude::*;
#[pallet::pallet]
#[pallet::without_storage_info]
pub struct Pallet<T>(PhantomData<T>);
#[pallet::config]
pub trait Config: frame_system::Config {}
#[pallet::storage]
pub type ExtrinsicData<T> = StorageMap<_, Blake2_128Concat, u32, Vec<u8>, ValueQuery>;
// The current block number being processed. Set by `execute_block`.
#[pallet::storage]
pub type Number<T> = StorageValue<_, BlockNumber, OptionQuery>;
#[pallet::storage]
pub type ParentHash<T> = StorageValue<_, Hash, ValueQuery>;
#[pallet::storage]
pub type NewAuthorities<T> = StorageValue<_, Vec<AuthorityId>, OptionQuery>;
#[pallet::storage]
pub type StorageDigest<T> = StorageValue<_, Digest, OptionQuery>;
#[pallet::storage]
pub type Authorities<T> = StorageValue<_, Vec<AuthorityId>, ValueQuery>;
#[pallet::genesis_config]
#[cfg_attr(feature = "std", derive(Default))]
pub struct GenesisConfig {
pub authorities: Vec<AuthorityId>,
}
#[pallet::genesis_build]
impl<T: Config> GenesisBuild<T> for GenesisConfig {
fn build(&self) {
<Authorities<T>>::put(self.authorities.clone());
}
}
}
pub fn balance_of_key(who: AccountId) -> Vec<u8> {
who.to_keyed_vec(BALANCE_OF)
}
pub fn balance_of(who: AccountId) -> u64 {
storage::hashed::get_or(&blake2_256, &balance_of_key(who), 0)
}
pub fn nonce_of(who: AccountId) -> u64 {
storage::hashed::get_or(&blake2_256, &who.to_keyed_vec(NONCE_OF), 0)
}
pub fn initialize_block(header: &Header) {
// populate environment.
<Number<Runtime>>::put(&header.number);
<ParentHash<Runtime>>::put(&header.parent_hash);
<StorageDigest<Runtime>>::put(header.digest());
storage::unhashed::put(well_known_keys::EXTRINSIC_INDEX, &0u32);
// try to read something that depends on current header digest
// so that it'll be included in execution proof
if let Some(generic::DigestItem::Other(v)) = header.digest().logs().iter().next() {
let _: Option<u32> = storage::unhashed::get(v);
}
}
pub fn authorities() -> Vec<AuthorityId> {
<Authorities<Runtime>>::get()
}
pub fn get_block_number() -> Option<BlockNumber> {
<Number<Runtime>>::get()
}
pub fn take_block_number() -> Option<BlockNumber> {
<Number<Runtime>>::take()
}
#[derive(Copy, Clone)]
enum Mode {
Verify,
Overwrite,
}
/// Actually execute all transitioning for `block`.
pub fn polish_block(block: &mut Block) {
execute_block_with_state_root_handler(block, Mode::Overwrite);
}
pub fn execute_block(mut block: Block) -> Header {
execute_block_with_state_root_handler(&mut block, Mode::Verify)
}
fn execute_block_with_state_root_handler(block: &mut Block, mode: Mode) -> Header {
let header = &mut block.header;
initialize_block(header);
// execute transactions
block.extrinsics.iter().for_each(|e| {
let _ = execute_transaction(e.clone()).unwrap_or_else(|_| panic!("Invalid transaction"));
});
let new_header = finalize_block();
if let Mode::Overwrite = mode {
header.state_root = new_header.state_root;
} else {
info_expect_equal_hash(&new_header.state_root, &header.state_root);
assert_eq!(
new_header.state_root, header.state_root,
"Storage root must match that calculated.",
);
}
if let Mode::Overwrite = mode {
header.extrinsics_root = new_header.extrinsics_root;
} else {
info_expect_equal_hash(&new_header.extrinsics_root, &header.extrinsics_root);
assert_eq!(
new_header.extrinsics_root, header.extrinsics_root,
"Transaction trie root must be valid.",
);
}
new_header
}
/// The block executor.
pub struct BlockExecutor;
impl frame_support::traits::ExecuteBlock<Block> for BlockExecutor {
fn execute_block(block: Block) {
execute_block(block);
}
}
/// Execute a transaction outside of the block execution function.
/// This doesn't attempt to validate anything regarding the block.
pub fn validate_transaction(utx: Extrinsic) -> TransactionValidity {
if check_signature(&utx).is_err() {
return InvalidTransaction::BadProof.into()
}
let tx = utx.transfer();
let nonce_key = tx.from.to_keyed_vec(NONCE_OF);
let expected_nonce: u64 = storage::hashed::get_or(&blake2_256, &nonce_key, 0);
if tx.nonce < expected_nonce {
return InvalidTransaction::Stale.into()
}
if tx.nonce > expected_nonce + 64 {
return InvalidTransaction::Future.into()
}
let encode = |from: &AccountId, nonce: u64| (from, nonce).encode();
let requires = if tx.nonce != expected_nonce && tx.nonce > 0 {
vec![encode(&tx.from, tx.nonce - 1)]
} else {
vec![]
};
let provides = vec![encode(&tx.from, tx.nonce)];
Ok(ValidTransaction { priority: tx.amount, requires, provides, longevity: 64, propagate: true })
}
/// Execute a transaction outside of the block execution function.
/// This doesn't attempt to validate anything regarding the block.
pub fn execute_transaction(utx: Extrinsic) -> ApplyExtrinsicResult {
let extrinsic_index: u32 =
storage::unhashed::get(well_known_keys::EXTRINSIC_INDEX).unwrap_or_default();
let result = execute_transaction_backend(&utx, extrinsic_index);
<ExtrinsicData<Runtime>>::insert(extrinsic_index, utx.encode());
storage::unhashed::put(well_known_keys::EXTRINSIC_INDEX, &(extrinsic_index + 1));
result
}
/// Finalize the block.
pub fn finalize_block() -> Header {
use sp_core::storage::StateVersion;
let extrinsic_index: u32 = storage::unhashed::take(well_known_keys::EXTRINSIC_INDEX).unwrap();
let txs: Vec<_> = (0..extrinsic_index).map(<ExtrinsicData<Runtime>>::take).collect();
let extrinsics_root = trie::blake2_256_ordered_root(txs, StateVersion::V0);
let number = <Number<Runtime>>::take().expect("Number is set by `initialize_block`");
let parent_hash = <ParentHash<Runtime>>::take();
let mut digest =
<StorageDigest<Runtime>>::take().expect("StorageDigest is set by `initialize_block`");
let o_new_authorities = <NewAuthorities<Runtime>>::take();
// This MUST come after all changes to storage are done. Otherwise we will fail the
// “Storage root does not match that calculated” assertion.
let storage_root = Hash::decode(&mut &storage_root(StateVersion::V1)[..])
.expect("`storage_root` is a valid hash");
if let Some(new_authorities) = o_new_authorities {
digest.push(generic::DigestItem::Consensus(*b"aura", new_authorities.encode()));
digest.push(generic::DigestItem::Consensus(*b"babe", new_authorities.encode()));
}
Header { number, extrinsics_root, state_root: storage_root, parent_hash, digest }
}
#[inline(always)]
fn check_signature(utx: &Extrinsic) -> Result<(), TransactionValidityError> {
use sp_runtime::traits::BlindCheckable;
utx.clone().check().map_err(|_| InvalidTransaction::BadProof.into()).map(|_| ())
}
fn execute_transaction_backend(utx: &Extrinsic, extrinsic_index: u32) -> ApplyExtrinsicResult {
check_signature(utx)?;
match utx {
Extrinsic::Transfer { exhaust_resources_when_not_first: true, .. }
if extrinsic_index != 0 =>
Err(InvalidTransaction::ExhaustsResources.into()),
Extrinsic::Transfer { ref transfer, .. } => execute_transfer_backend(transfer),
Extrinsic::AuthoritiesChange(ref new_auth) => execute_new_authorities_backend(new_auth),
Extrinsic::IncludeData(_) => Ok(Ok(())),
Extrinsic::StorageChange(key, value) =>
execute_storage_change(key, value.as_ref().map(|v| &**v)),
Extrinsic::OffchainIndexSet(key, value) => {
sp_io::offchain_index::set(key, value);
Ok(Ok(()))
},
Extrinsic::OffchainIndexClear(key) => {
sp_io::offchain_index::clear(key);
Ok(Ok(()))
},
Extrinsic::Store(data) => execute_store(data.clone()),
Extrinsic::ReadAndPanic(i) => execute_read(*i, true),
Extrinsic::Read(i) => execute_read(*i, false),
}
}
fn execute_read(read: u32, panic_at_end: bool) -> ApplyExtrinsicResult {
let mut next_key = vec![];
for _ in 0..(read as usize) {
if let Some(next) = sp_io::storage::next_key(&next_key) {
// Read the value
sp_io::storage::get(&next);
next_key = next;
} else {
if panic_at_end {
return Ok(Ok(()))
} else {
panic!("Could not read {read} times from the state");
}
}
}
if panic_at_end {
panic!("BYE")
} else {
Ok(Ok(()))
}
}
fn execute_transfer_backend(tx: &Transfer) -> ApplyExtrinsicResult {
// check nonce
let nonce_key = tx.from.to_keyed_vec(NONCE_OF);
let expected_nonce: u64 = storage::hashed::get_or(&blake2_256, &nonce_key, 0);
if tx.nonce != expected_nonce {
return Err(InvalidTransaction::Stale.into())
}
// increment nonce in storage
storage::hashed::put(&blake2_256, &nonce_key, &(expected_nonce + 1));
// check sender balance
let from_balance_key = tx.from.to_keyed_vec(BALANCE_OF);
let from_balance: u64 = storage::hashed::get_or(&blake2_256, &from_balance_key, 0);
// enact transfer
if tx.amount > from_balance {
return Err(InvalidTransaction::Payment.into())
}
let to_balance_key = tx.to.to_keyed_vec(BALANCE_OF);
let to_balance: u64 = storage::hashed::get_or(&blake2_256, &to_balance_key, 0);
storage::hashed::put(&blake2_256, &from_balance_key, &(from_balance - tx.amount));
storage::hashed::put(&blake2_256, &to_balance_key, &(to_balance + tx.amount));
Ok(Ok(()))
}
fn execute_store(data: Vec<u8>) -> ApplyExtrinsicResult {
let content_hash = sp_io::hashing::blake2_256(&data);
let extrinsic_index: u32 = storage::unhashed::get(well_known_keys::EXTRINSIC_INDEX).unwrap();
sp_io::transaction_index::index(extrinsic_index, data.len() as u32, content_hash);
Ok(Ok(()))
}
fn execute_new_authorities_backend(new_authorities: &[AuthorityId]) -> ApplyExtrinsicResult {
<NewAuthorities<Runtime>>::put(new_authorities.to_vec());
Ok(Ok(()))
}
fn execute_storage_change(key: &[u8], value: Option<&[u8]>) -> ApplyExtrinsicResult {
match value {
Some(value) => storage::unhashed::put_raw(key, value),
None => storage::unhashed::kill(key),
}
Ok(Ok(()))
}
#[cfg(feature = "std")]
fn info_expect_equal_hash(given: &Hash, expected: &Hash) {
use sp_core::hexdisplay::HexDisplay;
if given != expected {
println!(
"Hash: given={}, expected={}",
HexDisplay::from(given.as_fixed_bytes()),
HexDisplay::from(expected.as_fixed_bytes()),
);
}
}
#[cfg(not(feature = "std"))]
fn info_expect_equal_hash(given: &Hash, expected: &Hash) {
if given != expected {
sp_runtime::print("Hash not equal");
sp_runtime::print(given.as_bytes());
sp_runtime::print(expected.as_bytes());
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::{wasm_binary_unwrap, Header, Transfer};
use sc_executor::{NativeElseWasmExecutor, WasmExecutor};
use sp_core::{
map,
traits::{CallContext, CodeExecutor, RuntimeCode},
};
use sp_io::{hashing::twox_128, TestExternalities};
use substrate_test_runtime_client::{AccountKeyring, Sr25519Keyring};
// Declare an instance of the native executor dispatch for the test runtime.
pub struct NativeDispatch;
impl sc_executor::NativeExecutionDispatch for NativeDispatch {
type ExtendHostFunctions = ();
fn dispatch(method: &str, data: &[u8]) -> Option<Vec<u8>> {
crate::api::dispatch(method, data)
}
fn native_version() -> sc_executor::NativeVersion {
crate::native_version()
}
}
fn executor() -> NativeElseWasmExecutor<NativeDispatch> {
NativeElseWasmExecutor::new_with_wasm_executor(WasmExecutor::builder().build())
}
fn new_test_ext() -> TestExternalities {
let authorities = vec![
Sr25519Keyring::Alice.to_raw_public(),
Sr25519Keyring::Bob.to_raw_public(),
Sr25519Keyring::Charlie.to_raw_public(),
];
TestExternalities::new_with_code(
wasm_binary_unwrap(),
sp_core::storage::Storage {
top: map![
twox_128(b"latest").to_vec() => vec![69u8; 32],
twox_128(b"sys:auth").to_vec() => authorities.encode(),
blake2_256(&AccountKeyring::Alice.to_raw_public().to_keyed_vec(b"balance:")).to_vec() => {
vec![111u8, 0, 0, 0, 0, 0, 0, 0]
},
],
children_default: map![],
},
)
}
fn block_import_works<F>(block_executor: F)
where
F: Fn(Block, &mut TestExternalities),
{
let h = Header {
parent_hash: [69u8; 32].into(),
number: 1,
state_root: Default::default(),
extrinsics_root: Default::default(),
digest: Default::default(),
};
let mut b = Block { header: h, extrinsics: vec![] };
new_test_ext().execute_with(|| polish_block(&mut b));
block_executor(b, &mut new_test_ext());
}
#[test]
fn block_import_works_native() {
block_import_works(|b, ext| {
ext.execute_with(|| {
execute_block(b);
})
});
}
#[test]
fn block_import_works_wasm() {
block_import_works(|b, ext| {
let mut ext = ext.ext();
let runtime_code = RuntimeCode {
code_fetcher: &sp_core::traits::WrappedRuntimeCode(wasm_binary_unwrap().into()),
hash: Vec::new(),
heap_pages: None,
};
executor()
.call(
&mut ext,
&runtime_code,
"Core_execute_block",
&b.encode(),
false,
CallContext::Offchain,
)
.0
.unwrap();
})
}
fn block_import_with_transaction_works<F>(block_executor: F)
where
F: Fn(Block, &mut TestExternalities),
{
let mut b1 = Block {
header: Header {
parent_hash: [69u8; 32].into(),
number: 1,
state_root: Default::default(),
extrinsics_root: Default::default(),
digest: Default::default(),
},
extrinsics: vec![Transfer {
from: AccountKeyring::Alice.into(),
to: AccountKeyring::Bob.into(),
amount: 69,
nonce: 0,
}
.into_signed_tx()],
};
let mut dummy_ext = new_test_ext();
dummy_ext.execute_with(|| polish_block(&mut b1));
let mut b2 = Block {
header: Header {
parent_hash: b1.header.hash(),
number: 2,
state_root: Default::default(),
extrinsics_root: Default::default(),
digest: Default::default(),
},
extrinsics: vec![
Transfer {
from: AccountKeyring::Bob.into(),
to: AccountKeyring::Alice.into(),
amount: 27,
nonce: 0,
}
.into_signed_tx(),
Transfer {
from: AccountKeyring::Alice.into(),
to: AccountKeyring::Charlie.into(),
amount: 69,
nonce: 1,
}
.into_signed_tx(),
],
};
dummy_ext.execute_with(|| polish_block(&mut b2));
drop(dummy_ext);
let mut t = new_test_ext();
t.execute_with(|| {
assert_eq!(balance_of(AccountKeyring::Alice.into()), 111);
assert_eq!(balance_of(AccountKeyring::Bob.into()), 0);
});
block_executor(b1, &mut t);
t.execute_with(|| {
assert_eq!(balance_of(AccountKeyring::Alice.into()), 42);
assert_eq!(balance_of(AccountKeyring::Bob.into()), 69);
});
block_executor(b2, &mut t);
t.execute_with(|| {
assert_eq!(balance_of(AccountKeyring::Alice.into()), 0);
assert_eq!(balance_of(AccountKeyring::Bob.into()), 42);
assert_eq!(balance_of(AccountKeyring::Charlie.into()), 69);
});
}
#[test]
fn block_import_with_transaction_works_native() {
block_import_with_transaction_works(|b, ext| {
ext.execute_with(|| {
execute_block(b);
})
});
}
#[test]
fn block_import_with_transaction_works_wasm() {
block_import_with_transaction_works(|b, ext| {
let mut ext = ext.ext();
let runtime_code = RuntimeCode {
code_fetcher: &sp_core::traits::WrappedRuntimeCode(wasm_binary_unwrap().into()),
hash: Vec::new(),
heap_pages: None,
};
executor()
.call(
&mut ext,
&runtime_code,
"Core_execute_block",
&b.encode(),
false,
CallContext::Offchain,
)
.0
.unwrap();
})
}
}