Files
pezkuwi-subxt/substrate/bin/node/executor/tests/common.rs
T
cheme 4c651637f2 Inner hashing of value in state trie (runtime versioning). (#9732)
* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.

* useless closure

* remove remaining state_hash variables.

* Remove outdated comment

* useless inner hash

* fmt

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* Apply review change (docs and error).

* fmt

* use explicit runtime_interface in doc test

* fix ui test

* fix doc test

* fmt

* use default for path and specname when resolving version.

* small review related changes.

* doc value size requirement.

* rename old_state feature

* Remove macro changes

* feature rename

* state version as host function parameter

* remove flag for client api

* fix tests

* switch storage chain proof to V1

* host functions, pass by state version enum

* use WrappedRuntimeCode

* start

* state_version in runtime version

* rust fmt

* Update storage proof of max size.

* fix runtime version rpc test

* right intent of convert from compat

* fix doc test

* fix doc test

* split proof

* decode without replay, and remove some reexports.

* Decode with compatibility by default.

* switch state_version to u8. And remove RuntimeVersionBasis.

* test

* use api when reading embedded version

* fix decode with apis

* extract core version instead

* test fix

* unused import

* review changes.

Co-authored-by: kianenigma <kian@parity.io>
2021-12-24 08:54:07 +00:00

222 lines
6.4 KiB
Rust

// This file is part of Substrate.
// Copyright (C) 2018-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use codec::{Decode, Encode};
use frame_support::Hashable;
use frame_system::offchain::AppCrypto;
use sc_executor::{error::Result, NativeElseWasmExecutor, WasmExecutionMethod};
use sp_consensus_babe::{
digests::{PreDigest, SecondaryPlainPreDigest},
Slot, BABE_ENGINE_ID,
};
use sp_core::{
crypto::KeyTypeId,
sr25519::Signature,
traits::{CodeExecutor, RuntimeCode},
NativeOrEncoded, NeverNativeValue,
};
use sp_runtime::{
traits::{BlakeTwo256, Header as HeaderT},
ApplyExtrinsicResult, Digest, DigestItem, MultiSignature, MultiSigner,
};
use sp_state_machine::TestExternalities as CoreTestExternalities;
use node_executor::ExecutorDispatch;
use node_primitives::{BlockNumber, Hash};
use node_runtime::{
constants::currency::*, Block, BuildStorage, CheckedExtrinsic, Header, Runtime,
UncheckedExtrinsic,
};
use node_testing::keyring::*;
use sp_externalities::Externalities;
pub const TEST_KEY_TYPE_ID: KeyTypeId = KeyTypeId(*b"test");
pub mod sr25519 {
mod app_sr25519 {
use super::super::TEST_KEY_TYPE_ID;
use sp_application_crypto::{app_crypto, sr25519};
app_crypto!(sr25519, TEST_KEY_TYPE_ID);
}
pub type AuthorityId = app_sr25519::Public;
}
pub struct TestAuthorityId;
impl AppCrypto<MultiSigner, MultiSignature> for TestAuthorityId {
type RuntimeAppPublic = sr25519::AuthorityId;
type GenericSignature = Signature;
type GenericPublic = sp_core::sr25519::Public;
}
/// The wasm runtime code.
///
/// `compact` since it is after post-processing with wasm-gc which performs tree-shaking thus
/// making the binary slimmer. There is a convention to use compact version of the runtime
/// as canonical.
pub fn compact_code_unwrap() -> &'static [u8] {
node_runtime::WASM_BINARY.expect(
"Development wasm binary is not available. Testing is only supported with the flag \
disabled.",
)
}
pub const GENESIS_HASH: [u8; 32] = [69u8; 32];
pub const SPEC_VERSION: u32 = node_runtime::VERSION.spec_version;
pub const TRANSACTION_VERSION: u32 = node_runtime::VERSION.transaction_version;
pub type TestExternalities<H> = CoreTestExternalities<H>;
pub fn sign(xt: CheckedExtrinsic) -> UncheckedExtrinsic {
node_testing::keyring::sign(xt, SPEC_VERSION, TRANSACTION_VERSION, GENESIS_HASH)
}
pub fn default_transfer_call() -> pallet_balances::Call<Runtime> {
pallet_balances::Call::<Runtime>::transfer { dest: bob().into(), value: 69 * DOLLARS }
}
pub fn from_block_number(n: u32) -> Header {
Header::new(n, Default::default(), Default::default(), [69; 32].into(), Default::default())
}
pub fn executor() -> NativeElseWasmExecutor<ExecutorDispatch> {
NativeElseWasmExecutor::new(WasmExecutionMethod::Interpreted, None, 8, 2)
}
pub fn executor_call<
R: Decode + Encode + PartialEq,
NC: FnOnce() -> std::result::Result<R, Box<dyn std::error::Error + Send + Sync>>
+ std::panic::UnwindSafe,
>(
t: &mut TestExternalities<BlakeTwo256>,
method: &str,
data: &[u8],
use_native: bool,
native_call: Option<NC>,
) -> (Result<NativeOrEncoded<R>>, bool) {
let mut t = t.ext();
let code = t.storage(sp_core::storage::well_known_keys::CODE).unwrap();
let heap_pages = t.storage(sp_core::storage::well_known_keys::HEAP_PAGES);
let runtime_code = RuntimeCode {
code_fetcher: &sp_core::traits::WrappedRuntimeCode(code.as_slice().into()),
hash: sp_core::blake2_256(&code).to_vec(),
heap_pages: heap_pages.and_then(|hp| Decode::decode(&mut &hp[..]).ok()),
};
sp_tracing::try_init_simple();
executor().call::<R, NC>(&mut t, &runtime_code, method, data, use_native, native_call)
}
pub fn new_test_ext(code: &[u8]) -> TestExternalities<BlakeTwo256> {
let ext = TestExternalities::new_with_code(
code,
node_testing::genesis::config(Some(code)).build_storage().unwrap(),
);
ext
}
/// Construct a fake block.
///
/// `extrinsics` must be a list of valid extrinsics, i.e. none of the extrinsics for example
/// can report `ExhaustResources`. Otherwise, this function panics.
pub fn construct_block(
env: &mut TestExternalities<BlakeTwo256>,
number: BlockNumber,
parent_hash: Hash,
extrinsics: Vec<CheckedExtrinsic>,
babe_slot: Slot,
) -> (Vec<u8>, Hash) {
use sp_trie::{LayoutV1 as Layout, TrieConfiguration};
// sign extrinsics.
let extrinsics = extrinsics.into_iter().map(sign).collect::<Vec<_>>();
// calculate the header fields that we can.
let extrinsics_root =
Layout::<BlakeTwo256>::ordered_trie_root(extrinsics.iter().map(Encode::encode))
.to_fixed_bytes()
.into();
let header = Header {
parent_hash,
number,
extrinsics_root,
state_root: Default::default(),
digest: Digest {
logs: vec![DigestItem::PreRuntime(
BABE_ENGINE_ID,
PreDigest::SecondaryPlain(SecondaryPlainPreDigest {
slot: babe_slot,
authority_index: 42,
})
.encode(),
)],
},
};
// execute the block to get the real header.
executor_call::<NeverNativeValue, fn() -> _>(
env,
"Core_initialize_block",
&header.encode(),
true,
None,
)
.0
.unwrap();
for extrinsic in extrinsics.iter() {
// Try to apply the `extrinsic`. It should be valid, in the sense that it passes
// all pre-inclusion checks.
let r = executor_call::<NeverNativeValue, fn() -> _>(
env,
"BlockBuilder_apply_extrinsic",
&extrinsic.encode(),
true,
None,
)
.0
.expect("application of an extrinsic failed")
.into_encoded();
match ApplyExtrinsicResult::decode(&mut &r[..])
.expect("apply result deserialization failed")
{
Ok(_) => {},
Err(e) => panic!("Applying extrinsic failed: {:?}", e),
}
}
let header = match executor_call::<NeverNativeValue, fn() -> _>(
env,
"BlockBuilder_finalize_block",
&[0u8; 0],
true,
None,
)
.0
.unwrap()
{
NativeOrEncoded::Native(_) => unreachable!(),
NativeOrEncoded::Encoded(h) => Header::decode(&mut &h[..]).unwrap(),
};
let hash = header.blake2_256();
(Block { header, extrinsics }.encode(), hash.into())
}