pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-04-27 02:17:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

Inner hashing of value in state trie (runtime versioning). (#9732)

Browse Source 
* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.

* useless closure

* remove remaining state_hash variables.

* Remove outdated comment

* useless inner hash

* fmt

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* Apply review change (docs and error).

* fmt

* use explicit runtime_interface in doc test

* fix ui test

* fix doc test

* fmt

* use default for path and specname when resolving version.

* small review related changes.

* doc value size requirement.

* rename old_state feature

* Remove macro changes

* feature rename

* state version as host function parameter

* remove flag for client api

* fix tests

* switch storage chain proof to V1

* host functions, pass by state version enum

* use WrappedRuntimeCode

* start

* state_version in runtime version

* rust fmt

* Update storage proof of max size.

* fix runtime version rpc test

* right intent of convert from compat

* fix doc test

* fix doc test

* split proof

* decode without replay, and remove some reexports.

* Decode with compatibility by default.

* switch state_version to u8. And remove RuntimeVersionBasis.

* test

* use api when reading embedded version

* fix decode with apis

* extract core version instead

* test fix

* unused import

* review changes.

Co-authored-by: kianenigma <kian@parity.io>
This commit is contained in:
cheme
2021-12-24 09:54:07 +01:00
committed by GitHub
parent ea30c739ea
commit 4c651637f2
78 changed files with 1814 additions and 782 deletions
Download Patch File Download Diff File Expand all files Collapse all files
substrate/client/executor/src/wasm_runtime.rs
+56 -30
View File
@@ -334,22 +334,11 @@ where
}
fn decode_version(mut version: &[u8]) -> Result<RuntimeVersion, WasmError> {
let v: RuntimeVersion = sp_api::OldRuntimeVersion::decode(&mut &version[..])
.map_err(|_| {
WasmError::Instantiation(
"failed to decode \"Core_version\" result using old runtime version".into(),
)
})?
.into();
let core_api_id = sp_core_hashing_proc_macro::blake2b_64!(b"Core");
if v.has_api_with(&core_api_id, |v| v >= 3) {
sp_api::RuntimeVersion::decode(&mut version).map_err(|_| {
WasmError::Instantiation("failed to decode \"Core_version\" result".into())
})
} else {
Ok(v)
}
Decode::decode(&mut version).map_err(|_| {
WasmError::Instantiation(
"failed to decode \"Core_version\" result using old runtime version".into(),
)
})
}
fn decode_runtime_apis(apis: &[u8]) -> Result<Vec<([u8; 8], u32)>, WasmError> {
@@ -373,17 +362,25 @@ fn decode_runtime_apis(apis: &[u8]) -> Result<Vec<([u8; 8], u32)>, WasmError> {
/// sections, `Err` will be returned.
pub fn read_embedded_version(blob: &RuntimeBlob) -> Result<Option<RuntimeVersion>, WasmError> {
if let Some(mut version_section) = blob.custom_section_contents("runtime_version") {
// We do not use `decode_version` here because the runtime_version section is not supposed
// to ever contain a legacy version. Apart from that `decode_version` relies on presence
// of a special API in the `apis` field to treat the input as a non-legacy version. However
// the structure found in the `runtime_version` always contain an empty `apis` field.
// Therefore the version read will be mistakenly treated as an legacy one.
let mut decoded_version = sp_api::RuntimeVersion::decode(&mut version_section)
.map_err(|_| WasmError::Instantiation("failed to decode version section".into()))?;
let apis = blob
.custom_section_contents("runtime_apis")
.map(decode_runtime_apis)
.transpose()?
.map(Into::into);
// Don't stop on this and check if there is a special section that encodes all runtime APIs.
if let Some(apis_section) = blob.custom_section_contents("runtime_apis") {
decoded_version.apis = decode_runtime_apis(apis_section)?.into();
let core_version = apis.as_ref().and_then(|apis| sp_version::core_version_from_apis(apis));
// We do not use `RuntimeVersion::decode` here because that `decode_version` relies on
// presence of a special API in the `apis` field to treat the input as a non-legacy version.
// However the structure found in the `runtime_version` always contain an empty `apis`
// field. Therefore the version read will be mistakenly treated as an legacy one.
let mut decoded_version = sp_version::RuntimeVersion::decode_with_version_hint(
&mut version_section,
core_version,
)
.map_err(|_| WasmError::Instantiation("failed to decode version section".into()))?;
if let Some(apis) = apis {
decoded_version.apis = apis;
}
Ok(Some(decoded_version))
@@ -455,9 +452,20 @@ mod tests {
use super::*;
use codec::Encode;
use sp_api::{Core, RuntimeApiInfo};
use sp_runtime::RuntimeString;
use sp_wasm_interface::HostFunctions;
use substrate_test_runtime::Block;
#[derive(Encode)]
pub struct OldRuntimeVersion {
pub spec_name: RuntimeString,
pub impl_name: RuntimeString,
pub authoring_version: u32,
pub spec_version: u32,
pub impl_version: u32,
pub apis: sp_version::ApisVec,
}
#[test]
fn host_functions_are_equal() {
let host_functions = sp_io::SubstrateHostFunctions::host_functions();
@@ -468,7 +476,7 @@ mod tests {
#[test]
fn old_runtime_version_decodes() {
let old_runtime_version = sp_api::OldRuntimeVersion {
let old_runtime_version = OldRuntimeVersion {
spec_name: "test".into(),
impl_name: "test".into(),
authoring_version: 1,
@@ -479,11 +487,12 @@ mod tests {
let version = decode_version(&old_runtime_version.encode()).unwrap();
assert_eq!(1, version.transaction_version);
assert_eq!(0, version.state_version);
}
#[test]
fn old_runtime_version_decodes_fails_with_version_3() {
let old_runtime_version = sp_api::OldRuntimeVersion {
let old_runtime_version = OldRuntimeVersion {
spec_name: "test".into(),
impl_name: "test".into(),
authoring_version: 1,
@@ -505,10 +514,27 @@ mod tests {
impl_version: 1,
apis: sp_api::create_apis_vec!([(<dyn Core::<Block>>::ID, 3)]),
transaction_version: 3,
state_version: 4,
};
let version = decode_version(&old_runtime_version.encode()).unwrap();
assert_eq!(3, version.transaction_version);
assert_eq!(0, version.state_version);
let old_runtime_version = sp_api::RuntimeVersion {
spec_name: "test".into(),
impl_name: "test".into(),
authoring_version: 1,
spec_version: 1,
impl_version: 1,
apis: sp_api::create_apis_vec!([(<dyn Core::<Block>>::ID, 4)]),
transaction_version: 3,
state_version: 4,
};
let version = decode_version(&old_runtime_version.encode()).unwrap();
assert_eq!(3, version.transaction_version);
assert_eq!(4, version.state_version);
}
#[test]
@@ -518,15 +544,15 @@ mod tests {
sp_maybe_compressed_blob::CODE_BLOB_BOMB_LIMIT,
)
.expect("Decompressing works");
let runtime_version = RuntimeVersion {
spec_name: "test_replace".into(),
impl_name: "test_replace".into(),
authoring_version: 100,
spec_version: 100,
impl_version: 100,
apis: sp_api::create_apis_vec!([(<dyn Core::<Block>>::ID, 3)]),
apis: sp_api::create_apis_vec!([(<dyn Core::<Block>>::ID, 4)]),
transaction_version: 100,
state_version: 1,
};
let embedded = sp_version::embed::embed_runtime_version(&wasm, runtime_version.clone())