mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-05-31 01:41:03 +00:00
Inner hashing of value in state trie (runtime versioning). (#9732)
* starting * Updated from other branch. * setting flag * flag in storage struct * fix flagging to access and insert. * added todo to fix * also missing serialize meta to storage proof * extract meta. * Isolate old trie layout. * failing test that requires storing in meta when old hash scheme is used. * old hash compatibility * Db migrate. * runing tests with both states when interesting. * fix chain spec test with serde default. * export state (missing trie function). * Pending using new branch, lacking genericity on layout resolution. * extract and set global meta * Update to branch 4 * fix iterator with root flag (no longer insert node). * fix trie root hashing of root * complete basic backend. * Remove old_hash meta from proof that do not use inner_hashing. * fix trie test for empty (force layout on empty deltas). * Root update fix. * debug on meta * Use trie key iteration that do not include value in proofs. * switch default test ext to use inner hash. * small integration test, and fix tx cache mgmt in ext. test failing * Proof scenario at state-machine level. * trace for db upgrade * try different param * act more like iter_from. * Bigger batches. * Update trie dependency. * drafting codec changes and refact * before removing unused branch no value alt hashing. more work todo rename all flag var to alt_hash, and remove extrinsic replace by storage query at every storage_root call. * alt hashing only for branch with value. * fix trie tests * Hash of value include the encoded size. * removing fields(broken) * fix trie_stream to also include value length in inner hash. * triedbmut only using alt type if inner hashing. * trie_stream to also only use alt hashing type when actually alt hashing. * Refactor meta state, logic should work with change of trie treshold. * Remove NoMeta variant. * Remove state_hashed trigger specific functions. * pending switching to using threshold, new storage root api does not make much sense. * refactoring to use state from backend (not possible payload changes). * Applying from previous state * Remove default from storage, genesis need a special build. * rem empty space * Catch problem: when using triedb with default: we should not revert nodes: otherwhise thing as trie codec cannot decode-encode without changing state. * fix compilation * Right logic to avoid switch on reencode when default layout. * Clean up some todos * remove trie meta from root upstream * update upstream and fix benches. * split some long lines. * UPdate trie crate to work with new design. * Finish update to refactored upstream. * update to latest triedb changes. * Clean up. * fix executor test. * rust fmt from master. * rust format. * rustfmt * fix * start host function driven versioning * update state-machine part * still need access to state version from runtime * state hash in mem: wrong * direction likely correct, but passing call to code exec for genesis init seem awkward. * state version serialize in runtime, wrong approach, just initialize it with no threshold for core api < 4 seems more proper. * stateversion from runtime version (core api >= 4). * update trie, fix tests * unused import * clean some TODOs * Require RuntimeVersionOf for executor * use RuntimeVersionOf to resolve genesis state version. * update runtime version test * fix state-machine tests * TODO * Use runtime version from storage wasm with fast sync. * rustfmt * fmt * fix test * revert useless changes. * clean some unused changes * fmt * removing useless trait function. * remove remaining reference to state_hash * fix some imports * Follow chain state version management. * trie update, fix and constant threshold for trie layouts. * update deps * Update to latest trie pr changes. * fix benches * Verify proof requires right layout. * update trie_root * Update trie deps to latest * Update to latest trie versioning * Removing patch * update lock * extrinsic for sc-service-test using layout v0. * Adding RuntimeVersionOf to CallExecutor works. * fmt * error when resolving version and no wasm in storage. * use existing utils to instantiate runtime code. * Patch to delay runtime switch. * Revert "Patch to delay runtime switch." This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da. * useless closure * remove remaining state_hash variables. * Remove outdated comment * useless inner hash * fmt * fmt and opt-in feature to apply state change. * feature gate core version, use new test feature for node and test node * Use a 'State' api version instead of Core one. * fix merge of test function * use blake macro. * Fix state api (require declaring the api in runtime). * Opt out feature, fix macro for io to select a given version instead of latest. * run test nodes on new state. * fix * Apply review change (docs and error). * fmt * use explicit runtime_interface in doc test * fix ui test * fix doc test * fmt * use default for path and specname when resolving version. * small review related changes. * doc value size requirement. * rename old_state feature * Remove macro changes * feature rename * state version as host function parameter * remove flag for client api * fix tests * switch storage chain proof to V1 * host functions, pass by state version enum * use WrappedRuntimeCode * start * state_version in runtime version * rust fmt * Update storage proof of max size. * fix runtime version rpc test * right intent of convert from compat * fix doc test * fix doc test * split proof * decode without replay, and remove some reexports. * Decode with compatibility by default. * switch state_version to u8. And remove RuntimeVersionBasis. * test * use api when reading embedded version * fix decode with apis * extract core version instead * test fix * unused import * review changes. Co-authored-by: kianenigma <kian@parity.io>
This commit is contained in:
cheme
@@ -18,21 +18,18 @@
|
||||
//! `TrieStream` implementation for Substrate's trie format.
|
||||
|
||||
use crate::{
|
||||
node_codec::Bitmap,
|
||||
node_header::{size_and_prefix_iterator, NodeKind},
|
||||
trie_constants,
|
||||
};
|
||||
use codec::Encode;
|
||||
use codec::{Compact, Encode};
|
||||
use hash_db::Hasher;
|
||||
use sp_std::vec::Vec;
|
||||
use trie_root;
|
||||
|
||||
const BRANCH_NODE_NO_VALUE: u8 = 254;
|
||||
const BRANCH_NODE_WITH_VALUE: u8 = 255;
|
||||
|
||||
#[derive(Default, Clone)]
|
||||
/// Codec-flavored TrieStream.
|
||||
pub struct TrieStream {
|
||||
/// Current node buffer.
|
||||
buffer: Vec<u8>,
|
||||
}
|
||||
|
||||
@@ -60,51 +57,76 @@ fn fuse_nibbles_node<'a>(nibbles: &'a [u8], kind: NodeKind) -> impl Iterator<Ite
|
||||
let size = sp_std::cmp::min(trie_constants::NIBBLE_SIZE_BOUND, nibbles.len());
|
||||
|
||||
let iter_start = match kind {
|
||||
NodeKind::Leaf => size_and_prefix_iterator(size, trie_constants::LEAF_PREFIX_MASK),
|
||||
NodeKind::Leaf => size_and_prefix_iterator(size, trie_constants::LEAF_PREFIX_MASK, 2),
|
||||
NodeKind::BranchNoValue =>
|
||||
size_and_prefix_iterator(size, trie_constants::BRANCH_WITHOUT_MASK),
|
||||
size_and_prefix_iterator(size, trie_constants::BRANCH_WITHOUT_MASK, 2),
|
||||
NodeKind::BranchWithValue =>
|
||||
size_and_prefix_iterator(size, trie_constants::BRANCH_WITH_MASK),
|
||||
size_and_prefix_iterator(size, trie_constants::BRANCH_WITH_MASK, 2),
|
||||
NodeKind::HashedValueLeaf =>
|
||||
size_and_prefix_iterator(size, trie_constants::ALT_HASHING_LEAF_PREFIX_MASK, 3),
|
||||
NodeKind::HashedValueBranch =>
|
||||
size_and_prefix_iterator(size, trie_constants::ALT_HASHING_BRANCH_WITH_MASK, 4),
|
||||
};
|
||||
iter_start
|
||||
.chain(if nibbles.len() % 2 == 1 { Some(nibbles[0]) } else { None })
|
||||
.chain(nibbles[nibbles.len() % 2..].chunks(2).map(|ch| ch[0] << 4 | ch[1]))
|
||||
}
|
||||
|
||||
use trie_root::Value as TrieStreamValue;
|
||||
impl trie_root::TrieStream for TrieStream {
|
||||
fn new() -> Self {
|
||||
TrieStream { buffer: Vec::new() }
|
||||
Self { buffer: Vec::new() }
|
||||
}
|
||||
|
||||
fn append_empty_data(&mut self) {
|
||||
self.buffer.push(trie_constants::EMPTY_TRIE);
|
||||
}
|
||||
|
||||
fn append_leaf(&mut self, key: &[u8], value: &[u8]) {
|
||||
self.buffer.extend(fuse_nibbles_node(key, NodeKind::Leaf));
|
||||
value.encode_to(&mut self.buffer);
|
||||
fn append_leaf(&mut self, key: &[u8], value: TrieStreamValue) {
|
||||
let kind = match &value {
|
||||
TrieStreamValue::Inline(..) => NodeKind::Leaf,
|
||||
TrieStreamValue::Node(..) => NodeKind::HashedValueLeaf,
|
||||
};
|
||||
self.buffer.extend(fuse_nibbles_node(key, kind));
|
||||
match &value {
|
||||
TrieStreamValue::Inline(value) => {
|
||||
Compact(value.len() as u32).encode_to(&mut self.buffer);
|
||||
self.buffer.extend_from_slice(value);
|
||||
},
|
||||
TrieStreamValue::Node(hash) => {
|
||||
self.buffer.extend_from_slice(hash.as_slice());
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
fn begin_branch(
|
||||
&mut self,
|
||||
maybe_partial: Option<&[u8]>,
|
||||
maybe_value: Option<&[u8]>,
|
||||
maybe_value: Option<TrieStreamValue>,
|
||||
has_children: impl Iterator<Item = bool>,
|
||||
) {
|
||||
if let Some(partial) = maybe_partial {
|
||||
if maybe_value.is_some() {
|
||||
self.buffer.extend(fuse_nibbles_node(partial, NodeKind::BranchWithValue));
|
||||
} else {
|
||||
self.buffer.extend(fuse_nibbles_node(partial, NodeKind::BranchNoValue));
|
||||
}
|
||||
let kind = match &maybe_value {
|
||||
None => NodeKind::BranchNoValue,
|
||||
Some(TrieStreamValue::Inline(..)) => NodeKind::BranchWithValue,
|
||||
Some(TrieStreamValue::Node(..)) => NodeKind::HashedValueBranch,
|
||||
};
|
||||
|
||||
self.buffer.extend(fuse_nibbles_node(partial, kind));
|
||||
let bm = branch_node_bit_mask(has_children);
|
||||
self.buffer.extend([bm.0, bm.1].iter());
|
||||
} else {
|
||||
debug_assert!(false, "trie stream codec only for no extension trie");
|
||||
self.buffer.extend(&branch_node(maybe_value.is_some(), has_children));
|
||||
unreachable!("trie stream codec only for no extension trie");
|
||||
}
|
||||
if let Some(value) = maybe_value {
|
||||
value.encode_to(&mut self.buffer);
|
||||
match maybe_value {
|
||||
None => (),
|
||||
Some(TrieStreamValue::Inline(value)) => {
|
||||
Compact(value.len() as u32).encode_to(&mut self.buffer);
|
||||
self.buffer.extend_from_slice(value);
|
||||
},
|
||||
Some(TrieStreamValue::Node(hash)) => {
|
||||
self.buffer.extend_from_slice(hash.as_slice());
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -124,18 +146,3 @@ impl trie_root::TrieStream for TrieStream {
|
||||
self.buffer
|
||||
}
|
||||
}
|
||||
|
||||
fn branch_node(has_value: bool, has_children: impl Iterator<Item = bool>) -> [u8; 3] {
|
||||
let mut result = [0, 0, 0];
|
||||
branch_node_buffered(has_value, has_children, &mut result[..]);
|
||||
result
|
||||
}
|
||||
|
||||
fn branch_node_buffered<I>(has_value: bool, has_children: I, output: &mut [u8])
|
||||
where
|
||||
I: Iterator<Item = bool>,
|
||||
{
|
||||
let first = if has_value { BRANCH_NODE_WITH_VALUE } else { BRANCH_NODE_NO_VALUE };
|
||||
output[0] = first;
|
||||
Bitmap::encode(has_children, &mut output[1..]);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user