Introduce Pallet paged-list (#14120)

* Prototype StoragePagedList

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add drain

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove stale docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add fuzzer tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review

Co-authored-by: Koute <koute@users.noreply.github.com>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs and clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Sum docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Undo WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add pallet-paged-list

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Move code to pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Move fuzzer

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename Appendix -> Appender

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename clear -> delete

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Feature gate testing stuff

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs review

Co-authored-by: Koute <koute@users.noreply.github.com>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* doc review

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review renames

Co-authored-by: Koute <koute@users.noreply.github.com>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix fuzzer

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs + examples

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove hasher

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove empty Event and Call

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove MaxPages

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test eager page removal

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/paged-list/src/paged_list.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Fix docs

Co-authored-by: Koute <koute@users.noreply.github.com>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove as_*_vec

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update versions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename ValuesPerPage -> ValuesPerNewPage

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update lockfile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix mock

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: parity-processbot <>
This commit is contained in:
Oliver Tale-Yazdi
2023-07-19 22:14:56 +02:00
committed by GitHub
parent 817c97d65d
commit daa9ecf77d
12 changed files with 1222 additions and 4 deletions
+1
View File
@@ -1553,6 +1553,7 @@ pub mod pallet_prelude {
CountedStorageMap, Key as NMapKey, OptionQuery, ResultQuery, StorageDoubleMap,
StorageMap, StorageNMap, StorageValue, ValueQuery,
},
StorageList,
},
traits::{
BuildGenesisConfig, ConstU32, EnsureOrigin, Get, GetDefault, GetStorageVersion, Hooks,
+84 -1
View File
@@ -160,6 +160,75 @@ pub trait StorageValue<T: FullCodec> {
}
}
/// A non-continuous container type.
pub trait StorageList<V: FullCodec> {
/// Iterator for normal and draining iteration.
type Iterator: Iterator<Item = V>;
/// Append iterator for fast append operations.
type Appender: StorageAppender<V>;
/// List the elements in append order.
fn iter() -> Self::Iterator;
/// Drain the elements in append order.
///
/// Note that this drains a value as soon as it is being inspected. For example `take_while(|_|
/// false)` still drains the first element. This also applies to `peek()`.
fn drain() -> Self::Iterator;
/// A fast append iterator.
fn appender() -> Self::Appender;
/// Append a single element.
///
/// Should not be called repeatedly; use `append_many` instead.
/// Worst case linear `O(len)` with `len` being the number if elements in the list.
fn append_one<EncodeLikeValue>(item: EncodeLikeValue)
where
EncodeLikeValue: EncodeLike<V>,
{
Self::append_many(core::iter::once(item));
}
/// Append many elements.
///
/// Should not be called repeatedly; use `appender` instead.
/// Worst case linear `O(len + items.count())` with `len` beings the number if elements in the
/// list.
fn append_many<EncodeLikeValue, I>(items: I)
where
EncodeLikeValue: EncodeLike<V>,
I: IntoIterator<Item = EncodeLikeValue>,
{
let mut ap = Self::appender();
ap.append_many(items);
}
}
/// Append iterator to append values to a storage struct.
///
/// Can be used in situations where appending does not have constant time complexity.
pub trait StorageAppender<V: FullCodec> {
/// Append a single item in constant time `O(1)`.
fn append<EncodeLikeValue>(&mut self, item: EncodeLikeValue)
where
EncodeLikeValue: EncodeLike<V>;
/// Append many items in linear time `O(items.count())`.
// Note: a default impl is provided since `Self` is already assumed to be optimal for single
// append operations.
fn append_many<EncodeLikeValue, I>(&mut self, items: I)
where
EncodeLikeValue: EncodeLike<V>,
I: IntoIterator<Item = EncodeLikeValue>,
{
for item in items.into_iter() {
self.append(item);
}
}
}
/// A strongly-typed map in storage.
///
/// Details on implementation can be found at [`generator::StorageMap`].
@@ -1193,6 +1262,20 @@ impl<T> Iterator for ChildTriePrefixIterator<T> {
}
}
/// Trait for storage types that store all its value after a unique prefix.
pub trait StoragePrefixedContainer {
/// Module prefix. Used for generating final key.
fn module_prefix() -> &'static [u8];
/// Storage prefix. Used for generating final key.
fn storage_prefix() -> &'static [u8];
/// Final full prefix that prefixes all keys.
fn final_prefix() -> [u8; 32] {
crate::storage::storage_prefix(Self::module_prefix(), Self::storage_prefix())
}
}
/// Trait for maps that store all its value after a unique prefix.
///
/// By default the final prefix is:
@@ -1201,7 +1284,7 @@ impl<T> Iterator for ChildTriePrefixIterator<T> {
/// ```
pub trait StoragePrefixedMap<Value: FullCodec> {
/// Module prefix. Used for generating final key.
fn module_prefix() -> &'static [u8];
fn module_prefix() -> &'static [u8]; // TODO move to StoragePrefixedContainer
/// Storage prefix. Used for generating final key.
fn storage_prefix() -> &'static [u8];
@@ -134,7 +134,7 @@ impl<T> ScaleContainerStreamIter<T> {
///
/// - `key`: Storage key of the container in the state.
///
/// Same as [`Self::try_new`], but logs a potential error and sets the length to `0`.
/// Same as [`Self::new_try`], but logs a potential error and sets the length to `0`.
pub fn new(key: Vec<u8>) -> Self {
let mut input = StorageInput::new(key);
let length = if input.exists() {