Bound uses of Call (#11649)

* Introduce preimages module in traits

* Multisize Preimages

* Len not actually necessary

* Tweaks to the preimage API

* Fixes

* Get Scheduler building with new API

* Scheduler tests pass

* Bounded Scheduler 🎉

* Use Agenda holes and introduce IncompleteSince to avoid need to reschedule

* Tests pass with new weight system

* New benchmarks

* Add missing file

* Drop preimage when permenantly overeight

* Drop preimage when permenantly overeight

* Referenda uses latest preimage API

* Testing ok

* Adding tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add preimage migration

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

* Remove dbg

* Refactor Democracy

* Refactor Democracy

* Add final MEL

* Remove silly maps

* Fixes

* Minor refactor

* Formatting

* Fixes

* Fixes

* Fixes

* Update frame/preimage/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Add migrations to Democracy

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Resolve conflicts

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Resolve conflicts"

This reverts commit 734d66d69e54553471ffa54fa52e3e304dc8f106.

* Undo wrong resolves...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* massage clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy annoyance

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy annoyance

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add missing file

* Test <Preimage as QueryPreimage>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy harassment

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fixup tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove old stuff

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test <Scheduler as Anon> trait functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update pallet-ui tests

Why is this needed? Should not be the case unless master is broken...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More scheduler trait test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Apply review suggestion

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Beauty fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add Scheduler test migration_v3_to_v4_works

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Merge fixup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Keep referenda benchmarks instantiatable

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new scheduler weight functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new democracy weight functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use weight compare functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update pallet-ui tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More renaming…

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More renaming…

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add comment

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Implement OnRuntimeUpgrade for scheduler::v3_to_v4 migration

Put the migration into a proper `MigrateToV4` struct and implement
the OnRuntimeUpgrade hooks for it. Also move the test to use that
instead.

This should make it easier for adding it to Polkadot.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Handle undecodable Agendas

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove trash

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new OnRuntimeUpgrade functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix BoundedSlice::truncate_from

Co-authored-by: jakoblell

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix pre_upgrade hook return values

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more error logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Find too large preimages in the pre_upgrade hook

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test that too large Calls in agendas are ignored

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new OnRuntimeUpgrade hooks

Why did the CI not catch this?!

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* works fine - just more logs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix staking migration

Causing issues on Kusama...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix UI tests

No idea why this is needed. This is actually undoing an earlier change.
Maybe the CI has different rustc versions!?

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove multisig's Calls (#12072)

* Remove multisig's Calls

* Multisig: Fix tests and re-introduce reserve logic (#12241)

* Fix tests and re-introduce reserve logic

* fix benches

* add todo

* remove irrelevant bench

* [Feature] Add a migration that drains and refunds stored calls (#12313)

* [Feature] Add a migration that drains and refunds stored calls

* migration fixes

* fixes

* address review comments

* consume the whole block weight

* fix assertions

* license header

* fix interface

Co-authored-by: parity-processbot <>

Co-authored-by: parity-processbot <>
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix multisig benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_democracy

* ".git/.scripts/bench-bot.sh" pallet dev pallet_scheduler

* ".git/.scripts/bench-bot.sh" pallet dev pallet_preimage

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: parity-processbot <>
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
This commit is contained in:
Gavin Wood
2022-10-05 19:21:37 +01:00
committed by GitHub
parent 24f0c3601c
commit 93e8ffed55
67 changed files with 5089 additions and 3488 deletions
+268 -11
View File
@@ -17,11 +17,35 @@
//! # Scheduler tests.
#![cfg(test)]
use super::*;
use crate::mock::*;
use frame_support::{assert_noop, assert_ok};
use frame_support::{
assert_err, assert_noop, assert_ok, assert_storage_noop, bounded_vec,
traits::{Bounded, BoundedInline, Hash as PreimageHash},
StorageNoopGuard,
};
use pallet_balances::Error as BalancesError;
use sp_core::{blake2_256, H256};
/// Returns one `Inline`, `Lookup` and `Legacy` item each with different data and hash.
pub fn make_bounded_values() -> (Bounded<Vec<u8>>, Bounded<Vec<u8>>, Bounded<Vec<u8>>) {
let data: BoundedInline = bounded_vec![1];
let inline = Bounded::<Vec<u8>>::Inline(data);
let data = vec![1, 2];
let hash: H256 = blake2_256(&data[..]).into();
let len = data.len() as u32;
let lookup = Bounded::<Vec<u8>>::unrequested(hash, len);
let data = vec![1, 2, 3];
let hash: H256 = blake2_256(&data[..]).into();
let legacy = Bounded::<Vec<u8>>::Legacy { hash, dummy: Default::default() };
(inline, lookup, legacy)
}
#[test]
fn user_note_preimage_works() {
@@ -56,10 +80,7 @@ fn manager_note_preimage_works() {
assert!(Preimage::have_preimage(&h));
assert_eq!(Preimage::get_preimage(&h), Some(vec![1]));
assert_noop!(
Preimage::note_preimage(RuntimeOrigin::signed(1), vec![1]),
Error::<Test>::AlreadyNoted
);
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(1), vec![1]));
});
}
@@ -130,14 +151,16 @@ fn requested_then_noted_preimage_cannot_be_unnoted() {
new_test_ext().execute_with(|| {
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(1), vec![1]));
assert_ok!(Preimage::request_preimage(RuntimeOrigin::signed(1), hashed([1])));
assert_noop!(
Preimage::unnote_preimage(RuntimeOrigin::signed(1), hashed([1])),
Error::<Test>::Requested
);
assert_ok!(Preimage::unnote_preimage(RuntimeOrigin::signed(1), hashed([1])));
// it's still here.
let h = hashed([1]);
assert!(Preimage::have_preimage(&h));
assert_eq!(Preimage::get_preimage(&h), Some(vec![1]));
// now it's gone
assert_ok!(Preimage::unrequest_preimage(RuntimeOrigin::signed(1), hashed([1])));
assert!(!Preimage::have_preimage(&hashed([1])));
});
}
@@ -145,15 +168,16 @@ fn requested_then_noted_preimage_cannot_be_unnoted() {
fn request_note_order_makes_no_difference() {
let one_way = new_test_ext().execute_with(|| {
assert_ok!(Preimage::request_preimage(RuntimeOrigin::signed(1), hashed([1])));
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(1), vec![1]));
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(2), vec![1]));
(
StatusFor::<Test>::iter().collect::<Vec<_>>(),
PreimageFor::<Test>::iter().collect::<Vec<_>>(),
)
});
new_test_ext().execute_with(|| {
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(1), vec![1]));
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(2), vec![1]));
assert_ok!(Preimage::request_preimage(RuntimeOrigin::signed(1), hashed([1])));
assert_ok!(Preimage::unnote_preimage(RuntimeOrigin::signed(2), hashed([1])));
let other_way = (
StatusFor::<Test>::iter().collect::<Vec<_>>(),
PreimageFor::<Test>::iter().collect::<Vec<_>>(),
@@ -189,6 +213,7 @@ fn request_user_note_order_makes_no_difference() {
new_test_ext().execute_with(|| {
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(2), vec![1]));
assert_ok!(Preimage::request_preimage(RuntimeOrigin::signed(1), hashed([1])));
assert_ok!(Preimage::unnote_preimage(RuntimeOrigin::signed(2), hashed([1])));
let other_way = (
StatusFor::<Test>::iter().collect::<Vec<_>>(),
PreimageFor::<Test>::iter().collect::<Vec<_>>(),
@@ -226,8 +251,240 @@ fn user_noted_then_requested_preimage_is_refunded_once_only() {
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(2), vec![1]));
assert_ok!(Preimage::request_preimage(RuntimeOrigin::signed(1), hashed([1])));
assert_ok!(Preimage::unrequest_preimage(RuntimeOrigin::signed(1), hashed([1])));
assert_ok!(Preimage::unnote_preimage(RuntimeOrigin::signed(2), hashed([1])));
// Still have reserve from `vec[1; 3]`.
assert_eq!(Balances::reserved_balance(2), 5);
assert_eq!(Balances::free_balance(2), 95);
});
}
#[test]
fn noted_preimage_use_correct_map() {
new_test_ext().execute_with(|| {
// Add one preimage per bucket...
for i in 0..7 {
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(1), vec![0; 128 << (i * 2)]));
}
assert_ok!(Preimage::note_preimage(RuntimeOrigin::signed(1), vec![0; MAX_SIZE as usize]));
assert_eq!(PreimageFor::<Test>::iter().count(), 8);
// All are present
assert_eq!(StatusFor::<Test>::iter().count(), 8);
// Now start removing them again...
for i in 0..7 {
assert_ok!(Preimage::unnote_preimage(
RuntimeOrigin::signed(1),
hashed(vec![0; 128 << (i * 2)])
));
}
assert_eq!(PreimageFor::<Test>::iter().count(), 1);
assert_ok!(Preimage::unnote_preimage(
RuntimeOrigin::signed(1),
hashed(vec![0; MAX_SIZE as usize])
));
assert_eq!(PreimageFor::<Test>::iter().count(), 0);
// All are gone
assert_eq!(StatusFor::<Test>::iter().count(), 0);
});
}
/// The `StorePreimage` and `QueryPreimage` traits work together.
#[test]
fn query_and_store_preimage_workflow() {
new_test_ext().execute_with(|| {
let _guard = StorageNoopGuard::default();
let data: Vec<u8> = vec![1; 512];
let encoded = data.encode();
// Bound an unbound value.
let bound = Preimage::bound(data.clone()).unwrap();
let (len, hash) = (bound.len().unwrap(), bound.hash());
assert_eq!(hash, blake2_256(&encoded).into());
assert_eq!(bound.len(), Some(len));
assert!(bound.lookup_needed(), "Should not be Inlined");
assert_eq!(bound.lookup_len(), Some(len));
// The value is requested and available.
assert!(Preimage::is_requested(&hash));
assert!(<Preimage as QueryPreimage>::have(&bound));
assert_eq!(Preimage::len(&hash), Some(len));
// It can be fetched with length.
assert_eq!(Preimage::fetch(&hash, Some(len)).unwrap(), encoded);
// ... and without length.
assert_eq!(Preimage::fetch(&hash, None).unwrap(), encoded);
// ... but not with wrong length.
assert_err!(Preimage::fetch(&hash, Some(0)), DispatchError::Unavailable);
// It can be peeked and decoded correctly.
assert_eq!(Preimage::peek::<Vec<u8>>(&bound).unwrap(), (data.clone(), Some(len)));
// Request it two more times.
assert_eq!(Preimage::pick::<Vec<u8>>(hash, len), bound);
Preimage::request(&hash);
// It is requested thrice.
assert!(matches!(
StatusFor::<Test>::get(&hash).unwrap(),
RequestStatus::Requested { count: 3, .. }
));
// It can be realized and decoded correctly.
assert_eq!(Preimage::realize::<Vec<u8>>(&bound).unwrap(), (data.clone(), Some(len)));
assert!(matches!(
StatusFor::<Test>::get(&hash).unwrap(),
RequestStatus::Requested { count: 2, .. }
));
// Dropping should unrequest.
Preimage::drop(&bound);
assert!(matches!(
StatusFor::<Test>::get(&hash).unwrap(),
RequestStatus::Requested { count: 1, .. }
));
// Is still available.
assert!(<Preimage as QueryPreimage>::have(&bound));
// Manually unnote it.
Preimage::unnote(&hash);
// Is not available anymore.
assert!(!<Preimage as QueryPreimage>::have(&bound));
assert_err!(Preimage::fetch(&hash, Some(len)), DispatchError::Unavailable);
// And not requested since the traits assume permissioned origin.
assert!(!Preimage::is_requested(&hash));
// No storage changes remain. Checked by `StorageNoopGuard`.
});
}
/// The request function behaves as expected.
#[test]
fn query_preimage_request_works() {
new_test_ext().execute_with(|| {
let _guard = StorageNoopGuard::default();
let data: Vec<u8> = vec![1; 10];
let hash: PreimageHash = blake2_256(&data[..]).into();
// Request the preimage.
<Preimage as QueryPreimage>::request(&hash);
// The preimage is requested with unknown length and cannot be fetched.
assert!(<Preimage as QueryPreimage>::is_requested(&hash));
assert!(<Preimage as QueryPreimage>::len(&hash).is_none());
assert_noop!(<Preimage as QueryPreimage>::fetch(&hash, None), DispatchError::Unavailable);
// Request again.
<Preimage as QueryPreimage>::request(&hash);
// The preimage is still requested.
assert!(<Preimage as QueryPreimage>::is_requested(&hash));
assert!(<Preimage as QueryPreimage>::len(&hash).is_none());
assert_noop!(<Preimage as QueryPreimage>::fetch(&hash, None), DispatchError::Unavailable);
// But there is only one entry in the map.
assert_eq!(StatusFor::<Test>::iter().count(), 1);
// Un-request the preimage.
<Preimage as QueryPreimage>::unrequest(&hash);
// It is still requested.
assert!(<Preimage as QueryPreimage>::is_requested(&hash));
// Un-request twice.
<Preimage as QueryPreimage>::unrequest(&hash);
// It is not requested anymore.
assert!(!<Preimage as QueryPreimage>::is_requested(&hash));
// And there is no entry in the map.
assert_eq!(StatusFor::<Test>::iter().count(), 0);
});
}
/// The `QueryPreimage` functions can be used together with `Bounded` values.
#[test]
fn query_preimage_hold_and_drop_work() {
new_test_ext().execute_with(|| {
let _guard = StorageNoopGuard::default();
let (inline, lookup, legacy) = make_bounded_values();
// `hold` does nothing for `Inline` values.
assert_storage_noop!(<Preimage as QueryPreimage>::hold(&inline));
// `hold` requests `Lookup` values.
<Preimage as QueryPreimage>::hold(&lookup);
assert!(<Preimage as QueryPreimage>::is_requested(&lookup.hash()));
// `hold` requests `Legacy` values.
<Preimage as QueryPreimage>::hold(&legacy);
assert!(<Preimage as QueryPreimage>::is_requested(&legacy.hash()));
// There are two values requested in total.
assert_eq!(StatusFor::<Test>::iter().count(), 2);
// Cleanup by dropping both.
<Preimage as QueryPreimage>::drop(&lookup);
assert!(!<Preimage as QueryPreimage>::is_requested(&lookup.hash()));
<Preimage as QueryPreimage>::drop(&legacy);
assert!(!<Preimage as QueryPreimage>::is_requested(&legacy.hash()));
// There are no values requested anymore.
assert_eq!(StatusFor::<Test>::iter().count(), 0);
});
}
/// The `StorePreimage` trait works as expected.
#[test]
fn store_preimage_basic_works() {
new_test_ext().execute_with(|| {
let _guard = StorageNoopGuard::default();
let data: Vec<u8> = vec![1; 512]; // Too large to inline.
let encoded = Cow::from(data.encode());
// Bound the data.
let bound = <Preimage as StorePreimage>::bound(data.clone()).unwrap();
// The preimage can be peeked.
assert_ok!(<Preimage as QueryPreimage>::peek(&bound));
// Un-note the preimage.
<Preimage as StorePreimage>::unnote(&bound.hash());
// The preimage cannot be peeked anymore.
assert_err!(<Preimage as QueryPreimage>::peek(&bound), DispatchError::Unavailable);
// Noting the wrong pre-image does not make it peek-able.
assert_ok!(<Preimage as StorePreimage>::note(Cow::Borrowed(&data)));
assert_err!(<Preimage as QueryPreimage>::peek(&bound), DispatchError::Unavailable);
// Manually note the preimage makes it peek-able again.
assert_ok!(<Preimage as StorePreimage>::note(encoded.clone()));
// Noting again works.
assert_ok!(<Preimage as StorePreimage>::note(encoded));
assert_ok!(<Preimage as QueryPreimage>::peek(&bound));
// Cleanup.
<Preimage as StorePreimage>::unnote(&bound.hash());
let data_hash = blake2_256(&data);
<Preimage as StorePreimage>::unnote(&data_hash.into());
// No storage changes remain. Checked by `StorageNoopGuard`.
});
}
#[test]
fn store_preimage_note_too_large_errors() {
new_test_ext().execute_with(|| {
// Works with `MAX_LENGTH`.
let len = <Preimage as StorePreimage>::MAX_LENGTH;
let data = vec![0u8; len];
assert_ok!(<Preimage as StorePreimage>::note(data.into()));
// Errors with `MAX_LENGTH+1`.
let data = vec![0u8; len + 1];
assert_err!(<Preimage as StorePreimage>::note(data.into()), DispatchError::Exhausted);
});
}
#[test]
fn store_preimage_bound_too_large_errors() {
new_test_ext().execute_with(|| {
// Using `MAX_LENGTH` number of bytes in a vector does not work
// since SCALE prepends the length.
let len = <Preimage as StorePreimage>::MAX_LENGTH;
let data: Vec<u8> = vec![0; len];
assert_err!(<Preimage as StorePreimage>::bound(data.clone()), DispatchError::Exhausted);
// Works with `MAX_LENGTH-4`.
let data: Vec<u8> = vec![0; len - 4];
assert_ok!(<Preimage as StorePreimage>::bound(data.clone()));
});
}