Files
pezkuwi-subxt/substrate/frame/collective/src/benchmarking.rs
T
Peter Goodspeed-Niklaus 44d5aba80d Create a macro which automates creation of benchmark test suites. (#8104)
* Create a macro which automates creation of benchmark test suites.

* bump impl_version

* allow unused on test_bench_by_name

* use proper doctest ignore attribute

* Explicitly hand the Module to the test suite

Much better practice than depending on it showing up implicitly in
the namespace.

* explicitly import what we need into `mod tests`

* bench_module is `ident` not `tt`

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* allow end users to specify arguments for new_test_ext

This turned out to be surprisingly easy. On reflection, it turns out
that of course the compiler can't eagerly evaluate the function call,
but needs to paste it in everywhere desired.

* enable explicitly specifying the path to the benchmarks invocation

also enable optional trailing commas

* Revert "bump impl_version"

This reverts commit 0209e4de33fd43873f8cfc6875815d0fd6151e63.

* list failing benchmark tests and the errors which caused the failure

* harden benchmark tests against internal panics

* suppress warning about ignored profiles

unfortunately, setting the profile here doesn't do anything; we'd
need to set it in every leaf package anyway. However, as this was
just making the default explicit anyway, I think it's safe enough
to remove entirely.

* impl_benchmark_test_suite for assets

* impl_benchmark_test_suite for balances

* impl_benchmark_test_suite for bounties

* impl_benchmark_test_suite for Collective

* impl_benchmark_test_suite for Contracts

* impl_benchmark_test_suite for Democracy

* don't impl_benchmark_test_suite for Elections-Phragmen

* impl_benchmark_test_suite for Identity

Note that Identity tests currently fail. They failed in an identical
way before this change, so as far as I'm concerned, the status quo is
good enough for now.

* impl_benchmark_test_suite for ImOnline

* impl_benchmark_test_suite for indices

For this crate also, the test suite fails identically with and without
this change, so we can say that this change is not the cause of the
tests' failure to compile.

* impl_benchmark_test_suite for lottery

* impl_benchmark_test_suite for merkle-mountain-range

* impl_benchmark_test_suite for Multisig

These tests fail identically with and without the change, so the change
seems unlikely to be the origin of the failures.

* impl_benchmark_test_suite for offences

* impl_benchmark_test_suite for Proxy

Fails identically with and without this change.

* impl_benchmark_test_suite for scheduler

* impl_benchmark_test_suite for session

It turns out to be important to be able to exclude items marked
`#[extra]` sometimes. Who knew?

* impl_benchmark_test_suite for staking

* impl_benchmark_test_suite for system

* impl_benchmark_test_suite for timestamp

* impl_benchmark_test_suite for tips

* impl_benchmark_test_suite for treasury

* impl_benchmark_test_suite for utility

Note that benchmark tests fail identically before and after this change.

* impl_benchmark_test_suite for vesting

* fix wrong module name in impl_benchmark_test_suite in Offences

* address line length nits

* enable optional keyword argument: exec_name

Took a _lot_ of macro-wrangling to get the functionality that I want,
but now you have the option to pass in

```rust
impl_benchmark_test_suite!(
	Elections,
	crate::tests::ExtBuilder::default().desired_members(13).desired_runners_up(7),
	crate::tests::Test,
	exec_name = build_and_execute,
);
```

and have it expand out properly. A selected fragment of the expansion:

```rust
        fn test_benchmarks() {
            crate::tests::ExtBuilder::default()
                .desired_members(13)
                .desired_runners_up(7)
                .build_and_execute(|| {
```

* get rid of dead code

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-02-16 10:01:20 +01:00

646 lines
20 KiB
Rust

// This file is part of Substrate.
// Copyright (C) 2020-2021 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Staking pallet benchmarking.
use super::*;
use frame_system::RawOrigin as SystemOrigin;
use frame_system::EventRecord;
use frame_benchmarking::{
benchmarks_instance,
account,
whitelisted_caller,
impl_benchmark_test_suite,
};
use sp_runtime::traits::Bounded;
use sp_std::mem::size_of;
use frame_system::Call as SystemCall;
use frame_system::Module as System;
use crate::Module as Collective;
const SEED: u32 = 0;
const MAX_BYTES: u32 = 1_024;
fn assert_last_event<T: Config<I>, I: Instance>(generic_event: <T as Config<I>>::Event) {
let events = System::<T>::events();
let system_event: <T as frame_system::Config>::Event = generic_event.into();
// compare to the last event record
let EventRecord { event, .. } = &events[events.len() - 1];
assert_eq!(event, &system_event);
}
benchmarks_instance! {
set_members {
let m in 1 .. T::MaxMembers::get();
let n in 1 .. T::MaxMembers::get();
let p in 1 .. T::MaxProposals::get();
// Set old members.
// We compute the difference of old and new members, so it should influence timing.
let mut old_members = vec![];
let mut last_old_member = T::AccountId::default();
for i in 0 .. m {
last_old_member = account("old member", i, SEED);
old_members.push(last_old_member.clone());
}
let old_members_count = old_members.len() as u32;
Collective::<T, _>::set_members(
SystemOrigin::Root.into(),
old_members.clone(),
Some(last_old_member.clone()),
T::MaxMembers::get(),
)?;
// Set a high threshold for proposals passing so that they stay around.
let threshold = m.max(2);
// Length of the proposals should be irrelevant to `set_members`.
let length = 100;
for i in 0 .. p {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; length]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(last_old_member.clone()).into(),
threshold,
Box::new(proposal.clone()),
MAX_BYTES,
)?;
let hash = T::Hashing::hash_of(&proposal);
// Vote on the proposal to increase state relevant for `set_members`.
// Not voting for `last_old_member` because they proposed and not voting for the first member
// to keep the proposal from passing.
for j in 2 .. m - 1 {
let voter = &old_members[j as usize];
let approve = true;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
hash,
i,
approve,
)?;
}
}
// Construct `new_members`.
// It should influence timing since it will sort this vector.
let mut new_members = vec![];
let mut last_member = T::AccountId::default();
for i in 0 .. n {
last_member = account("member", i, SEED);
new_members.push(last_member.clone());
}
}: _(SystemOrigin::Root, new_members.clone(), Some(last_member), T::MaxMembers::get())
verify {
new_members.sort();
assert_eq!(Collective::<T, _>::members(), new_members);
}
execute {
let b in 1 .. MAX_BYTES;
let m in 1 .. T::MaxMembers::get();
let bytes_in_storage = b + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
for i in 0 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let caller: T::AccountId = whitelisted_caller();
members.push(caller.clone());
Collective::<T, _>::set_members(SystemOrigin::Root.into(), members, None, T::MaxMembers::get())?;
let proposal: T::Proposal = SystemCall::<T>::remark(vec![1; b as usize]).into();
}: _(SystemOrigin::Signed(caller), Box::new(proposal.clone()), bytes_in_storage)
verify {
let proposal_hash = T::Hashing::hash_of(&proposal);
// Note that execution fails due to mis-matched origin
assert_last_event::<T, I>(
RawEvent::MemberExecuted(proposal_hash, Err(DispatchError::BadOrigin)).into()
);
}
// This tests when execution would happen immediately after proposal
propose_execute {
let b in 1 .. MAX_BYTES;
let m in 1 .. T::MaxMembers::get();
let bytes_in_storage = b + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
for i in 0 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let caller: T::AccountId = whitelisted_caller();
members.push(caller.clone());
Collective::<T, _>::set_members(SystemOrigin::Root.into(), members, None, T::MaxMembers::get())?;
let proposal: T::Proposal = SystemCall::<T>::remark(vec![1; b as usize]).into();
let threshold = 1;
}: propose(SystemOrigin::Signed(caller), threshold, Box::new(proposal.clone()), bytes_in_storage)
verify {
let proposal_hash = T::Hashing::hash_of(&proposal);
// Note that execution fails due to mis-matched origin
assert_last_event::<T, I>(
RawEvent::Executed(proposal_hash, Err(DispatchError::BadOrigin)).into()
);
}
// This tests when proposal is created and queued as "proposed"
propose_proposed {
let b in 1 .. MAX_BYTES;
let m in 2 .. T::MaxMembers::get();
let p in 1 .. T::MaxProposals::get();
let bytes_in_storage = b + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
for i in 0 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let caller: T::AccountId = whitelisted_caller();
members.push(caller.clone());
Collective::<T, _>::set_members(SystemOrigin::Root.into(), members, None, T::MaxMembers::get())?;
let threshold = m;
// Add previous proposals.
for i in 0 .. p - 1 {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; b as usize]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(caller.clone()).into(),
threshold,
Box::new(proposal),
bytes_in_storage,
)?;
}
assert_eq!(Collective::<T, _>::proposals().len(), (p - 1) as usize);
let proposal: T::Proposal = SystemCall::<T>::remark(vec![p as u8; b as usize]).into();
}: propose(SystemOrigin::Signed(caller.clone()), threshold, Box::new(proposal.clone()), bytes_in_storage)
verify {
// New proposal is recorded
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
let proposal_hash = T::Hashing::hash_of(&proposal);
assert_last_event::<T, I>(RawEvent::Proposed(caller, p - 1, proposal_hash, threshold).into());
}
vote {
// We choose 5 as a minimum so we always trigger a vote in the voting loop (`for j in ...`)
let m in 5 .. T::MaxMembers::get();
let p = T::MaxProposals::get();
let b = MAX_BYTES;
let bytes_in_storage = b + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
let proposer: T::AccountId = account("proposer", 0, SEED);
members.push(proposer.clone());
for i in 1 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let voter: T::AccountId = account("voter", 0, SEED);
members.push(voter.clone());
Collective::<T, _>::set_members(SystemOrigin::Root.into(), members.clone(), None, T::MaxMembers::get())?;
// Threshold is 1 less than the number of members so that one person can vote nay
let threshold = m - 1;
// Add previous proposals
let mut last_hash = T::Hash::default();
for i in 0 .. p {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; b as usize]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(proposer.clone()).into(),
threshold,
Box::new(proposal.clone()),
bytes_in_storage,
)?;
last_hash = T::Hashing::hash_of(&proposal);
}
let index = p - 1;
// Have almost everyone vote aye on last proposal, while keeping it from passing.
// Proposer already voted aye so we start at 1.
for j in 1 .. m - 3 {
let voter = &members[j as usize];
let approve = true;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
index,
approve,
)?;
}
// Voter votes aye without resolving the vote.
let approve = true;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
index,
approve,
)?;
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
// Voter switches vote to nay, but does not kill the vote, just updates + inserts
let approve = false;
// Whitelist voter account from further DB operations.
let voter_key = frame_system::Account::<T>::hashed_key_for(&voter);
frame_benchmarking::benchmarking::add_to_whitelist(voter_key.into());
}: _(SystemOrigin::Signed(voter), last_hash.clone(), index, approve)
verify {
// All proposals exist and the last proposal has just been updated.
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
let voting = Collective::<T, _>::voting(&last_hash).ok_or(Error::<T, I>::ProposalMissing)?;
assert_eq!(voting.ayes.len(), (m - 3) as usize);
assert_eq!(voting.nays.len(), 1);
}
close_early_disapproved {
// We choose 4 as a minimum so we always trigger a vote in the voting loop (`for j in ...`)
let m in 4 .. T::MaxMembers::get();
let p in 1 .. T::MaxProposals::get();
let bytes = 100;
let bytes_in_storage = bytes + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
let proposer: T::AccountId = account("proposer", 0, SEED);
members.push(proposer.clone());
for i in 1 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let voter: T::AccountId = account("voter", 0, SEED);
members.push(voter.clone());
Collective::<T, _>::set_members(SystemOrigin::Root.into(), members.clone(), None, T::MaxMembers::get())?;
// Threshold is total members so that one nay will disapprove the vote
let threshold = m;
// Add previous proposals
let mut last_hash = T::Hash::default();
for i in 0 .. p {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; bytes as usize]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(proposer.clone()).into(),
threshold,
Box::new(proposal.clone()),
bytes_in_storage,
)?;
last_hash = T::Hashing::hash_of(&proposal);
}
let index = p - 1;
// Have most everyone vote aye on last proposal, while keeping it from passing.
// Proposer already voted aye so we start at 1.
for j in 1 .. m - 2 {
let voter = &members[j as usize];
let approve = true;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
index,
approve,
)?;
}
// Voter votes aye without resolving the vote.
let approve = true;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
index,
approve,
)?;
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
// Voter switches vote to nay, which kills the vote
let approve = false;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
index,
approve,
)?;
// Whitelist voter account from further DB operations.
let voter_key = frame_system::Account::<T>::hashed_key_for(&voter);
frame_benchmarking::benchmarking::add_to_whitelist(voter_key.into());
}: close(SystemOrigin::Signed(voter), last_hash.clone(), index, Weight::max_value(), bytes_in_storage)
verify {
// The last proposal is removed.
assert_eq!(Collective::<T, _>::proposals().len(), (p - 1) as usize);
assert_last_event::<T, I>(RawEvent::Disapproved(last_hash).into());
}
close_early_approved {
let b in 1 .. MAX_BYTES;
// We choose 4 as a minimum so we always trigger a vote in the voting loop (`for j in ...`)
let m in 4 .. T::MaxMembers::get();
let p in 1 .. T::MaxProposals::get();
let bytes_in_storage = b + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
for i in 0 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let caller: T::AccountId = whitelisted_caller();
members.push(caller.clone());
Collective::<T, _>::set_members(SystemOrigin::Root.into(), members.clone(), None, T::MaxMembers::get())?;
// Threshold is 2 so any two ayes will approve the vote
let threshold = 2;
// Add previous proposals
let mut last_hash = T::Hash::default();
for i in 0 .. p {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; b as usize]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(caller.clone()).into(),
threshold,
Box::new(proposal.clone()),
bytes_in_storage,
)?;
last_hash = T::Hashing::hash_of(&proposal);
}
// Caller switches vote to nay on their own proposal, allowing them to be the deciding approval vote
Collective::<T, _>::vote(
SystemOrigin::Signed(caller.clone()).into(),
last_hash.clone(),
p - 1,
false,
)?;
// Have almost everyone vote nay on last proposal, while keeping it from failing.
for j in 2 .. m - 1 {
let voter = &members[j as usize];
let approve = false;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
p - 1,
approve,
)?;
}
// Member zero is the first aye
Collective::<T, _>::vote(
SystemOrigin::Signed(members[0].clone()).into(),
last_hash.clone(),
p - 1,
true,
)?;
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
// Caller switches vote to aye, which passes the vote
let index = p - 1;
let approve = true;
Collective::<T, _>::vote(
SystemOrigin::Signed(caller.clone()).into(),
last_hash.clone(),
index, approve,
)?;
}: close(SystemOrigin::Signed(caller), last_hash.clone(), index, Weight::max_value(), bytes_in_storage)
verify {
// The last proposal is removed.
assert_eq!(Collective::<T, _>::proposals().len(), (p - 1) as usize);
assert_last_event::<T, I>(RawEvent::Executed(last_hash, Err(DispatchError::BadOrigin)).into());
}
close_disapproved {
// We choose 4 as a minimum so we always trigger a vote in the voting loop (`for j in ...`)
let m in 4 .. T::MaxMembers::get();
let p in 1 .. T::MaxProposals::get();
let bytes = 100;
let bytes_in_storage = bytes + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
for i in 0 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let caller: T::AccountId = whitelisted_caller();
members.push(caller.clone());
Collective::<T, _>::set_members(
SystemOrigin::Root.into(),
members.clone(),
Some(caller.clone()),
T::MaxMembers::get(),
)?;
// Threshold is one less than total members so that two nays will disapprove the vote
let threshold = m - 1;
// Add proposals
let mut last_hash = T::Hash::default();
for i in 0 .. p {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; bytes as usize]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(caller.clone()).into(),
threshold,
Box::new(proposal.clone()),
bytes_in_storage,
)?;
last_hash = T::Hashing::hash_of(&proposal);
}
let index = p - 1;
// Have almost everyone vote aye on last proposal, while keeping it from passing.
// A few abstainers will be the nay votes needed to fail the vote.
for j in 2 .. m - 1 {
let voter = &members[j as usize];
let approve = true;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
index,
approve,
)?;
}
// caller is prime, prime votes nay
Collective::<T, _>::vote(
SystemOrigin::Signed(caller.clone()).into(),
last_hash.clone(),
index,
false,
)?;
System::<T>::set_block_number(T::BlockNumber::max_value());
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
// Prime nay will close it as disapproved
}: close(SystemOrigin::Signed(caller), last_hash, index, Weight::max_value(), bytes_in_storage)
verify {
assert_eq!(Collective::<T, _>::proposals().len(), (p - 1) as usize);
assert_last_event::<T, I>(RawEvent::Disapproved(last_hash).into());
}
close_approved {
let b in 1 .. MAX_BYTES;
// We choose 4 as a minimum so we always trigger a vote in the voting loop (`for j in ...`)
let m in 4 .. T::MaxMembers::get();
let p in 1 .. T::MaxProposals::get();
let bytes_in_storage = b + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
for i in 0 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let caller: T::AccountId = whitelisted_caller();
members.push(caller.clone());
Collective::<T, _>::set_members(
SystemOrigin::Root.into(),
members.clone(),
Some(caller.clone()),
T::MaxMembers::get(),
)?;
// Threshold is two, so any two ayes will pass the vote
let threshold = 2;
// Add proposals
let mut last_hash = T::Hash::default();
for i in 0 .. p {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; b as usize]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(caller.clone()).into(),
threshold,
Box::new(proposal.clone()),
bytes_in_storage,
)?;
last_hash = T::Hashing::hash_of(&proposal);
}
// Have almost everyone vote nay on last proposal, while keeping it from failing.
// A few abstainers will be the aye votes needed to pass the vote.
for j in 2 .. m - 1 {
let voter = &members[j as usize];
let approve = false;
Collective::<T, _>::vote(
SystemOrigin::Signed(voter.clone()).into(),
last_hash.clone(),
p - 1,
approve
)?;
}
// caller is prime, prime already votes aye by creating the proposal
System::<T>::set_block_number(T::BlockNumber::max_value());
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
// Prime aye will close it as approved
}: close(SystemOrigin::Signed(caller), last_hash, p - 1, Weight::max_value(), bytes_in_storage)
verify {
assert_eq!(Collective::<T, _>::proposals().len(), (p - 1) as usize);
assert_last_event::<T, I>(RawEvent::Executed(last_hash, Err(DispatchError::BadOrigin)).into());
}
disapprove_proposal {
let p in 1 .. T::MaxProposals::get();
let m = 3;
let b = MAX_BYTES;
let bytes_in_storage = b + size_of::<u32>() as u32;
// Construct `members`.
let mut members = vec![];
for i in 0 .. m - 1 {
let member = account("member", i, SEED);
members.push(member);
}
let caller: T::AccountId = account("caller", 0, SEED);
members.push(caller.clone());
Collective::<T, _>::set_members(
SystemOrigin::Root.into(),
members.clone(),
Some(caller.clone()),
T::MaxMembers::get(),
)?;
// Threshold is one less than total members so that two nays will disapprove the vote
let threshold = m - 1;
// Add proposals
let mut last_hash = T::Hash::default();
for i in 0 .. p {
// Proposals should be different so that different proposal hashes are generated
let proposal: T::Proposal = SystemCall::<T>::remark(vec![i as u8; b as usize]).into();
Collective::<T, _>::propose(
SystemOrigin::Signed(caller.clone()).into(),
threshold,
Box::new(proposal.clone()),
bytes_in_storage,
)?;
last_hash = T::Hashing::hash_of(&proposal);
}
System::<T>::set_block_number(T::BlockNumber::max_value());
assert_eq!(Collective::<T, _>::proposals().len(), p as usize);
}: _(SystemOrigin::Root, last_hash)
verify {
assert_eq!(Collective::<T, _>::proposals().len(), (p - 1) as usize);
assert_last_event::<T, I>(RawEvent::Disapproved(last_hash).into());
}
}
impl_benchmark_test_suite!(
Collective,
crate::tests::new_test_ext(),
crate::tests::Test,
);