* Create a macro which automates creation of benchmark test suites.
* bump impl_version
* allow unused on test_bench_by_name
* use proper doctest ignore attribute
* Explicitly hand the Module to the test suite
Much better practice than depending on it showing up implicitly in
the namespace.
* explicitly import what we need into `mod tests`
* bench_module is `ident` not `tt`
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* allow end users to specify arguments for new_test_ext
This turned out to be surprisingly easy. On reflection, it turns out
that of course the compiler can't eagerly evaluate the function call,
but needs to paste it in everywhere desired.
* enable explicitly specifying the path to the benchmarks invocation
also enable optional trailing commas
* Revert "bump impl_version"
This reverts commit 0209e4de33fd43873f8cfc6875815d0fd6151e63.
* list failing benchmark tests and the errors which caused the failure
* harden benchmark tests against internal panics
* suppress warning about ignored profiles
unfortunately, setting the profile here doesn't do anything; we'd
need to set it in every leaf package anyway. However, as this was
just making the default explicit anyway, I think it's safe enough
to remove entirely.
* impl_benchmark_test_suite for assets
* impl_benchmark_test_suite for balances
* impl_benchmark_test_suite for bounties
* impl_benchmark_test_suite for Collective
* impl_benchmark_test_suite for Contracts
* impl_benchmark_test_suite for Democracy
* don't impl_benchmark_test_suite for Elections-Phragmen
* impl_benchmark_test_suite for Identity
Note that Identity tests currently fail. They failed in an identical
way before this change, so as far as I'm concerned, the status quo is
good enough for now.
* impl_benchmark_test_suite for ImOnline
* impl_benchmark_test_suite for indices
For this crate also, the test suite fails identically with and without
this change, so we can say that this change is not the cause of the
tests' failure to compile.
* impl_benchmark_test_suite for lottery
* impl_benchmark_test_suite for merkle-mountain-range
* impl_benchmark_test_suite for Multisig
These tests fail identically with and without the change, so the change
seems unlikely to be the origin of the failures.
* impl_benchmark_test_suite for offences
* impl_benchmark_test_suite for Proxy
Fails identically with and without this change.
* impl_benchmark_test_suite for scheduler
* impl_benchmark_test_suite for session
It turns out to be important to be able to exclude items marked
`#[extra]` sometimes. Who knew?
* impl_benchmark_test_suite for staking
* impl_benchmark_test_suite for system
* impl_benchmark_test_suite for timestamp
* impl_benchmark_test_suite for tips
* impl_benchmark_test_suite for treasury
* impl_benchmark_test_suite for utility
Note that benchmark tests fail identically before and after this change.
* impl_benchmark_test_suite for vesting
* fix wrong module name in impl_benchmark_test_suite in Offences
* address line length nits
* enable optional keyword argument: exec_name
Took a _lot_ of macro-wrangling to get the functionality that I want,
but now you have the option to pass in
```rust
impl_benchmark_test_suite!(
Elections,
crate::tests::ExtBuilder::default().desired_members(13).desired_runners_up(7),
crate::tests::Test,
exec_name = build_and_execute,
);
```
and have it expand out properly. A selected fragment of the expansion:
```rust
fn test_benchmarks() {
crate::tests::ExtBuilder::default()
.desired_members(13)
.desired_runners_up(7)
.build_and_execute(|| {
```
* get rid of dead code
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* commented use of common
* hack to pass tests
* another hack
* remove all commented code
* fix the easy tests
* temp hack
* follow through comma hack until better solution
* patch macro
* missed one
* update benchmarks
* update docs
* fix docs
* removed too much
* fix changes
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Andrew Jones