* Simplify some Option / Result / ? operator patterns
When those match a combinator exactly.
Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
* adjust after review
* adjust post-review
* Remove `serde` requirement from FRAME macros
Currently there is some implicit requirement on `serde` being present in
the `Cargo.toml` of a pallet when `GenesisConfig` is used. This pr
removes this requirement by using the serde attribute `serde(crate = "..")`.
* build a unique reexport of serde in impl_opaque_keys, by abusing paste doc concatenation
* Optimize
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
* Base features and traits.
* pallet and unsigned phase
* Undo bad formattings.
* some formatting cleanup.
* Small self-cleanup.
* Make it all build
* self-review
* Some doc tests.
* Some changes from other PR
* Fix session test
* Update Cargo.lock
* Update frame/election-provider-multi-phase/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Some review comments
* Rename + make encode/decode
* Do an assert as well, just in case.
* Fix build
* Update frame/election-provider-multi-phase/src/unsigned.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Las comment
* fix staking fuzzer.
* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Add one last layer of feasibility check as well.
* Last fixes to benchmarks
* Some more docs.
* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Some nits
* It all works
* Some self cleanup
* Update frame/staking/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* remove most todos.
* Round of self-review.
* Fix migration
* clean macro
* Revert wrong merge
* Make the number of nominations configurable
* Self reivew
* renmae.
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Use 'Pallet' struct in construct_runtime.
* Fix genesis and metadata macro.
* Fix 'Pallet' type alias.
* Replace 'Module' with 'Pallet' for all construct_runtime use cases.
* Replace more deprecated 'Module' struct.
* Bring back AllModules and AllPalletsWithSystem type, but deprecate them.
* Replace deprecated 'Module' struct from merge master.
* Minor fix.
* Fix UI tests.
* Revert UI override in derive_no_bound.
* Fix more deprecated 'Module' use from master branch.
* Fix more deprecated 'Module' use from master branch.
* use more efficient search through candidates in offchain-election
* mark linear accessors as test-only in election-provider-multi-phase
This prevents production code which uses them from compiling.
Also write an efficient helper for getting the target index.
* doc grammar
* use faster target_index_fn in benchmarks
* unbox helper functions
* remove unnecessary import
* write lifetime after primary trait
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Don't log so many stkaing events on genesis block
* Clean a few more warn and info logs
* Update frame/staking/src/lib.rs
* Update frame/staking/src/lib.rs
* Apply.
* get rid of glob import
* use meaningful generic type name
* pjr_check operates on `Supports` struct used elsewhere
* improve algorithmic complexity of `prepare_pjr_input`
* fix rustdoc warnings
* improve module docs
* typo
* simplify debug assertion
* add test finding the phase-change threshold value for a constructed scenario
* add more threshold scenarios to disambiguate plausible interpretations
* add link to npos paper reference
* docs: staked_assignment -> supports
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* add utility method for generating npos inputs
* add a fuzzer which asserts that all unbalanced seq_phragmen are PJR
Note that this currently fails. I hope that this can be rectified
by calculating the threshold instead of choosing some arbitrary number.
* assert in all cases, not just debug
* leverage a native solution to choose candidates
* use existing helper methods
* add pjr-check and incorporate into the fuzzer
We should probably have one of the W3F people look at this to ensure
we're not misconstruing any definitions, but this seems like a
fairly straightforward implementation.
* fix compilation errors
* Enable manually setting iteration parameters in single run.
This gives us the ability to reproducably extract cases where
honggfuzz has discovered a panic. For example:
$ cargo run --release --bin phragmen_pjr -- --candidates 569 --voters 100
Tue 23 Feb 2021 11:23:39 AM CET
Compiling bitflags v1.2.1
Compiling unicode-width v0.1.8
Compiling unicode-segmentation v1.7.1
Compiling ansi_term v0.11.0
Compiling strsim v0.8.0
Compiling vec_map v0.8.2
Compiling proc-macro-error-attr v1.0.4
Compiling proc-macro-error v1.0.4
Compiling textwrap v0.11.0
Compiling atty v0.2.14
Compiling heck v0.3.2
Compiling clap v2.33.3
Compiling structopt-derive v0.4.14
Compiling structopt v0.3.21
Compiling sp-npos-elections-fuzzer v2.0.0-alpha.5 (/home/coriolinus/Documents/Projects/paritytech/substrate/primitives/npos-elections/fuzzer)
Finished release [optimized] target(s) in 6.15s
Running `/home/coriolinus/Documents/Projects/paritytech/substrate/target/release/phragmen_pjr -c 569 -v 100`
thread 'main' panicked at 'unbalanced sequential phragmen must satisfy PJR', primitives/npos-elections/fuzzer/src/phragmen_pjr.rs:133:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
This is still not adequate proof that seq_phragmen is broken; it could
very well be that our PJR checker is doing the wrong thing, or we've
somehow missed a parameter of interest. Still, it's concerning.
* update comment verbiage for accuracy
* it is valid in PJR for an elected candidate to have 0 support
* Fix phragmen_pjr fuzzer
It turns out that the fundamental problem causing previous implementations
of the fuzzer to fail wasn't in `seq_phragmen` _or_ in `pjr_check`: it was
in the rounding errors introduced in the various conversions between the
internal data representation and the external one.
Fixing the fuzzer is then simply an issue of using the internal representation
and staying in that representation. However, that leaves the issue that
`seq_phragmen` occasionally produces an output which is technically not
PJR due to rounding errors. In the future we will need to add some kind of
"close-enough" threshold. However, that is explicitly out of scope of
this PR.
* restart ci; it appears to be stalled
* use necessary import for no-std
* use a more realistic distribution of voters and candidates
This isn't ideal; more realistic numbers would be about twice these.
However, either case generation or voting has nonlinear execution
time, and doubling these values brings iteration time from ~20s to
~180s. Fuzzing 6x as fast should make up for fuzzing cases half the size.
* identify specifically which PJR check may fail
* move candidate collection comment into correct place
* standard_threshold: use a calculation method which cannot overflow
* Apply suggestions from code review (update comments)
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* clarify the effectiveness bounds for t-pjr check
* how to spell "committee"
* reorganize: high -> low abstraction
* ensure standard threshold calc cannot panic
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: kianenigma <kian.peymani@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Add MMR custom RPC.
* Change RuntimeApi to avoid hardcoding leaf type.
* Properly implement the new RuntimeAPI and wire up RPC.
* Extract Offchain DB as separate execution extension.
* Enable offchain DB access for offchain calls.
* Fix offchain_election tests.
* Skip block initialisation for proof generation.
* Fix integration test setup.
* Fix offchain tests. Not sure how I missed them earlier 🤷.
* Fix long line.
* One more test missing.
* Update mock for multi-phase.
* Address review grumbbles.
* Address review grumbles.
* Fix line width of a comment
* Init `RuntimeLogger` automatically for each runtime api call
This pr change the runtime api in such a way to always and automatically
enable the `RuntimeLogger`. This enables the user to use `log` or
`tracing` from inside the runtime to create log messages. As logging
introduces some extra code and especially increases the size of the wasm
blob. It is advised to disable all logging completely with
`sp-api/disable-logging` when doing the wasm builds for the on-chain
wasm runtime.
Besides these changes, the pr also brings most of the logging found in
frame to the same format "runtime::*".
* Update frame/im-online/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update test-utils/runtime/Cargo.toml
* Fix test
* Don't use tracing in the runtime, as we don't support it :D
* Fixes
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Create a macro which automates creation of benchmark test suites.
* bump impl_version
* allow unused on test_bench_by_name
* use proper doctest ignore attribute
* Explicitly hand the Module to the test suite
Much better practice than depending on it showing up implicitly in
the namespace.
* explicitly import what we need into `mod tests`
* bench_module is `ident` not `tt`
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* allow end users to specify arguments for new_test_ext
This turned out to be surprisingly easy. On reflection, it turns out
that of course the compiler can't eagerly evaluate the function call,
but needs to paste it in everywhere desired.
* enable explicitly specifying the path to the benchmarks invocation
also enable optional trailing commas
* Revert "bump impl_version"
This reverts commit 0209e4de33fd43873f8cfc6875815d0fd6151e63.
* list failing benchmark tests and the errors which caused the failure
* harden benchmark tests against internal panics
* suppress warning about ignored profiles
unfortunately, setting the profile here doesn't do anything; we'd
need to set it in every leaf package anyway. However, as this was
just making the default explicit anyway, I think it's safe enough
to remove entirely.
* impl_benchmark_test_suite for assets
* impl_benchmark_test_suite for balances
* impl_benchmark_test_suite for bounties
* impl_benchmark_test_suite for Collective
* impl_benchmark_test_suite for Contracts
* impl_benchmark_test_suite for Democracy
* don't impl_benchmark_test_suite for Elections-Phragmen
* impl_benchmark_test_suite for Identity
Note that Identity tests currently fail. They failed in an identical
way before this change, so as far as I'm concerned, the status quo is
good enough for now.
* impl_benchmark_test_suite for ImOnline
* impl_benchmark_test_suite for indices
For this crate also, the test suite fails identically with and without
this change, so we can say that this change is not the cause of the
tests' failure to compile.
* impl_benchmark_test_suite for lottery
* impl_benchmark_test_suite for merkle-mountain-range
* impl_benchmark_test_suite for Multisig
These tests fail identically with and without the change, so the change
seems unlikely to be the origin of the failures.
* impl_benchmark_test_suite for offences
* impl_benchmark_test_suite for Proxy
Fails identically with and without this change.
* impl_benchmark_test_suite for scheduler
* impl_benchmark_test_suite for session
It turns out to be important to be able to exclude items marked
`#[extra]` sometimes. Who knew?
* impl_benchmark_test_suite for staking
* impl_benchmark_test_suite for system
* impl_benchmark_test_suite for timestamp
* impl_benchmark_test_suite for tips
* impl_benchmark_test_suite for treasury
* impl_benchmark_test_suite for utility
Note that benchmark tests fail identically before and after this change.
* impl_benchmark_test_suite for vesting
* fix wrong module name in impl_benchmark_test_suite in Offences
* address line length nits
* enable optional keyword argument: exec_name
Took a _lot_ of macro-wrangling to get the functionality that I want,
but now you have the option to pass in
```rust
impl_benchmark_test_suite!(
Elections,
crate::tests::ExtBuilder::default().desired_members(13).desired_runners_up(7),
crate::tests::Test,
exec_name = build_and_execute,
);
```
and have it expand out properly. A selected fragment of the expansion:
```rust
fn test_benchmarks() {
crate::tests::ExtBuilder::default()
.desired_members(13)
.desired_runners_up(7)
.build_and_execute(|| {
```
* get rid of dead code
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Decouple the session validators from im-online
* .
* Add SessionInterface trait in im-online
Add ValidatorId in im-online Trait
Make im-online compile
Make substrate binary compile
* Fix merging issue
* Make all compile
* Fix tests
* Avoid using frame dep in primitives via pallet-session-common
* Merge ValidatorSet into SessionInterface trait
Wrap a few too long lines
Add some docs
* Move pallet-sesion-common into pallet-session
* Move SessionInterface to sp-session and impl it in session pallet
Ref https://github.com/paritytech/substrate/pull/7127#discussion_r494892472
* Split put historical::FullValidatorIdentification trait
* Fix line width
* Fix staking mock
* Fix session doc test
* Simplify <T as ValidatorIdentification<AccountId>>::ValidatorId as ValidatorId<T>
* Nits
* Clean up.
* Make it compile by commenting out report_offence_im_online bench
* Tests
* Nits
* Move OneSessionHandler to sp-session
* Fix tests
* Add some docs
* .
* Fix typo
* Rename to ValidatorSet::session_index()
* Add some more docs
* .
* Remove extra empty line
* Fix line width check
.
* Apply suggestions from code review
* Cleaup Cargo.toml
* Aura has migrated to Pallet now
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
* Attempt to remove the where bounds.
* Fix further and further.
* Format better.
* Update primitives/npos-elections/src/lib.rs
* fix build
* remove unused
* Fix incorrect use of syn::exports
Instead of using `syn::exports` we should import the trait from the
quote crate directly.
* Use own macro for test cases to fix compilation with latest syn
* Fix test
* upgrade a few dependencies
* make it compile at the expense of duplicate deps
* fix web-wasm and a warning
* introduce activate-wasm-bindgen-features crate
* Revert "introduce activate-wasm-bindgen-features crate"
This reverts commit 5a6e41e683f8a4844c0a735dcd08caabb2313f11.
* add getrandom feature to sc-consensus-aura
* commented use of common
* hack to pass tests
* another hack
* remove all commented code
* fix the easy tests
* temp hack
* follow through comma hack until better solution
* patch macro
* missed one
* update benchmarks
* update docs
* fix docs
* removed too much
* fix changes
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Lohann Paterno Coutinho Ferreira