* Better description for assert in frame-session genesis
There is an assert that checks that an account exists, after setting a
key. However, this assert isn't very self-descriptive.
* Update frame/session/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Improve logging in network gossip
This adds some more information to the logging output to get a better
understanding when something fails.
* Update client/network-gossip/src/state_machine.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Use 'Pallet' struct in construct_runtime.
* Fix genesis and metadata macro.
* Fix 'Pallet' type alias.
* Replace 'Module' with 'Pallet' for all construct_runtime use cases.
* Replace more deprecated 'Module' struct.
* Bring back AllModules and AllPalletsWithSystem type, but deprecate them.
* Replace deprecated 'Module' struct from merge master.
* Minor fix.
* Fix UI tests.
* Revert UI override in derive_no_bound.
* Fix more deprecated 'Module' use from master branch.
* Fix more deprecated 'Module' use from master branch.
* use more efficient search through candidates in offchain-election
* mark linear accessors as test-only in election-provider-multi-phase
This prevents production code which uses them from compiling.
Also write an efficient helper for getting the target index.
* doc grammar
* use faster target_index_fn in benchmarks
* unbox helper functions
* remove unnecessary import
* write lifetime after primary trait
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* primitives/runtime: initial changes on supporting multiple Justifications
* primitives/runtime: make Justifications strongly typed
* Encode/decode Justifications
* primitives/runtime: add Justification type
* backend: apply_finality and finalize_block takes a single Justification
* manual-seal: create engine id and let rpc take encoded justification
* backend: skeleton functions for appending justifications
* backend: initial implementation append_justification
Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k
* backend: guard against duplicate consensus engine id
* client/db: add check for block finality
* client/api: add append_justification to in_mem db
* client/light: add no-op append_justification
* network: fix decode call for Justification
* network: only send a single Justification in BlockData
* network: minor comment update
* protocol: update field names to distinguish single justification
* client: further field renames to plural
* client: update function names to plural justifications
* client/db: upgrade existing database for new format
* network: remove dependency on grandpa crate
* db: fix check for finalized block
* grandpa: check for multiple grandpa justifications hwne importing
* backend: update Finalizer trait to take multiple Justifications
* db: remove debugging statements in migration code
* manual-seal: update note about engine id
* db: fix check for finalized block
* client: update variable name to reflect it is now plural
* grandpa: fix incorrect empty Justications in test
* primitives: make Justifications opaque to avoid being empty
* network: fix detecting empty Justification
* runtime: doc strings for Justifications functions
* runtime: add into_justifications
* primitives: check for duplicates in when adding to Justifications
* network/test: use real grandpa engine id in test
* client: fix reviewer comments
* primitives: rename Justifications::push to append
* backend: revert changes to Finalizer trait
* backend: revert mark_finalized
* backend: revert changes to finalize_block
* backend: revert finalized_blocks
* db: add a quick early return for performance
* client: minor reviewer comments
* service/test: use local ConsensusEngineId
* network: add link to issue for sending multiple Justifications
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* network: tweaks to review suggestions
* network: revert change to BlockData for backwards compatibility
* Apply suggestion from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* primitives: update doc comment for Justifications
* client/db/upgrade: avoid grandpa crate dependency
* consensus: revert to single Justification for import_justification
* primitives: improve justifications docs
* style cleanups
* use and_then
* client: rename JUSTIFICATIONS db column
* network: revert to using FRNK in network-test
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Migrate pallet-proxy to pallet attribute macro
Part of #7882.
Converts the `Proxy` pallet to the new pallet attribute macro introduced in #6877.
[Upgrade guidelines used](https://substrate.dev/rustdocs/v3.0.0/frame_support/attr.pallet.html#upgrade-guidelines).
## ⚠️ Breaking Change ⚠️
From [checking upgrade guidelines](https://crates.parity.io/frame_support/attr.pallet.html#checking-upgrade-guidelines)
> storages now use PalletInfo for module_prefix instead of the one given to `decl_storage`: use of this pallet in `construct_runtime!` needs careful updating of the name in order to not break storage or to upgrade storage (moreover for instantiable pallet). If pallet is published, make sure to warn about this breaking change.
So users of the `Assets` pallet must be careful about the name they used in `construct_runtime!`. Hence the `runtime-migration` label, which might not be needed depending on the configuration of the `Assets` pallet.
### Notes
There are some changes to the docs in metadata for the constants. The docs in the metadata for constants are now more complete.
* Improve complexity of CompactAssignments::unique_targets
Original implementation was O(n**2). Current impl is O(n log n).
Avoided the original proposed mitigation because it does not retain
the de-duplicating property present in the original implementation.
This implementation does a little more work, but retains that property.
* Explicitly choose sp_std Vec and BTreeSet
Ensures that the macro still works if someone uses it in a context
in which sp_std is not imported or is renamed.
* explicitly use sp_std vectors throughout compact macro
* Update diener to enable polkadot companions that use new crates
Diener will be updated in the CI image to make it possible to have
companions that add new crates from Substrate.
Besides that the wasm-bindgen is updated, because `0.2.70` is not
available anymore on crates.io.
* update lock
Co-authored-by: kianenigma <kian@parity.io>
* Add PJR challenge functions
- Updates the PJR check to return a counterexample if one exists
- Adds functions to cheaply check counterexamples
This is in support of off-chain PJR challenges: if a miner discovers
that an accepted election solution does not satisfy PJR, it will be
eligible for substantial rewards. This helps ensure that validator
elections have an absolute quality floor, so even if someone manages
to censor well-behaved solutions to give themselves unfair representation,
we can catch them in the act and penalize them.
* counterexample -> counter_example
* reorganize: high -> low abstraction
* reorganize challenges high -> low abstraction
* add note justifying linear search
* Simplify max_pre_score validation
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* add minor test of pjr challenge validation
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Don't log so many stkaing events on genesis block
* Clean a few more warn and info logs
* Update frame/staking/src/lib.rs
* Update frame/staking/src/lib.rs
We introduced banning of peers who spam us with the same request (more
than 2 times). However, we missed that it is completely legal to send
the same request multiple times as long as we did not provide any
answer. An example for that is the justification request. This request
is send multiple times until we could fetch the justification from one
of our peers. So, the solution to this problem is to tag requests as
fulfilled and to start counting these fulfilled requests. If the number
is higher than what we allow, the peer should be banned.
The timestamp inherent type was up to now just a simple `u64`. This
worked, but doesn't give you that much guarantees at compile time about
the type. This pr changes that by converting this type to a unit type
wrapper, similar to what we have done for `Slot`.
This is required for some future pr that touches quite a lot of the
inherents stuff :)
Besides this unit wrapper type, this pr also moves the `OnTimestampSet`
trait to `frame_support::traits`.
* Apply.
* get rid of glob import
* use meaningful generic type name
* pjr_check operates on `Supports` struct used elsewhere
* improve algorithmic complexity of `prepare_pjr_input`
* fix rustdoc warnings
* improve module docs
* typo
* simplify debug assertion
* add test finding the phase-change threshold value for a constructed scenario
* add more threshold scenarios to disambiguate plausible interpretations
* add link to npos paper reference
* docs: staked_assignment -> supports
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* add utility method for generating npos inputs
* add a fuzzer which asserts that all unbalanced seq_phragmen are PJR
Note that this currently fails. I hope that this can be rectified
by calculating the threshold instead of choosing some arbitrary number.
* assert in all cases, not just debug
* leverage a native solution to choose candidates
* use existing helper methods
* add pjr-check and incorporate into the fuzzer
We should probably have one of the W3F people look at this to ensure
we're not misconstruing any definitions, but this seems like a
fairly straightforward implementation.
* fix compilation errors
* Enable manually setting iteration parameters in single run.
This gives us the ability to reproducably extract cases where
honggfuzz has discovered a panic. For example:
$ cargo run --release --bin phragmen_pjr -- --candidates 569 --voters 100
Tue 23 Feb 2021 11:23:39 AM CET
Compiling bitflags v1.2.1
Compiling unicode-width v0.1.8
Compiling unicode-segmentation v1.7.1
Compiling ansi_term v0.11.0
Compiling strsim v0.8.0
Compiling vec_map v0.8.2
Compiling proc-macro-error-attr v1.0.4
Compiling proc-macro-error v1.0.4
Compiling textwrap v0.11.0
Compiling atty v0.2.14
Compiling heck v0.3.2
Compiling clap v2.33.3
Compiling structopt-derive v0.4.14
Compiling structopt v0.3.21
Compiling sp-npos-elections-fuzzer v2.0.0-alpha.5 (/home/coriolinus/Documents/Projects/paritytech/substrate/primitives/npos-elections/fuzzer)
Finished release [optimized] target(s) in 6.15s
Running `/home/coriolinus/Documents/Projects/paritytech/substrate/target/release/phragmen_pjr -c 569 -v 100`
thread 'main' panicked at 'unbalanced sequential phragmen must satisfy PJR', primitives/npos-elections/fuzzer/src/phragmen_pjr.rs:133:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
This is still not adequate proof that seq_phragmen is broken; it could
very well be that our PJR checker is doing the wrong thing, or we've
somehow missed a parameter of interest. Still, it's concerning.
* update comment verbiage for accuracy
* it is valid in PJR for an elected candidate to have 0 support
* Fix phragmen_pjr fuzzer
It turns out that the fundamental problem causing previous implementations
of the fuzzer to fail wasn't in `seq_phragmen` _or_ in `pjr_check`: it was
in the rounding errors introduced in the various conversions between the
internal data representation and the external one.
Fixing the fuzzer is then simply an issue of using the internal representation
and staying in that representation. However, that leaves the issue that
`seq_phragmen` occasionally produces an output which is technically not
PJR due to rounding errors. In the future we will need to add some kind of
"close-enough" threshold. However, that is explicitly out of scope of
this PR.
* restart ci; it appears to be stalled
* use necessary import for no-std
* use a more realistic distribution of voters and candidates
This isn't ideal; more realistic numbers would be about twice these.
However, either case generation or voting has nonlinear execution
time, and doubling these values brings iteration time from ~20s to
~180s. Fuzzing 6x as fast should make up for fuzzing cases half the size.
* identify specifically which PJR check may fail
* move candidate collection comment into correct place
* standard_threshold: use a calculation method which cannot overflow
* Apply suggestions from code review (update comments)
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* clarify the effectiveness bounds for t-pjr check
* how to spell "committee"
* reorganize: high -> low abstraction
* ensure standard threshold calc cannot panic
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: kianenigma <kian.peymani@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* more clear randomness API for BABE
* babe: move randomness utilities to its own file
* node: use babe::RandomnessFromOneEpochAgo in random_seed implementation
* frame-support: annotate randomness trait with block number
* pallet-randomness-collective-flip: fix for new randomness trait
* pallet-society: fix randomness usage
* pallet-lottery: fix randomness usage
* pallet-contracts: fix randomness usage
* pallet-babe: fix randomness usage
we need to track when the current and previous epoch started so that we
know the block number by each existing on-chain was known
* node: fix random_seed
* node-template: fix random_seed
* frame-support: extend docs
* babe: add test for epoch starting block number tracking
* babe: fix epoch randomness docs
* frame: add todos for dealing with randomness api changes
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Add MMR custom RPC.
* Change RuntimeApi to avoid hardcoding leaf type.
* Properly implement the new RuntimeAPI and wire up RPC.
* Extract Offchain DB as separate execution extension.
* Enable offchain DB access for offchain calls.
* Fix offchain_election tests.
* Skip block initialisation for proof generation.
* Fix integration test setup.
* Fix offchain tests. Not sure how I missed them earlier 🤷.
* Fix long line.
* One more test missing.
* Update mock for multi-phase.
* Address review grumbbles.
* Address review grumbles.
* Fix line width of a comment
* Make changes
* Add serialize/deserialize, copy babe epoch config defaults from node runtime
* Fix line widths and turn default features off for serde
* Remove ser/deser from Epoch, fix node-cli
* Apply suggestions
* Add comment to BABE_GENESIS_EPOCH_CONFIG in bin
* Apply suggestions
* Add a sketchy migration function
* Add a migration test
* Check for PendingEpochConfigChange as well
* Make epoch_config in node-cli
* Move updating EpochConfig out of the if
* Fix executor tests
* Calculate weight for add_epoch_configurations
* Fix babe test
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Add more asserts to tests, remove unused changes to primitives/slots
* Allow setting the migration pallet prefix
* Rename to BabePalletPrefix
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Guillaume Thiolliere