* Apply.
* get rid of glob import
* use meaningful generic type name
* pjr_check operates on `Supports` struct used elsewhere
* improve algorithmic complexity of `prepare_pjr_input`
* fix rustdoc warnings
* improve module docs
* typo
* simplify debug assertion
* add test finding the phase-change threshold value for a constructed scenario
* add more threshold scenarios to disambiguate plausible interpretations
* add link to npos paper reference
* docs: staked_assignment -> supports
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* add utility method for generating npos inputs
* add a fuzzer which asserts that all unbalanced seq_phragmen are PJR
Note that this currently fails. I hope that this can be rectified
by calculating the threshold instead of choosing some arbitrary number.
* assert in all cases, not just debug
* leverage a native solution to choose candidates
* use existing helper methods
* add pjr-check and incorporate into the fuzzer
We should probably have one of the W3F people look at this to ensure
we're not misconstruing any definitions, but this seems like a
fairly straightforward implementation.
* fix compilation errors
* Enable manually setting iteration parameters in single run.
This gives us the ability to reproducably extract cases where
honggfuzz has discovered a panic. For example:
$ cargo run --release --bin phragmen_pjr -- --candidates 569 --voters 100
Tue 23 Feb 2021 11:23:39 AM CET
Compiling bitflags v1.2.1
Compiling unicode-width v0.1.8
Compiling unicode-segmentation v1.7.1
Compiling ansi_term v0.11.0
Compiling strsim v0.8.0
Compiling vec_map v0.8.2
Compiling proc-macro-error-attr v1.0.4
Compiling proc-macro-error v1.0.4
Compiling textwrap v0.11.0
Compiling atty v0.2.14
Compiling heck v0.3.2
Compiling clap v2.33.3
Compiling structopt-derive v0.4.14
Compiling structopt v0.3.21
Compiling sp-npos-elections-fuzzer v2.0.0-alpha.5 (/home/coriolinus/Documents/Projects/paritytech/substrate/primitives/npos-elections/fuzzer)
Finished release [optimized] target(s) in 6.15s
Running `/home/coriolinus/Documents/Projects/paritytech/substrate/target/release/phragmen_pjr -c 569 -v 100`
thread 'main' panicked at 'unbalanced sequential phragmen must satisfy PJR', primitives/npos-elections/fuzzer/src/phragmen_pjr.rs:133:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
This is still not adequate proof that seq_phragmen is broken; it could
very well be that our PJR checker is doing the wrong thing, or we've
somehow missed a parameter of interest. Still, it's concerning.
* update comment verbiage for accuracy
* it is valid in PJR for an elected candidate to have 0 support
* Fix phragmen_pjr fuzzer
It turns out that the fundamental problem causing previous implementations
of the fuzzer to fail wasn't in `seq_phragmen` _or_ in `pjr_check`: it was
in the rounding errors introduced in the various conversions between the
internal data representation and the external one.
Fixing the fuzzer is then simply an issue of using the internal representation
and staying in that representation. However, that leaves the issue that
`seq_phragmen` occasionally produces an output which is technically not
PJR due to rounding errors. In the future we will need to add some kind of
"close-enough" threshold. However, that is explicitly out of scope of
this PR.
* restart ci; it appears to be stalled
* use necessary import for no-std
* use a more realistic distribution of voters and candidates
This isn't ideal; more realistic numbers would be about twice these.
However, either case generation or voting has nonlinear execution
time, and doubling these values brings iteration time from ~20s to
~180s. Fuzzing 6x as fast should make up for fuzzing cases half the size.
* identify specifically which PJR check may fail
* move candidate collection comment into correct place
* standard_threshold: use a calculation method which cannot overflow
* Apply suggestions from code review (update comments)
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* clarify the effectiveness bounds for t-pjr check
* how to spell "committee"
* reorganize: high -> low abstraction
* ensure standard threshold calc cannot panic
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: kianenigma <kian.peymani@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* more clear randomness API for BABE
* babe: move randomness utilities to its own file
* node: use babe::RandomnessFromOneEpochAgo in random_seed implementation
* frame-support: annotate randomness trait with block number
* pallet-randomness-collective-flip: fix for new randomness trait
* pallet-society: fix randomness usage
* pallet-lottery: fix randomness usage
* pallet-contracts: fix randomness usage
* pallet-babe: fix randomness usage
we need to track when the current and previous epoch started so that we
know the block number by each existing on-chain was known
* node: fix random_seed
* node-template: fix random_seed
* frame-support: extend docs
* babe: add test for epoch starting block number tracking
* babe: fix epoch randomness docs
* frame: add todos for dealing with randomness api changes
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Add MMR custom RPC.
* Change RuntimeApi to avoid hardcoding leaf type.
* Properly implement the new RuntimeAPI and wire up RPC.
* Extract Offchain DB as separate execution extension.
* Enable offchain DB access for offchain calls.
* Fix offchain_election tests.
* Skip block initialisation for proof generation.
* Fix integration test setup.
* Fix offchain tests. Not sure how I missed them earlier 🤷.
* Fix long line.
* One more test missing.
* Update mock for multi-phase.
* Address review grumbbles.
* Address review grumbles.
* Fix line width of a comment
* Make changes
* Add serialize/deserialize, copy babe epoch config defaults from node runtime
* Fix line widths and turn default features off for serde
* Remove ser/deser from Epoch, fix node-cli
* Apply suggestions
* Add comment to BABE_GENESIS_EPOCH_CONFIG in bin
* Apply suggestions
* Add a sketchy migration function
* Add a migration test
* Check for PendingEpochConfigChange as well
* Make epoch_config in node-cli
* Move updating EpochConfig out of the if
* Fix executor tests
* Calculate weight for add_epoch_configurations
* Fix babe test
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Add more asserts to tests, remove unused changes to primitives/slots
* Allow setting the migration pallet prefix
* Rename to BabePalletPrefix
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* A clean new attempt
* Checkpoint to move remote.
* A lot of dependency wiring to make it feature gated.
* bad macro, bad macro.
* refactor(remote ext): use jsonrpsee
* refactor(remote ext): use jsonrpsee
* Undo the DB mess.
* fix(remote ext): use max limit `u32::MAX`
* resolve TODOs
* jsonrpsee switch to `hyper` as backend
* Update utils/frame/try-runtime/remote-externalities/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* update jsonrpsee
* remove boiler-plate
* suppress warnings to CI happy
* Unbreak his build
* Use option
* fix nit; make it work again
* fix err message.
* Update utils/frame/remote-externalities/Cargo.toml
* Fix uri stuff
* remove needless clone
Co-authored-by: kianenigma <kian.peymani@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>
* Init `RuntimeLogger` automatically for each runtime api call
This pr change the runtime api in such a way to always and automatically
enable the `RuntimeLogger`. This enables the user to use `log` or
`tracing` from inside the runtime to create log messages. As logging
introduces some extra code and especially increases the size of the wasm
blob. It is advised to disable all logging completely with
`sp-api/disable-logging` when doing the wasm builds for the on-chain
wasm runtime.
Besides these changes, the pr also brings most of the logging found in
frame to the same format "runtime::*".
* Update frame/im-online/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update test-utils/runtime/Cargo.toml
* Fix test
* Don't use tracing in the runtime, as we don't support it :D
* Fixes
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* A clean new attempt
* Checkpoint to move remote.
* A lot of dependency wiring to make it feature gated.
* bad macro, bad macro.
* Undo the DB mess.
* Update frame/support/src/traits.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* Apply suggestions from code review
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* unbreak the build
* Better logging and ids for migrations
* Fix doc.
* Test
* Update frame/try-runtime/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update utils/frame/try-runtime/cli/Cargo.toml
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Update frame/try-runtime/Cargo.toml
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Address most review grumbles.
* Fix build
* Add some comments
* Remove allowing one pallet at a time.
* Rework the PR
* nit
* Slightly better error handling.
* Remove files
* Update utils/frame/remote-externalities/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update frame/support/src/dispatch.rs
* Update frame/support/src/dispatch.rs
* Fix test
* Make extension trait.
* Bring back try-runtime/std
* remove bincode
* Remove warning
* Change test features
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* contracts: Update README
* contracts: Add CHANGELOG.md
* contracts: Bump version to v3.0.0 and allow publish
* Typos
Co-authored-by: Andrew Jones <ascjones@gmail.com>
* Improve wording in the changelog
* contracts: Add reserved field to ContractInfoOf for future proofing
* also bump frame-benchmarking
* update lockfile
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* A clean new attempt
* Checkpoint to move remote.
* A lot of dependency wiring to make it feature gated.
* bad macro, bad macro.
* Undo the DB mess.
* Update frame/support/src/traits.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* Apply suggestions from code review
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* unbreak the build
* Update frame/try-runtime/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update utils/frame/try-runtime/cli/Cargo.toml
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Update frame/try-runtime/Cargo.toml
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Address most review grumbles.
* Fix build
* Add some comments
* Remove allowing one pallet at a time.
* More grumbles.
* relocate remote-ext
* Fix build
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Fix tracing tests (#8022)
* Fix tracing tests
The tests were not working properly.
1. Some test was setting a global subscriber, this could lead to racy
conditions with other tests.
2. A logging test called `process::exit` which is completly wrong.
* Update client/tracing/src/lib.rs
Co-authored-by: David <dvdplm@gmail.com>
* Review comments
Co-authored-by: David <dvdplm@gmail.com>
* Fix tracing spans are not being forwarded to spawned task (#8009)
* Fix tracing spans are not being forwarded to spawned task
There is a bug that tracing spans are not forwarded to spawned task. The
problem was that only the telemetry span was forwarded. The solution to
this is to use the tracing provided `in_current_span` to capture the
current active span and pass the telemetry span explictely. We will now
always enter the span when the future is polled. This is essentially the
same strategy as tracing is doing with its `Instrumented`, but now
extended for our use case with having multiple spans active.
* More tests
* Proper test for telemetry and prefix span
* WIP
* Fix test (need to create & enter the span at the same time)
* WIP
* Remove telemtry_span from sc_service config
* CLEANUP
* Update comment
* Incorrect indent
* More meaningful name
* Dedent
* Naming XD
* Attempt to make a more complete test
* lint
* Missing licenses
* Remove user data
* CLEANUP
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* CLEANUP
* Apply suggestion
* Update bin/node/cli/tests/telemetry.rs
Co-authored-by: David <dvdplm@gmail.com>
* Wrapping lines
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
* Convert AURA to new pallet macro
* AURA: Switch to `CurrentSlot` instead of `LastTimestamp`
This switches AURA to use `CurrentSlot` instead of `LastTimestamp`.
* Add missing file
* Update frame/aura/src/migrations.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Remove the runtime side provide inherent code
* Use correct weight
* Add TODO
* Remove the Inherent from AURA
* 🤦
* Remove unused stuff
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Decouple the session validators from im-online
* .
* Add SessionInterface trait in im-online
Add ValidatorId in im-online Trait
Make im-online compile
Make substrate binary compile
* Fix merging issue
* Make all compile
* Fix tests
* Avoid using frame dep in primitives via pallet-session-common
* Merge ValidatorSet into SessionInterface trait
Wrap a few too long lines
Add some docs
* Move pallet-sesion-common into pallet-session
* Move SessionInterface to sp-session and impl it in session pallet
Ref https://github.com/paritytech/substrate/pull/7127#discussion_r494892472
* Split put historical::FullValidatorIdentification trait
* Fix line width
* Fix staking mock
* Fix session doc test
* Simplify <T as ValidatorIdentification<AccountId>>::ValidatorId as ValidatorId<T>
* Nits
* Clean up.
* Make it compile by commenting out report_offence_im_online bench
* Tests
* Nits
* Move OneSessionHandler to sp-session
* Fix tests
* Add some docs
* .
* Fix typo
* Rename to ValidatorSet::session_index()
* Add some more docs
* .
* Remove extra empty line
* Fix line width check
.
* Apply suggestions from code review
* Cleaup Cargo.toml
* Aura has migrated to Pallet now
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
* Fix tracing spans are not being forwarded to spawned task
There is a bug that tracing spans are not forwarded to spawned task. The
problem was that only the telemetry span was forwarded. The solution to
this is to use the tracing provided `in_current_span` to capture the
current active span and pass the telemetry span explictely. We will now
always enter the span when the future is polled. This is essentially the
same strategy as tracing is doing with its `Instrumented`, but now
extended for our use case with having multiple spans active.
* More tests
* Introduce a `Slot` type
Instead of having some type definition that only was used in half of the
code or directly using `u64`, this adds a new unit type wrapper `Slot`.
This makes it especially easy for the outside api to know what type is
expected/returned.
* Change epoch duratioC
* rename all instances of slot number to slot
* Make the constructor private
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Made a start
* So the proof between authority set is phragmen one, this is crazy big,
or is there some signing of the result : that is the storage key, damn?
* ok getting from header digest seems doable.
* for testing
* get set id from storage directly (should use runtime to handler change).
* move test to init
* correct auth key
* fix iteration
* Correct proof content
* actually update block number.
* actually check last justif against its header
* justification relation to new authorities through header hash check is
needed here. This assumes the hash from header is calculated.
* Few changes
* Connected up cheme's branch
* Clean up
* Move things around a bit so that adding the grandpa warp sync request response protocol happens in the node code
* Nits
* Changes to comments
* Cheme changes
* Remove todos and test compile.
* Rename _authority_ related proof function to _warp_sync_ .
* Update client/grandpa-warp-sync/src/lib.rs
quick fix
* Put the warp sync request response protocol behind a feature flag because we dont' need it on a light client.
* Update client/grandpa-warp-sync/src/lib.rs
Quick fix
* Update Cargo.lock
* Adding test, comment on limitation related to 'delay', this could
be implemented but with a cost.
* Set between a delay override last fragment.
* Check for pending authority set change at start.
* adjust index
* custom cache is not a good idea.
* Use a simple cache instead.
* restore broken indentation
* Address crate rename
* Merge conflict badly resolved, sorry
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Make offchain indexing work
This fixes some bugs with offchain indexing to make it actually working ;)
* Fix tests
* Fix browser build
* Update client/db/src/offchain.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Remove seperation between prefix and key
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Sync: Propagate block announcement data
This pr adds a feature to the sync protocol to propagate the data that
we received alongside a block announcement. This is done by adding a
cache that caches the last X block announcement data where X is set to
the number of `in_peers` (giving every peer the chance to send us a
different block). This will be required by parachains to ensure that
even peers who are not connected to a collator receive the data
alongside the block announcement to properly validate it and request the
block.
* Review comment
* Bring back the code and add new variant to ensure we don't insert block
announce data when something wasn't checked
* Also use out_peers
dependabot[bot]