mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-07-10 00:37:27 +00:00
0057c0b53f4a73afbc5136e1c72efc976d5ab3b3
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b5f23bfd1c |
Ensure election offchain workers don't overlap (#8828)
* Initial version, well tested, should work fine. * Add one last log line * Update frame/election-provider-multi-phase/src/unsigned.rs Co-authored-by: Gavin Wood <gavin@parity.io> * Update frame/election-provider-multi-phase/src/unsigned.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Update frame/election-provider-multi-phase/src/unsigned.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Fix a few more things * fix build * rewrite the whole thing with a proper lock * clean * clean some nits * Add unit tests. * Update primitives/runtime/src/offchain/storage_lock.rs Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Fix test * Fix tests Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> |
||
|
|
fc449c8aad | Fix panic in election-provider offchain worker. (#8732) | ||
|
|
de5d0b2312 |
Multi-phase elections solution resubmission (#8290)
* not climate * explain the intent of the bool in the unsigned phase * remove glob imports from unsigned.rs * add OffchainRepeat parameter to ElectionProviderMultiPhase * migrate core logic from #7976 This is a much smaller diff than that PR contained, but I think it contains all the essentials. * improve formatting * fix test build failures * cause test to pass * Apply suggestions from code review Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * collapse imports * threshold acquired directly within try_acquire_offchain_lock * add test of resubmission after interval * add test that ocw can regenerate a failed cache when resubmitting * ensure that OCW solutions are of the correct round This should help prevent stale cached solutions from persisting past the election for which they are intended. * add test of pre-dispatch round check * use `RawSolution.round` instead of redundantly externally * unpack imports Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * rename `OFFCHAIN_HEAD_DB` -> `OFFCHAIN_LOCK` * rename `mine_call` -> `mine_checked_call` * eliminate extraneous comma * check cached call is current before submitting * remove unused consts introduced by bad merge. Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * resubmit when our solution beats queued solution * clear call cache if solution fails to submit * use local storage; clear on ElectionFinalized * Revert "use local storage; clear on ElectionFinalized" This reverts commit 4b46a9388532d0c09b337dc7c7edf76044a6cee8. * BROKEN: try to filter local events in OCW * use local storage; simplify score fetching * fix event filter * mutate storage instead of setting it * StorageValueRef::local isn't actually implemented yet * add logging for some events of interest in OCW miner * rename kill_solution -> kill_ocw_solution to avoid ambiguity * defensive err instead of unreachable given unreachable code * doc punctuation Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * distinguish miner errors between "out of date" and "call invalid" * downgrade info logs -> debug * ensure encoded call decodes as a call * fix semantics of validation of pre-dispatch failure for wrong round * move score check within `and_then` * add test that offchain workers clear their cache after election * ensure that bad ocw submissions are not retained for resubmission * simplify fn ocw_solution_exists * add feasibility check when restoring cached solution should address https://github.com/paritytech/substrate/pull/8290/files#r617533358 restructures how the checks are sequenced, which simplifies legibility. * simplify checks again Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> |
||
|
|
c786fb21a0 |
Refactor election solution trimming for efficiency (#8614)
* Refactor election solution trimming for efficiency
The previous version always trimmed the `CompactOf<T>` instance,
which was intrinsically inefficient: that's a packed data structure,
which is naturally expensive to edit. It's much easier to edit
the unpacked data structures: the `voters` and `assignments` lists.
* rework length-trim tests to work with the new interface
Test suite now compiles. Tests still don't pass because the macro
generating the compact structure still generates `unimplemented!()`
for the actual `compact_length_of` implementation.
* simplify
* add a fuzzer which can validate `Compact::encoded_size_for`
The `Compact` solution type is generated distinctly for each runtime,
and has both three type parameters and a built-in limit to the number
of candidates that each voter can vote for. Finally, they have an
optional `#[compact]` attribute which changes the encoding behavior.
The assignment truncation algorithm we're using depends on the ability
to efficiently and accurately determine how much space a `Compact`
solution will take once encoded.
Together, these two facts imply that simple unit tests are not
sufficient to validate the behavior of `Compact::encoded_size_for`.
This commit adds such a fuzzer. It is designed such that it is possible
to add a new fuzzer to the family by simply adjusting the
`generate_solution_type` macro invocation as desired, and making a
few minor documentation edits.
Of course, the fuzzer still fails for now: the generated implementation
for `encoded_size_for` is still `unimplemented!()`. However, once
the macro is updated appropriately, this fuzzer family should allow
us to gain confidence in the correctness of the generated code.
* Revert "add a fuzzer which can validate `Compact::encoded_size_for`"
This reverts commit 916038790887e64217c6a46e9a6d281386762bfb.
The design of `Compact::encoded_size_for` is flawed. When `#[compact]`
mode is enabled, every integer in the dataset is encoded using run-
length encoding. This means that it is impossible to compute the final
length faster than actually encoding the data structure, because the
encoded length of every field varies with the actual value stored.
Given that we won't be adding that method to the trait, we won't be
needing a fuzzer to validate its performance.
* revert changes to `trait CompactSolution`
If `CompactSolution::encoded_size_for` can't be implemented in the
way that we wanted, there's no point in adding it.
* WIP: restructure trim_assignments_length by actually encoding
This is not as efficient as what we'd hoped for, but it should still
be better than what it's replacing. Overall efficiency of
`fn trim_assignments_length` is now `O(edges * lg assignments.len())`.
* fix compiler errors
* don't sort voters, just assignments
Sorting the `voters` list causes lots of problems; an invariant that
we need to maintain is that an index into the voters list has a stable
meaning.
Luckily, it turns out that there is no need for the assignments list
to correspond to the voters list. That isn't an invariant, though previously
I'd thought that it was.
This simplifies things; we can just leave the voters list alone,
and sort the assignments list the way that is convenient.
* WIP: add `IndexAssignment` type to speed up repeatedly creating `Compact`
Next up: `impl<'a, T> From<&'a [IndexAssignmentOf<T>]> for Compact`,
in the proc-macro which makes `Compact`. Should be a pretty straightforward
adaptation of `from_assignment`.
* Add IndexAssignment and conversion method to CompactSolution
This involves a bit of duplication of types from
`election-provider-multi-phase`; we'll clean those up shortly.
I'm not entirely happy that we had to add a `from_index_assignments`
method to `CompactSolution`, but we couldn't define
`trait CompactSolution: TryFrom<&'a [Self::IndexAssignment]` because
that made trait lookup recursive, and I didn't want to propagate
`CompactSolutionOf<T> + TryFrom<&[IndexAssignmentOf<T>]>` everywhere
that compact solutions are specified.
* use `CompactSolution::from_index_assignment` and clean up dead code
* get rid of `from_index_assignments` in favor of `TryFrom`
* cause `pallet-election-provider-multi-phase` tests to compile successfully
Mostly that's just updating the various test functions to keep track of
refactorings elsewhere, though in a few places we needed to refactor some
test-only helpers as well.
* fix infinite binary search loop
Turns out that moving `low` and `high` into an averager function is a
bad idea, because the averager gets copies of those values, which
of course are never updated. Can't use mutable references, because
we want to read them elsewhere in the code. Just compute the average
directly; life is better that way.
* fix a test failure
* fix the rest of test failures
* remove unguarded subtraction
* fix npos-elections tests compilation
* ensure we use sp_std::vec::Vec in assignments
* add IndexAssignmentOf to sp_npos_elections
* move miner types to `unsigned`
* use stable sort
* rewrap some long comments
* use existing cache instead of building a dedicated stake map
* generalize the TryFrom bound on CompactSolution
* undo adding sp-core dependency
* consume assignments to produce index_assignments
* Add a test of Assignment -> IndexAssignment -> Compact
* fix `IndexAssignmentOf` doc
* move compact test from sp-npos-elections-compact to sp-npos-elections
This means that we can put the mocking parts of that into a proper
mock package, put the test into a test package among other tests.
Having the mocking parts in a mock package enables us to create a
benchmark (which is treated as a separate crate) import them.
* rename assignments -> sorted_assignments
* sort after reducing to avoid potential re-sort issues
* add runtime benchmark, fix critical binary search error
"Why don't you add a benchmark?", he said. "It'll be good practice,
and can help demonstrate that this isn't blowing up the runtime."
He was absolutely right.
The biggest discovery is that adding a parametric benchmark means that
you get a bunch of new test cases, for free. This is excellent, because
those test cases uncovered a binary search bug. Fixing that simplified
that part of the code nicely.
The other nice thing you get from a parametric benchmark is data about
what each parameter does. In this case, `f` is the size factor: what
percent of the votes (by size) should be removed. 0 means that we should
keep everything, 95 means that we should trim down to 5% of original size
or less.
```
Median Slopes Analysis
========
-- Extrinsic Time --
Model:
Time ~= 3846
+ v 0.015
+ t 0
+ a 0.192
+ d 0
+ f 0
µs
Min Squares Analysis
========
-- Extrinsic Time --
Data points distribution:
v t a d f mean µs sigma µs %
<snip>
6000 1600 3000 800 0 4385 75.87 1.7%
6000 1600 3000 800 9 4089 46.28 1.1%
6000 1600 3000 800 18 3793 36.45 0.9%
6000 1600 3000 800 27 3365 41.13 1.2%
6000 1600 3000 800 36 3096 7.498 0.2%
6000 1600 3000 800 45 2774 17.96 0.6%
6000 1600 3000 800 54 2057 37.94 1.8%
6000 1600 3000 800 63 1885 2.515 0.1%
6000 1600 3000 800 72 1591 3.203 0.2%
6000 1600 3000 800 81 1219 25.72 2.1%
6000 1600 3000 800 90 859 5.295 0.6%
6000 1600 3000 800 95 684.6 2.969 0.4%
Quality and confidence:
param error
v 0.008
t 0.029
a 0.008
d 0.044
f 0.185
Model:
Time ~= 3957
+ v 0.009
+ t 0
+ a 0.185
+ d 0
+ f 0
µs
```
What's nice about this is the clear negative correlation between
amount removed and total time. The more we remove, the less total
time things take.
|
||
|
|
33425ce21f |
Trim compact solution for length during preparation (#8317)
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> |
||
|
|
24311eee3e |
Change assert(is_err()) to assert_noop to check state consistency on errors (#8587)
* Change is_err() asserts in tests to assert_noop to check state consistency fixes #8545 * Update frame/transaction-payment/src/lib.rs Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/contracts/src/exec.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Update frame/democracy/src/benchmarking.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Update frame/transaction-payment/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Don't assert no-changing state. see: https://github.com/paritytech/substrate/pull/8587#issuecomment-817137906 * fix expected error * Fix non-extrinsic-call asserts Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> |
||
|
|
4e334f59d6 |
Use log::error! for bad mandatory (#8521)
To make the life of people easier ;) |
||
|
|
7b2cf33b73 |
Bring back the on_finalize weight of staking. (#8463)
* Bring back the on_finalize weighg of stakin. * Better logs * Also make a few things pub * Fix build * Add assertions * Add test. * remove dbg * Update frame/election-provider-multi-phase/src/unsigned.rs * Update frame/staking/src/tests.rs * Fix * Fix * Update frame/election-provider-multi-phase/src/unsigned.rs |
||
|
|
0c69651830 |
Decouple Staking and Election - Part 2.1: Unleash Multi Phase (#8113)
* Base features and traits. * pallet and unsigned phase * Undo bad formattings. * some formatting cleanup. * Small self-cleanup. * Make it all build * self-review * Some doc tests. * Some changes from other PR * Fix session test * Update Cargo.lock * Update frame/election-provider-multi-phase/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Some review comments * Rename + make encode/decode * Do an assert as well, just in case. * Fix build * Update frame/election-provider-multi-phase/src/unsigned.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Las comment * fix staking fuzzer. * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Add one last layer of feasibility check as well. * Last fixes to benchmarks * Some more docs. * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Some nits * It all works * Some self cleanup * Update frame/staking/src/lib.rs Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com> * remove most todos. * Round of self-review. * Fix migration * clean macro * Revert wrong merge * remove fuzzer stuff. * Self review * Update frame/staking/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * review comments * add logs * Add tests to demonstrate the capacity of the snapshot. * Replace upgrade * Last touches * Fix benchmakrs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * remove unused stuff * Fix tests. Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com> |
||
|
|
05f24931a9 |
Fast CompactAssignment search (#8385)
* use more efficient search through candidates in offchain-election * mark linear accessors as test-only in election-provider-multi-phase This prevents production code which uses them from compiling. Also write an efficient helper for getting the target index. * doc grammar * use faster target_index_fn in benchmarks * unbox helper functions * remove unnecessary import * write lifetime after primary trait Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> |
||
|
|
b6c626399e |
Audit fixes for election/staking decoupling part 2 (#8167)
* Base features and traits. * pallet and unsigned phase * Undo bad formattings. * some formatting cleanup. * Small self-cleanup. * Make it all build * self-review * Some doc tests. * Some changes from other PR * Fix session test * Update Cargo.lock * Update frame/election-provider-multi-phase/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Some review comments * Rename + make encode/decode * Do an assert as well, just in case. * Fix build * Update frame/election-provider-multi-phase/src/unsigned.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Las comment * fix staking fuzzer. * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Add one last layer of feasibility check as well. * Last fixes to benchmarks * Some more docs. * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Some nits * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Fix doc * Mkae ci green * Audit fixes for election-provider: part 2 signed phase. * Fix weight * Some grumbles. * Try and weigh to get_npos_voters * Fix build * Fix line width * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Fix tests. * Fix build * Reorg some stuff * More reorg. * Reorg done. * Fix build * Another rename * Fix build * Update frame/election-provider-multi-phase/src/mock.rs Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com> * nit * better doc * Line width * Fix build * Self-review * Self-review * Fix wan * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix build and review comments. * Update frame/election-provider-multi-phase/src/lib.rs Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * add comment Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com> |
||
|
|
7205eea40d |
Decouple Staking and Election - Part 2 Unsigned Phase (#7909)
* Base features and traits. * pallet and unsigned phase * Undo bad formattings. * some formatting cleanup. * Small self-cleanup. * Make it all build * self-review * Some doc tests. * Some changes from other PR * Fix session test * Update Cargo.lock * Update frame/election-provider-multi-phase/src/lib.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Some review comments * Rename + make encode/decode * Do an assert as well, just in case. * Fix build * Update frame/election-provider-multi-phase/src/unsigned.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Las comment * fix staking fuzzer. * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Add one last layer of feasibility check as well. * Last fixes to benchmarks * Some more docs. * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Some nits * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Fix doc * Mkae ci green Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> |