Fixes#3826.
The docs on the `candidates` field of `BlockEntry` were incorrectly
stating that they are sorted by core index. The (incorrect) optimization
was introduced in #3747 based on this assumption. The actual ordering is
based on `CandidateIncluded` events ordering in the runtime. We revert
this optimization here.
- [x] verify the underlying issue
- [x] add a regression test
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Small refactoring to reduce the algorithmic complexity of the initial
message distribution in approval voting after a sync from O(n_candidates
^ 2) to O(n_candidates).
On top of #3302.
We want the validators to upgrade first before we add changes to the
collation side to send the new variants, which is why this part is
extracted into a separate PR.
The detection of when to send the parent head is based on the core
assignments at the relay parent of the candidate. We probably want to
make it more flexible in the future, but for now, it will work for a
simple use case when a para always has multiple cores assigned to it.
---------
Signed-off-by: Matteo Muraca <mmuraca247@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Matteo Muraca <56828990+muraca@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Ignacio Rios <54085674+JuaniRios@users.noreply.github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Fixes#3128.
This introduces a new variant for the collation response from the
collator that includes the parent head data. For now, collators won't
send this new variant. We'll need to change the collator side of the
collator protocol to detect all the cores assigned to a para and send
the parent head data in the case when it's more than 1 core.
- [x] validate approach
- [x] check head data hash
Closes#1591.
The purpose of this PR is filter out backing statements from the network
signed by disabled validators. This is just an optimization, since we
will do filtering in the runtime in #1863 to avoid nodes to filter
garbage out at block production time.
- [x] Ensure it's ok to fiddle with the mask of manifests
- [x] Write more unit tests
- [x] Test locally
- [x] simple zombienet test
- [x] PRDoc
---------
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
#1259 was merged into a feature branch, but we've decided to merge
node-side changes for disabling straight into master.
This is a dependency of #1841 and #2637.
---------
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
* polkadot: propagate UnpinHandle to ActiveLeafUpdate
Also extract the leaf creation for tests
into a common function.
* dispute-coordinator: try pinned blocks for slashin
* apparently 1.72 is smarter than 1.70
* address nits
* rename fresh_leaf to new_leaf
* zombienet: use test-parachain image for the slashing test
* use the right image
* try polkadot-parachain image
* try naming collator alice 🙈
* add needed job for the pipeline
* fix user id in polkadot-parachain-debug image
* small tweaks to the test
* another small tweak
* yet another small tweak
* bump zombienet version
---------
Co-authored-by: Javier Viola <javier@parity.io>
* runtime/vstaging: unapplied_slashes runtime API
* runtime/vstaging: key_ownership_proof runtime API
* runtime/ParachainHost: submit_report_dispute_lost
* fix key_ownership_proof API
* runtime: submit_report_dispute_lost runtime API
* nits
* Update node/subsystem-types/src/messages.rs
Co-authored-by: Marcin S. <marcin@bytedude.com>
* revert unrelated fmt changes
* dispute-coordinator: past session dispute slashing
* encapsule runtime api call for submitting report
* prettify: extract into a function
* do not exit on runtime api error
* fix tests
* try initial zombienet test
* try something
* fix a typo
* try cumulus-based collator
* fix clippy
* build polkadot-debug images with fast-runtime enabled
* wip
* runtime/inclusion: fix availability_threshold
* fix wip
* fix wip II
* revert native provider
* propagate tx submission error
* DEBUG: sync=trace
* print key ownership proof len
* panic repro
* log validator index in panic message
* post merge fixes
* replace debug assertion with a log
* fix compilation
* Let's log the dispatch info in validate block.
* fix double encoding
* Revert "Let's log the dispatch info in validate block."
This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.
* Revert "Let's log the dispatch info in validate block."
This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.
* fix compilation
* update to latest zombienet and fix test
* lower finality lag to 11
* bump zombienet again
* add a workaround, but still does not work
* Update .gitlab-ci.yml
bump zombienet.
* add a comment and search logs on all nodes
---------
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Javier Viola <javier@parity.io>
* disputes/slashing: slash only backers for ForInvalid
* add an assertion in mock impl
* fix tests
* do not slash backers on onconcluded disputes
* slash an intersection of backers and losers
* zombienet/disputes: check for offence only for invalid disputes
* add backing votes to disputes bench builder
* Update runtime/parachains/src/builder.rs
* Brad implementers guide revisions 2 (#6239)
* Add disputes subsystems fix
* Updated dispute approval vote import reasoning
* Improved wording of my changes
* Resolving issues brought up in comments
* Update disputes prioritisation in `dispute-coordinator` (#6130)
* Scraper processes CandidateBacked events
* Change definition of best-effort
* Fix `dispute-coordinator` tests
* Unit test for dispute filtering
* Clarification comment
* Add tests
* Fix logic
If a dispute is not backed, not included and not confirmed we
don't participate but we do import votes.
* Add metrics for refrained participations
* Revert "Add tests"
This reverts commit 7b8391a087922ced942cde9cd2b50ff3f633efc0.
* Revert "Unit test for dispute filtering"
This reverts commit 92ba5fe678214ab360306313a33c781338e600a0.
* fix dispute-coordinator tests
* Fix scraping
* new tests
* Small fixes in guide
* Apply suggestions from code review
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
* Fix some comments and remove a pointless test
* Code review feedback
* Clarification comment in tests
* Some tests
* Reference counted `CandidateHash` in scraper
* Proper handling for Backed and Included candidates in scraper
Backed candidates which are not included should be kept for a
predetermined window of finalized blocks. E.g. if a candidate is backed
but not included in block 2, and the window size is 2, the same
candidate should be cleaned after block 4 is finalized.
Add reference counting for candidates in scraper. A candidate can be
added on multiple block heights so we have to make sure we don't clean
it prematurely from the scraper.
Add tests.
* Update comments in tests
* Guide update
* Fix cleanup logic for `backed_candidates_by_block_number`
* Simplify cleanup
* Make spellcheck happy
* Update tests
* Extract candidate backing logic in separate struct
* Code review feedback
* Treat backed and included candidates in the same fashion
* Update some comments
* Small improvements in test
* spell check
* Fix some more comments
* clean -> prune
* Code review feedback
* Reword comment
* spelling
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
* approval-voting: remove redundant validation check (#6266)
* approval-voting: remove a redundant check
* candidate-validation: remove unreachable check
* remove fill_block (#6200)
Co-authored-by: parity-processbot <>
* fix a compilation warning (#6279)
Fixes#6277.
* Only report concluded if there is an actual dispute. (#6270)
* Only report concluded if there is an actual dispute.
Hence no "non"-disputes will be added to disputes anymore.
* Fix redundant check.
* Test for no onesided disputes.
Co-authored-by: eskimor <eskimor@no-such-url.com>
* [ci] fix buildah image (#6281)
* Revert special casing of Kusama for grandpa rounds. (#6217)
Co-authored-by: eskimor <eskimor@no-such-url.com>
* Fixes "for loop over an `Option`" warnings (#6291)
Was seeing these warnings when running `cargo check --all`:
```
warning: for loop over an `Option`. This is more readably written as an `if let` statement
--> node/core/approval-voting/src/lib.rs:1147:21
|
1147 | for activated in update.activated {
| ^^^^^^^^^^^^^^^^
|
= note: `#[warn(for_loops_over_fallibles)]` on by default
help: to check pattern in a loop use `while let`
|
1147 | while let Some(activated) = update.activated {
| ~~~~~~~~~~~~~~~ ~~~
help: consider using `if let` to clear intent
|
1147 | if let Some(activated) = update.activated {
| ~~~~~~~~~~~~ ~~~
```
My guess is that `activated` used to be a SmallVec or similar, as is
`deactivated`. It was changed to an `Option`, the `for` still compiled (it's
technically correct, just weird), and the compiler didn't catch it until now.
* companion for #12599 (#6290)
* companion for #12599
* update Cargo.lock
* use cargo path instead of diener
* update lockfile for {"substrate"}
Co-authored-by: parity-processbot <>
* remove the runtime check and test
* append keys on past-session slashing
* runtime/disputes: allow importing backing votes after explicit for
* explicit MaliciousBacker error and a test
* update an outdated comment
* Revert "update an outdated comment"
This reverts commit 7c4c3f5a848f16e2b61435e981d814f00333ed41.
* Revert "remove the runtime check and test"
This reverts commit a5bff0c75e77effb5b7d3a1691de1b14bcdbd648.
* incremental punishment post conclusion + test
* punish backers post FOR vote
* remove unnecessary lifetime annotation
* add a comment to zombinet test
* typo
* fmt
* post merge test fixes
* fix test after changes in master
* address review nits
---------
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Sergej Sakac <73715684+Szegoo@users.noreply.github.com>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Marcin S <marcin@bytedude.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
* zombienet/pov-recovery: do not setup reserved connection for collators
* connect eve to dave
* REVERTME: diener update --polkadot --branch ao-recovery-try-connect
* REVERTME: cargo update -p sp-io
* REVERTME: cargo update -p sp-io and hope for the best
* pass reserved flags for eve after --
* Revert "REVERTME: cargo update -p sp-io and hope for the best"
This reverts commit 0d66b70e5ccedc7a35bcf1b30a070e68cd6a8ccc.
* Revert "REVERTME: cargo update -p sp-io"
This reverts commit f8d16c375afd41b8e0c84b59fe62019b86ba95b9.
* REVERTME: cargo update -p sp-io
* Revert "REVERTME: cargo update -p sp-io"
This reverts commit d48e1e6aa2134591b285e284077a063356d44eed.
* REVERTME: cargo update -p sp-io
* REVERTME: cargo update -p sp-io
* try some logs and flags
* apply some suggestions from Sebastian
* let Charlie produce blocks
* Revert "REVERTME: cargo update -p sp-io"
This reverts commit 02838a656eadfcbd26520e6b60ee21ffd8b27afd.
* git checkout master Cargo.lock
* Revert "REVERTME: diener update --polkadot --branch ao-recovery-try-connect"
This reverts commit 8ae8d5347b787ce5e08d6fdb5e7f266299fd70dd.
* zombienet/pov-recovery: 4 validators
* REVERTME: diener update --polkadot --branch ao-recovery-try-connect && cargo update -p polkadot-primitives
* Revert "REVERTME: diener update --polkadot --branch ao-recovery-try-connect && cargo update -p polkadot-primitives"
This reverts commit 3cc918220de5bf8ca5368586e55556464d106afd.
* make sure it is not due to backing group connections
* clean up flags
* try no-mdns on collators, limit out-peers on validators
* try no-mdns on validators and bob
* fix bob flag
* try 7 validators (3 chunks needed for recovery)
* reserved-only on validators
* reserved-only on alice and charlie
* explicitly pass log targets
* rm default_args
* parachain=debug and cumulus-pov-recovery=debug
* try another idea
* try 13 validators (4 chunks required)
* bump block height to 20
* Register parachain after 20sec, increase block target to 20
* bump zombienet version
* remove debug
* bump zombienet
* update lockfile for {"polkadot", "substrate"}
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <info@kchr.de>
* disputes: runtime part of slashing
* disputes: reward winners
* disputes/slashing: validate_unsigned impl
* fmt
* disputes/slashing: report_dispute_lost_unsigned
* disputes/slashing: separate winners from losers and report winners
* disputes/slashing: refactoring
* impl HandleReports
* enable on Wenstend
* fmt
* add slashing pallet to the mock and test runtimes
* fix a bug in report_dispute_lost_unsigned
* fmt
* disputes: remove new_participants from summary
* disputes: remove punish_inconclusive
* impl SlashingHandler for Pallet for type-safety
* do not impl slashing::Config on mainnets yet
* teach spellcheck deduplication
* simplify interfaces and resolve some TODOs
* resolve some more TODOs
* minor typos
* move slashing into a folder
* remove unnecessary clone
* fix validator_set_count calculation
* introduce ValidatorSetCount
* store ValidatorSetCount
* fmt
* add the benchmark
* fmt
* unflatten slashing
* post-rebase fixes
* remove winners eagerly
* use real slashing weights for westend
* remove bench test suite
* zombinet: modify disputes test to check for an offence report
* zombinet: add a timeout
* add slashing pallet to Rococo
* zombienet: revert back to rococo-local
* fmt
* remove TODOs
* revert some accidental changes
* slashing is submodule of disputes
* Change the log target
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* wrap comments with rustfmt, more docs, constants
* use Defensive trait
* cargo update -p sp-io
* merge offence types, remove rewards for now
* cargo update -p sp-io
* benchmark fixes
* fmt
* unused var
* fix block_author impl
* ressurect RewardValidators trait
* remove outdated comment
* more module docs
* introduce BenchmarkingConfig
* typo fix
* teach spellcheck unapplied
* use Weight::new()
* fix mocking rewards
* use RefTimeWeight
* ".git/.scripts/bench-bot.sh" runtime westend-dev runtime_parachains::disputes::slashing
* refactor maybe_identify_validators
* no more ticket in disguise
* remove outdated comments
* lower against valid to 0.1%
* bump zombienet version for debug
* use from_perthousand
* post-merge fixes
* another day, another Weight changes
* Revert "bump zombienet version for debug"
This reverts commit 0d9978711f8ec9a746a5e1c45e8ffbe7c75e7b5c.
* do not reward block authors
* fix outdated comment
* use Pays from frame_support::dispatch::Pays
* add timeout to is up
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
* refactor backing points to only reward active set
* impl disputes::RewardValidators
* enable rewards on westend, kusama, polkadot
* fmt
* make dispute points same as backing
* disable on polkadot for now
* zombienet: try to fix parachains upgrade test
* try a different collator image
* Revert "try a different collator image"
This reverts commit c83b2929f5a703be0979b4429ab01808408b074d.
* try setting cumulusBased
* fix zombienet upgrade test
* change image
* update command for test
* update zombienet image
* bump zombienet version
* bump zn version
* Revert "try setting cumulusBased"
This reverts commit 0312fa0a7a19065e097b186a08757938abd7c130.
* Revert "update command for test"
This reverts commit 81707e30188a1838e952d66deedaecbff6cb2971.
* Revert "change image"
This reverts commit 43a4f4a6a0286f5bf6698a44b82de835a723b531.
* zombienet: change validator image to a PR branch
Co-authored-by: Javier Viola <javier@parity.io>
* ci: add rococo for runtime changes check
* rococo: bump spec_version to 9180
* Update Rococo to 1-day lease (#4761)
* changed lease period to 1 day
* bumped version
* bumped version again
* changed 356 to 365 days to mimic one year
Co-authored-by: Santi Balaguer <santibalaguer@Santis-MacBook-Pro.local>
* rococo: bump spec_version to 9170 for consistency
Co-authored-by: Santi Balaguer <santiago.balaguer@gmail.com>
Co-authored-by: Santi Balaguer <santibalaguer@Santis-MacBook-Pro.local>
* approval-voting: add more logs
* approval-voting: query finalized block on startup and increase look back
* runtime-api: do not cache None SessionInfo
* collator-protocol: add to reserved peers on every relay parent
* bump collator slots from 25 to 100
* collator-protocol: reduce inactivity timeout from 24s to 5s
* try to satisfy spellcheck
* add connection log
* fmt
* bring a warn back
* gather validators across all active leaves
* add custom profiles and specify MSVR as 1.57
* fix host-perf-check command
* use testnet profile for CI images
* do not do lto for testnet profile
* fix artifact path
* test with testnet profile to reuse build artifacts
* Revert "fix host-perf-check command"
This reverts commit f1d15492204b8251685a97636cbb5a5f394f21da.
* bump zombienet version
Co-authored-by: Javier Viola <javier@parity.io>
* zombinet: fixed adder-collator image for the smoke test
* update COL_IMAGE
* bump zombienet version
* try a different version
Co-authored-by: Javier Viola <javier@parity.io>
* collator-protocol/validator: do not wait 1s to poll requested collations
* collator-protocol/validator: do not request collation for the next group
* put everything into select
* fmt
* more hacks yay
* a test
* review nits
* remove outdated comment
* validator-discovery: remove from peer set before inserting
* bump spec versions
* rework into a companion
* fmt
* fix
* fix
* one more time
* one more try
* one more try
* Revert "one more try"
This reverts commit ab6568d3b828a33dc06f5650037597fc88dd06b1.
* one more try
* one more try
* Revert "one more try"
This reverts commit 8d7369f7b78633bd1b1c5ba3e0f2a0544bdd77a5.
* fix a warning
* fix another warn
* correct log
* fix compilation
* ffs
* less cloning
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* add comments and a small refactoring
* use set_reserved_peers
* cargo update -p sp-io
* rename added to num_peers
* update Substrate
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: parity-processbot <>