* Move vstaging to production (and thus past session slashing).
WIP: test-runtime still needs to be fixed.
* Fix test-runtime.
---------
Co-authored-by: eskimor <eskimor@no-such-url.com>
* runtime/vstaging: unapplied_slashes runtime API
* runtime/vstaging: key_ownership_proof runtime API
* runtime/ParachainHost: submit_report_dispute_lost
* fix key_ownership_proof API
* runtime: submit_report_dispute_lost runtime API
* nits
* Update node/subsystem-types/src/messages.rs
Co-authored-by: Marcin S. <marcin@bytedude.com>
* revert unrelated fmt changes
* dispute-coordinator: past session dispute slashing
* encapsule runtime api call for submitting report
* prettify: extract into a function
* do not exit on runtime api error
* fix tests
* try initial zombienet test
* try something
* fix a typo
* try cumulus-based collator
* fix clippy
* build polkadot-debug images with fast-runtime enabled
* wip
* runtime/inclusion: fix availability_threshold
* fix wip
* fix wip II
* revert native provider
* propagate tx submission error
* DEBUG: sync=trace
* print key ownership proof len
* panic repro
* log validator index in panic message
* post merge fixes
* replace debug assertion with a log
* fix compilation
* Let's log the dispatch info in validate block.
* fix double encoding
* Revert "Let's log the dispatch info in validate block."
This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.
* Revert "Let's log the dispatch info in validate block."
This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.
* fix compilation
* update to latest zombienet and fix test
* lower finality lag to 11
* bump zombienet again
* add a workaround, but still does not work
* Update .gitlab-ci.yml
bump zombienet.
* add a comment and search logs on all nodes
---------
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Javier Viola <javier@parity.io>
* Properly set the pricing for the DMP router
* Publicize price types
* Use FixedU128 instead of Percent
* Add sp-arithmetic as a dependency for rococo runtime
* Add sp-arithmetic as a dependency to all runtimes
* Remove duplicate import
* Add missing import
* Fix tests
* Create an appropriate QueueDownwardMessageError variant
* Recalculate delivery fee factor based on past queue sizes
* Remove unused error variant
* Fixes
* Fixes
* Remove unused imports
* Rewrite fee factor update mechanism
* Remove unused imports
* Fixes
* Update runtime/parachains/src/dmp.rs
Co-authored-by: Squirrel <gilescope@gmail.com>
* Make DeliveryFeeFactor be a StorageMap keyed on ParaIds
* Fixes
* introduce limit for fee increase on dmp queue
* add message_size based fee factor to increment_fee_factor
* change message_size fee rate to correct value
* fix div by 0 error
* bind limit to variable
* fix message_size_factor and add DeliveryFeeFactor test
* add test for ExponentialPrice implementation
* make test formula based
* make delivery fee factor test formula based
* add max value test for DeliveryFeeFactor and move limit to config
* change threshold back to dynamic value and fix tests
* fmt
* suggested changes and fmt
* small stylistic change
* fmt
* change to tokenlocation
* small fixes
* fmt
* remove sp_arithmetic dependency
* Update runtime/parachains/src/dmp.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
---------
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Just van Stam <just.van.stam@gmail.com>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Happy New Year!
* Remove year entierly
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Remove years from copyright notice in the entire repo
---------
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Histogram support in runtime metrics
Add support for histograms to the runtime metrics. Additionally add
`polkadot_parachain_verify_dispute_signature` histogram which tracks the
time needed from the runtime to verify a single validator signature of a
dispute statement.
* Add noops
* u64 instead of f64
* Update buckets
* Wrap `get_current_time()` in runtime metrics
* Change the dimension of the Histogram from usec to sec
* Fix a compilation error
* Update buckets
* Fix `on_signature_check_complete` calculation
* Update buckets
* Update buckets
* formatting
* Another weights update
* Adjust buckets again
* Final buckets adjustment
* Revert "Fix a compilation error"
This reverts commit 06290b40a39eeb78de2602d8916a39edf7a8b714.
* Update primitives/src/v4/metrics.rs
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
* Use `saturating_sub` for time difference calculation
* Pass nanoseconds to client instead of seconds (using f64 in runtime is dangerous)
---------
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
* Some renames
* Fix
* Fix build for new APIs
* Remove diener
* Fixes
* Fixes
* Fix integration tests
* Fixes
* fix nis issuance
* Update Cargo.toml
* Polkadot doesn't have freezes/holds yet
* No networks use freezes/holds
* update lockfile for {"substrate"}
* Fix tests
There are more failing tests; just starting with the easy ones.
Reserved balance does not count towards ED anymore, therefore reducing
all the reserves by ED (1).
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fixes for Polkadot pallets
* Fix parachains benchmarks
* Update Substrate
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Remove use of Store trait from runtime directory
* Remove Store trait usage from xcm directory
* Run cargo fmt
* update lockfile for {"substrate"}
---------
Co-authored-by: parity-processbot <>
* Use a `BoundedVec` in `ValidationResult`
> Use a `BoundedVec` for `upward_messages` and `horizontal_messages` in order to
> limit the number of individual messages/memory allocations right at decoding
> time. The reason for this is that the `ValidationResult` may contain a code
> upgrade (including a full PVF binary), so the total size limit can't be set
> too low and this limit will still allow several millions of upward messages,
> which will (due to the memory allocator overhead) already have a
> non-negligible memory footprint in decoded form.
* List all fields when hashing so we don't miss one
* Define types for `BoundedVec`s of messages
* Fix test compile errors
* Depend on `bounded-collections` 0.1.4 (fixes allocation issue)
* Fix compilation issue
* Derive `Hash` instead of manual `impl`
* Avoid use of unwrap
* Re-apply changes without Diener, rebase to the lastest master
* Cache pruning
* Bit-pack InstantiationStrategy
* Move ExecutorParams version inside the structure itself
* Rework runtime API and executor parameters storage
* Pass executor parameters through backing subsystem
* Update Cargo.lock
* Introduce `ExecutorParams` to approval voting subsys
* Introduce `ExecutorParams` to dispute coordinator
* `cargo fmt`
* Simplify requests from backing subsys
* Fix tests
* Replace manual config cloning with `.clone()`
* Move constants to module
* Parametrize executor performing PVF pre-check
* Fix Malus
* Fix test runtime
* Introduce session executor params as a constant defined by session info
pallet
* Use Parity SCALE codec instead of hand-crafted binary encoding
* Get rid of constants; Add docs
* Get rid of constants
* Minor typo
* Fix Malus after rebase
* `cargo fmt`
* Use transparent SCALE encoding instead of explicit
* Clean up
* Get rid of relay parent to session index mapping
* Join environment type and version in a single enum element
* Use default execution parameters if running an old runtime
* `unwrap()` -> `expect()`
* Correct API version
* Constants are back in town
* Use constants for execution environment types
* Artifact separation, first try
* Get rid of explicit version
* PVF execution queue worker separation
* Worker handshake
* Global renaming
* Minor fixes resolving discussions
* Two-stage requesting of executor params to make use of runtime API cache
* Proper error handling in pvf-checker
* Executor params storage bootstrapping
* Propagate migration to v3 network runtimes
* Fix storage versioning
* Ensure `ExecutorParams` serialization determinism; Add comments
* Rename constants to make things a bit more deterministic
Get rid of stale code
* Tidy up a structure of active PVFs
* Minor formatting
* Fix comment
* Add try-runtime hooks
* Add storage version write on upgrade
Co-authored-by: Andronik <write@reusable.software>
* Add pre- and post-upgrade assertions
* Require to specify environment type; Remove redundant `impl`s
* Add `ExecutorParamHash` creation from `H256`
* Fix candidate validation subsys tests
* Return splittable error from executor params request fn
* Revert "Return splittable error from executor params request fn"
This reverts commit a0b274177d8bb2f6e13c066741892ecd2e72a456.
* Decompose approval voting metrics
* Use more relevant errors
* Minor formatting fix
* Assert a valid environment type instead of checking
* Fix `try-runtime` hooks
* After-merge fixes
* Add migration logs
* Remove dead code
* Fix tests
* Fix tests
* Back to the strongly typed implementation
* Promote strong types to executor interface
* Remove stale comment
* Move executor params to `SessionInfo`: primitives and runtime
* Move executor params to `SessionInfo`: node
* Try to bump primitives and API version
* Get rid of `MallocSizeOf`
* Bump target API version to v4
* Make use of session index already in place
* Back to v3
* Fix all the tests
* Add migrations to all the runtimes
* Make use of existing `SessionInfo` in approval voting subsys
* Rename `TARGET` -> `LOG_TARGET`
* Bump all the primitives to v3
* Fix Rococo ParachainHost API version
* Use `RollingSessionWindow` to acquire `ExecutorParams` in disputes
* Fix nits from discussions; add comments
* Re-evaluate queue logic
* Rework job assignment in execution queue
* Add documentation
* Use `RuntimeInfo` to obtain `SessionInfo` (with blackjack and caching)
* Couple `Pvf` with `ExecutorParams` wherever possible
* Put members of `PvfWithExecutorParams` under `Arc` for cheap cloning
* Fix comment
* Fix CI tests
* Fix clippy warnings
* Address nits from discussions
* Add a placeholder for raw data
* Fix non exhaustive match
* Remove redundant reexports and fix imports
* Keep only necessary semantic features, as discussed
* Rework `RuntimeInfo` to support mock implementation for tests
* Remove unneeded bound
* `cargo fmt`
* Revert "Remove unneeded bound"
This reverts commit 932463f26b00ce290e1e61848eb9328632ef8a61.
* Fix PVF host tests
* Fix PVF checker tests
* Fix overseer declarations
* Simplify tests
* `MAX_KEEP_WAITING` timeout based on `BACKGING_EXECUTION_TIMEOUT`
* Add a unit test for varying executor parameters
* Minor fixes from discussions
* Add prechecking max. memory parameter (see paritytech/srlabs_findings#110)
* Fix and improve a test
* Remove `ExecutionEnvironment` and `RawData`
* New primitives versioning in parachain host API
* `disputes()` implementation for Kusama and Polkadot
* Move `ExecutorParams` from `vstaging` to stable primitives
* Move disputes from `vstaging` to stable implementation
* Fix `try-runtime`
* Fixes after merge
* Move `ExecutorParams` to the bottom of `SessionInfo`
* Revert "Move executor params to `SessionInfo`: primitives and runtime"
This reverts commit dfcfb85fefd1c5be6c8a8f72dc09fd1809cfa9ce.
* Always use fresh activated live hash in pvf precheck
(re-apply 34b09a4c20de17e7926ed942cd0d657d18f743fa)
* Fixing tests (broken commit)
* Fix candidate validation tests
* Fix PVF host test
* Minor fixes
* Address discussions
* Restore migration
* Fix `use` to only include what is needed instead of `*`
* Add comment to never touch `DEFAULT_CONFIG`
* Update migration to set default `ExecutorParams` for `dispute_period`
sessions back
* Use `earliest_stored_session` instead of calculations
* Nit
* Add logs
* Treat any runtime error as `NotSupported` again
* Always return default executor params if not available
* Revert "Always return default executor params if not available"
This reverts commit b58ac4482ef444c67a9852d5776550d08e312f30.
* Add paritytech/substrate#9997 workaround
* `cargo fmt`
* Remove migration (again!)
* Bump executor params to API v4 (backport from #6698)
---------
Co-authored-by: Andronik <write@reusable.software>
* disputes/slashing: slash only backers for ForInvalid
* add an assertion in mock impl
* fix tests
* do not slash backers on onconcluded disputes
* slash an intersection of backers and losers
* zombienet/disputes: check for offence only for invalid disputes
* add backing votes to disputes bench builder
* Update runtime/parachains/src/builder.rs
* Brad implementers guide revisions 2 (#6239)
* Add disputes subsystems fix
* Updated dispute approval vote import reasoning
* Improved wording of my changes
* Resolving issues brought up in comments
* Update disputes prioritisation in `dispute-coordinator` (#6130)
* Scraper processes CandidateBacked events
* Change definition of best-effort
* Fix `dispute-coordinator` tests
* Unit test for dispute filtering
* Clarification comment
* Add tests
* Fix logic
If a dispute is not backed, not included and not confirmed we
don't participate but we do import votes.
* Add metrics for refrained participations
* Revert "Add tests"
This reverts commit 7b8391a087922ced942cde9cd2b50ff3f633efc0.
* Revert "Unit test for dispute filtering"
This reverts commit 92ba5fe678214ab360306313a33c781338e600a0.
* fix dispute-coordinator tests
* Fix scraping
* new tests
* Small fixes in guide
* Apply suggestions from code review
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
* Fix some comments and remove a pointless test
* Code review feedback
* Clarification comment in tests
* Some tests
* Reference counted `CandidateHash` in scraper
* Proper handling for Backed and Included candidates in scraper
Backed candidates which are not included should be kept for a
predetermined window of finalized blocks. E.g. if a candidate is backed
but not included in block 2, and the window size is 2, the same
candidate should be cleaned after block 4 is finalized.
Add reference counting for candidates in scraper. A candidate can be
added on multiple block heights so we have to make sure we don't clean
it prematurely from the scraper.
Add tests.
* Update comments in tests
* Guide update
* Fix cleanup logic for `backed_candidates_by_block_number`
* Simplify cleanup
* Make spellcheck happy
* Update tests
* Extract candidate backing logic in separate struct
* Code review feedback
* Treat backed and included candidates in the same fashion
* Update some comments
* Small improvements in test
* spell check
* Fix some more comments
* clean -> prune
* Code review feedback
* Reword comment
* spelling
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
* approval-voting: remove redundant validation check (#6266)
* approval-voting: remove a redundant check
* candidate-validation: remove unreachable check
* remove fill_block (#6200)
Co-authored-by: parity-processbot <>
* fix a compilation warning (#6279)
Fixes#6277.
* Only report concluded if there is an actual dispute. (#6270)
* Only report concluded if there is an actual dispute.
Hence no "non"-disputes will be added to disputes anymore.
* Fix redundant check.
* Test for no onesided disputes.
Co-authored-by: eskimor <eskimor@no-such-url.com>
* [ci] fix buildah image (#6281)
* Revert special casing of Kusama for grandpa rounds. (#6217)
Co-authored-by: eskimor <eskimor@no-such-url.com>
* Fixes "for loop over an `Option`" warnings (#6291)
Was seeing these warnings when running `cargo check --all`:
```
warning: for loop over an `Option`. This is more readably written as an `if let` statement
--> node/core/approval-voting/src/lib.rs:1147:21
|
1147 | for activated in update.activated {
| ^^^^^^^^^^^^^^^^
|
= note: `#[warn(for_loops_over_fallibles)]` on by default
help: to check pattern in a loop use `while let`
|
1147 | while let Some(activated) = update.activated {
| ~~~~~~~~~~~~~~~ ~~~
help: consider using `if let` to clear intent
|
1147 | if let Some(activated) = update.activated {
| ~~~~~~~~~~~~ ~~~
```
My guess is that `activated` used to be a SmallVec or similar, as is
`deactivated`. It was changed to an `Option`, the `for` still compiled (it's
technically correct, just weird), and the compiler didn't catch it until now.
* companion for #12599 (#6290)
* companion for #12599
* update Cargo.lock
* use cargo path instead of diener
* update lockfile for {"substrate"}
Co-authored-by: parity-processbot <>
* remove the runtime check and test
* append keys on past-session slashing
* runtime/disputes: allow importing backing votes after explicit for
* explicit MaliciousBacker error and a test
* update an outdated comment
* Revert "update an outdated comment"
This reverts commit 7c4c3f5a848f16e2b61435e981d814f00333ed41.
* Revert "remove the runtime check and test"
This reverts commit a5bff0c75e77effb5b7d3a1691de1b14bcdbd648.
* incremental punishment post conclusion + test
* punish backers post FOR vote
* remove unnecessary lifetime annotation
* add a comment to zombinet test
* typo
* fmt
* post merge test fixes
* fix test after changes in master
* address review nits
---------
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Sergej Sakac <73715684+Szegoo@users.noreply.github.com>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Marcin S <marcin@bytedude.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Anton