Commit Graph

3655 Commits

Author SHA1 Message Date
alexgparity 9ea14e66c8 Clippyfy (#6341)
* Add clippy config and remove .cargo from gitignore

* first fixes

* Clippyfied

* Add clippy CI job

* comment out rusty-cachier

* minor

* fix ci

* remove DAG from check-dependent-project

* add DAG to clippy

Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-11-30 08:34:06 +00:00
Mara Robin B b76086c617 sync versions with current release (0.9.33) (#6363)
* westend: update transaction version

* polkadot: update transaction version

* kusama: update transaction version

* Bump spec_version to 9330

* bump versions to 0.9.33
2022-11-30 10:32:20 +02:00
joe petrowski 8042b1cb2a Add Collectives as Trusted Teleporter (#6326)
* add collectives as trusted teleporter

* fix statemint-specific doc
2022-11-29 16:14:47 +00:00
Marcin S b5d5a7ee4f Remove parity-util-mem from runtime-api cache (#6367) 2022-11-29 15:28:44 +00:00
Serban Iorga 4cc40bf08f Companion for: pallet-mmr: move offchain logic to client-side gadget (#6321)
* Spawn MMR gadget when offchain indexing is enabled

* companion PR code review changes: 1st iteration

* Code review changes: 2nd iteration

* update lockfile for {"substrate"}

Co-authored-by: acatangiu <adrian@parity.io>
Co-authored-by: parity-processbot <>
2022-11-29 15:23:34 +00:00
Mara Robin B 97bcb6dfe4 remove executed migrations (0.9.33) (#6364) 2022-11-29 14:33:24 +00:00
Andrei Sandu eb00a21ef7 dispute-coordinator: fix earliest session checks for pruning and import (#6358)
* RollingSession: add fn contains

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* handle_import_statements fix ancient dispute check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* work with earliest session instead of latest

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-11-28 17:43:13 +01:00
Alexander Samusev b108fd854b [ci] fix implementer guide job (#6357)
* [DNM] debug implementer guide job

* remove git depth

* change git strategy

* add git depth

* try k8s runner

* fix k8s template

* test job

* fix test

* fix

* return pipeline back

* enable disabled deploy-parity-testnet
2022-11-28 16:31:49 +01:00
Marcin S 186a572907 Provide some more granular metrics for polkadot_pvf_execution_time (#6346) 2022-11-28 10:26:17 -05:00
Mattia L.V. Bradascio 893a3ad784 Add more granularity to prometheus histogram buckets (#6348)
* Add buckets below 5ms

* Add more specific histogram buckets

* Add more specific buckets

* cargo fmt
2022-11-28 11:30:27 +00:00
ordian 5311911a39 cargo update -p sp-io (#6351) 2022-11-27 14:33:59 +00:00
Sebastian Kunert 53d9a50c06 Add support for outbound only configs on request/response protocols (#6343)
* Add option ot add outbound_only configurations

* Improve comment
2022-11-26 10:22:09 +01:00
Marcin S 01f8154110 ig: Fix description of execution retry delay (#6342)
The delay is 3s and not 1s. I removed the reference to a specific number of
seconds as it may be too specific for a high-level description.
2022-11-24 08:06:56 -05:00
tugy 7a6460e132 Added Amforc bootnodes for Polkadot and Kusama (#6077)
* add wss and update dns

* fix wrong node id
2022-11-24 14:06:48 +01:00
Alexander Samusev bc25c41aaf [ci] fix build-implementers-guide (#6335)
* [ci] fix build-implementers-guide

* fix user
2022-11-24 10:06:07 +01:00
eskimor 7ba0847349 Rate limit improvements (#6315)
* We actually don't need to rate limit redundant requests.

Those redundant requests should not actually happen, but still.

* Add some logging.

* Also log message when the receiving side hit the rate limit.

* Update node/network/dispute-distribution/src/sender/mod.rs

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
2022-11-23 16:43:30 +01:00
Marcin S 1dec2433ae Add PVF module documentation (#6293)
* Add PVF module documentation

TODO (once the PRs land):

- [ ] Document executor parametrization.

- [ ] Document CPU time measurement of timeouts.

* Update node/core/pvf/src/lib.rs

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* Clarify meaning of PVF acronym

* Move PVF doc to implementer's guide

* Clean up implementer's guide a bit

* Add page for PVF types

* pvf: Better separation between crate docs and implementer's guide

* ci: Add "prevalidating" to the dictionary

* ig: Remove types/chain.md

The types contained therein did not exist and the file was not referenced
anywhere.

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
2022-11-23 08:20:25 -05:00
Aaro Altonen 17ad9f5694 Update async-trait version to v0.1.58 (#6319)
* Update async-trait version

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-11-22 09:04:18 +00:00
Koute 9040eb303c Extend lower bound of manage_lease_period_start from runtime_common::slots (#6318) 2022-11-21 11:40:47 +00:00
joe petrowski 8456b2ec90 Add starts_with to v0 and v1 MultiLocation (#6311)
* add `starts_with` to v0 and v1 MultiLocation

* add tests

* fmt
2022-11-20 17:31:06 +01:00
Koute 9c2561f07a Remove the wasmtime feature flag (companion for substrate#12684) (#6268)
* Remove the `wasmtime` feature flag

* Update `substrate` to the newest `master`

* Update `substrate` to the newest `master`
2022-11-18 14:39:40 +00:00
Alexander Samusev d19484e972 [ci] fix build implementers guide (#6306)
* [ci] fix build implementers guide

* add comment

* rm git fetch from publish-docs
2022-11-17 16:42:46 +01:00
Tsvetomir Dimitrov ccad411e46 Change best effort queue behaviour in dispute-coordinator (#6275)
* Change best effort queue behaviour in `dispute-coordinator`

Use the same type of queue (`BTreeMap<CandidateComparator,
ParticipationRequest>`) for best effort and priority in
`dispute-coordinator`.

Rework `CandidateComparator` to handle unavailable parent
block numbers.

Best effort queue will order disputes the same way as priority does - by
parent's block height. Disputes on candidates for which the parent's
block number can't be obtained will be treated with the lowest priority.

* Fix tests: Handle `ChainApiMessage::BlockNumber` in `handle_sync_queries`

* Some tests are deadlocking on sending messages via overseer so change `SingleItemSink`to `mpsc::Sender` with a buffer of 1

* Fix a race in test after adding a buffered queue for overseer messages

* Fix the rest of the tests

* Guide update - best-effort queue

* Guide update: clarification about spam votes

* Fix tests in `availability-distribution`

* Update comments

* Add `make_buffered_subsystem_context` in `subsystem-test-helpers`

* Code review feedback

* Code review feedback

* Code review feedback

* Don't add best effort candidate if it is already in priority queue

* Remove an old comment

* Fix insert in best_effort
2022-11-17 15:41:19 +00:00
Andrei Sandu ad41e56e6e Dedup subsystem name (#6305)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-11-17 12:59:07 +00:00
Kutsal Kaan Bilgin 0c9187a01a Add Helikon boot nodes for Polkadot, Kusama and Westend. (#6240) 2022-11-17 12:22:25 +01:00
Tsvetomir Dimitrov a0f4287dd9 Provisioner should ignore unconfirmed disputes (#6294)
* Fix typos

* Filter unconfirmed disputes in provisioner -  random_selection

* Rework dispute coordinator to return `DisputeStatus` with
`ActiveDisputes` message.
* Rework the random_selection implementation of `select_disptues` in
  `provisioner` to return only confirmed disputes.

* Filter unconfirmed disputes in provisioner - prioritized_selection

* Add test for unconfirmed disputes handling

* Fix `dispute-distribution` tests
2022-11-16 14:39:05 +00:00
Alexander Samusev 40221806d4 [ci] Improve pipeline stopper (#6300)
* [ci] Improve pipeline stopper

* break test-linux-stable

* fix test-linux-stable
2022-11-16 14:49:17 +01:00
Niklas Adolfsson 4b9f3aa82c companion for #12599 (#6290)
* companion for #12599

* update Cargo.lock

* use cargo path instead of diener

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-11-15 17:30:50 +01:00
Marcin S d53513ff66 Fixes "for loop over an Option" warnings (#6291)
Was seeing these warnings when running `cargo check --all`:

```
warning: for loop over an `Option`. This is more readably written as an `if let` statement
    --> node/core/approval-voting/src/lib.rs:1147:21
     |
1147 |             for activated in update.activated {
     |                              ^^^^^^^^^^^^^^^^
     |
     = note: `#[warn(for_loops_over_fallibles)]` on by default
help: to check pattern in a loop use `while let`
     |
1147 |             while let Some(activated) = update.activated {
     |             ~~~~~~~~~~~~~~~         ~~~
help: consider using `if let` to clear intent
     |
1147 |             if let Some(activated) = update.activated {
     |             ~~~~~~~~~~~~         ~~~
```

My guess is that `activated` used to be a SmallVec or similar, as is
`deactivated`. It was changed to an `Option`, the `for` still compiled (it's
technically correct, just weird), and the compiler didn't catch it until now.
2022-11-15 09:58:26 -05:00
eskimor c606deb32d Revert special casing of Kusama for grandpa rounds. (#6217)
Co-authored-by: eskimor <eskimor@no-such-url.com>
2022-11-15 10:04:24 +01:00
Alexander Samusev da4df1809d [ci] fix buildah image (#6281) 2022-11-14 12:59:56 +01:00
eskimor 97dddbab93 Only report concluded if there is an actual dispute. (#6270)
* Only report concluded if there is an actual dispute.

Hence no "non"-disputes will be added to disputes anymore.

* Fix redundant check.

* Test for no onesided disputes.

Co-authored-by: eskimor <eskimor@no-such-url.com>
2022-11-14 12:41:45 +01:00
Andronik e4eb6266f8 fix a compilation warning (#6279)
Fixes #6277.
2022-11-13 19:14:24 +00:00
Sergej Sakac 92ea52b112 remove fill_block (#6200)
Co-authored-by: parity-processbot <>
2022-11-12 14:14:35 +00:00
Andronik 283f40f3ed approval-voting: remove redundant validation check (#6266)
* approval-voting: remove a redundant check

* candidate-validation: remove unreachable check
2022-11-12 11:52:26 +00:00
Tsvetomir Dimitrov ff09b18d31 Update disputes prioritisation in dispute-coordinator (#6130)
* Scraper processes CandidateBacked events

* Change definition of best-effort

* Fix `dispute-coordinator` tests

* Unit test for dispute filtering

* Clarification comment

* Add tests

* Fix logic

If a dispute is not backed, not included and not confirmed we
don't participate but we do import votes.

* Add metrics for refrained participations

* Revert "Add tests"

This reverts commit 7b8391a087922ced942cde9cd2b50ff3f633efc0.

* Revert "Unit test for dispute filtering"

This reverts commit 92ba5fe678214ab360306313a33c781338e600a0.

* fix dispute-coordinator tests

* Fix scraping

* new tests

* Small fixes in guide

* Apply suggestions from code review

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* Fix some comments and remove a pointless test

* Code review feedback

* Clarification comment in tests

* Some tests

* Reference counted `CandidateHash` in scraper

* Proper handling for Backed and Included candidates in scraper

Backed candidates which are not included should be kept for a
predetermined window of finalized blocks. E.g. if a candidate is backed
but not included in block 2, and the window size is 2, the same
candidate should be cleaned after block 4 is finalized.

Add reference counting for candidates in scraper. A candidate can be
added on multiple block heights so we have to make sure we don't clean
it prematurely from the scraper.

Add tests.

* Update comments in tests

* Guide update

* Fix cleanup logic for `backed_candidates_by_block_number`

* Simplify cleanup

* Make spellcheck happy

* Update tests

* Extract candidate backing logic in separate struct

* Code review feedback

* Treat  backed and included candidates in the same fashion

* Update some comments

* Small improvements in test

* spell check

* Fix some more comments

* clean -> prune

* Code review feedback

* Reword comment

* spelling

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
2022-11-11 22:47:29 +01:00
Bradley Olson 0cea4eedce Brad implementers guide revisions 2 (#6239)
* Add disputes subsystems fix

* Updated dispute approval vote import reasoning

* Improved wording of my changes

* Resolving issues brought up in comments
2022-11-11 20:24:40 +01:00
cheme 79d9ea4f62 State trie migration rococo runtime changes. (#6127)
* add state-trie-migration (warn key need to be changed)

* rococo root

* restore master benchs (weights from substrate are used).

* use ord_parameter macro.

* do not upgrade runtime version yet

* apply review changes

* to test ci

* Revert "to test ci"

This reverts commit 5df6c5c74c5d172f254579cbb76caeb44af80ec2.

* test ci

* Revert "test ci"

This reverts commit 0747761644ba566f9756e195e5304758370dc52c.

Co-authored-by: parity-processbot <>
2022-11-11 11:06:44 +00:00
moh-eulith de7acfac5b Add a few staking params to fast-runtime build (#5424)
Co-authored-by: Squirrel <gilescope@gmail.com>
2022-11-10 13:40:45 +00:00
Robert Hambrock 20965b4e64 Companion for substrate#12530: Consolidate and deduplicate MMR API methods (#6167)
* histor. batch proof: make best block arg optional

* make generate_batch_proof stub for historical

* merge generate_{historical_}batch_proof functions

* merge generate_{batch_}proof functions

* merge verify_{batch_}proof functions

* merge verify_{batch_}proof_stateless functions

* rename BatchProof->Proof

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-11-09 16:04:48 +00:00
Qinxuan Chen 7ccc92414a Companion for substrate#12560 (#6226)
* Companion for substrate#12560

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* update num-format v0.4.0 ==> v0.4.3

* Fix

* update lockfile for {"substrate"}

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: parity-processbot <>
2022-11-09 12:18:13 +00:00
Ankan a3fefccbeb [Companion] Bound Election and Staking by MaxActiveValidators (#6157)
* add maximum winners to multi phase election provider

* fallback to noelection

* fmt

* missing values

* convert boundedvec to inner before sort

* dont clone

* pr feedback

* update lockfile for {"substrate"}

* run onchain election on westend benchmark

Co-authored-by: parity-processbot <>
2022-11-09 12:00:03 +01:00
Marcin S 6d7f33e612 Retry failed PVF execution (AmbiguousWorkerDeath) (#6235)
* Fix a couple of typos

* Retry failed PVF execution

PVF execution that fails due to AmbiguousWorkerDeath should be retried once.
This should reduce the occurrence of failures due to transient conditions.

Closes #6195

* Address a couple of nits

* Write tests; refactor (add `validate_candidate_with_retry`)

* Update node/core/candidate-validation/src/lib.rs

Co-authored-by: Andronik <write@reusable.software>

Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: Andronik <write@reusable.software>
2022-11-08 15:36:36 -05:00
Andrei Sandu c261353380 Make rolling session more resilient in case of long finality stalls (#6106)
* Impl dynamic window size. Keep sessions for unfinalized chain

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Stretch also in contructor plus  tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix approval-voting tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* grunting: dispute coordinator tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add session window column

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* integrate approval vote and fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix rolling session tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Small refactor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* WIP, tests failing

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix approval voting tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix dispute-coordinator tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove uneeded param

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix loose ends

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* allow failure and tests for it

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* comment fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* style fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* new col doesn't need to be ordered

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt and spellcheck

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* db persist tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add v2 config and cols

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* DB upgrade WIP

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add todo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update to parity-db to "0.4.2"

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* migration complete

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* One session window size

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build errors

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* comment fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* make error more explicit

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* refactor conflict merge

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* rename col_data

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add doc comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* migration: move all cols to v2

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-11-08 18:16:31 +01:00
Kian Paimani be4b5fad47 companion for fast unstake batching (#6253)
* update

* add migrations

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-11-08 16:57:57 +00:00
Niklas Adolfsson e1ccb8499d staking miner: remove needless queue check (#6221)
* staking miner: remove needless queue check

If the queue is full and the "mined solution" is better than solutions in the queue according to the "strategy"
then the solution with worse score will be kicked out from the queue.

Thus, the check can be removed completly.

* fix compile warns
2022-11-08 16:19:48 +00:00
Marcin S 63c6f184cb Retry failed PVF prepare jobs (#6213) 2022-11-08 13:30:14 +00:00
Marcin S 6cc91f9187 Add a last change footer to the implementers guide (#6216)
* Add a `last change` footer to the implementers guide

Some of the newcomers were noticing outdated pages in the implementer's guide.
This idea came up as a heuristic for how up-to-date an individual page is.

* Update `build-implementers-guide` CI job
2022-11-08 13:06:02 +00:00
Kian Paimani fb40925ebe Update polkadot inflation to take into account auctions (#5872)
* Update polkadot inflation to take into account auctions

* a possible solution -- but needs a rather untrivial data seeding

* some additional comments

* Use LOWEST_PUBLIC_ID as a guide to filter out system/common good para ids

* Fixes

* move tests

* fix

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-11-08 08:04:46 +00:00
Kian Paimani 1ed5e1ad7e Increase max rewardable nominators (#6230)
* Increase max rewardable nominators

* update kusama as well
2022-11-08 08:03:54 +00:00