We currently use a bit of a hack in `.cargo/config` to make sure that
clippy isn't too annoying by specifying the list of lints.
There is now a stable way to define lints for a workspace. The only down
side is that every crate seems to have to opt into this so there's a
*few* files modified in this PR.
Dependencies:
- [x] PR that upgrades CI to use rust 1.74 is merged.
---------
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Initial implementation for the plan discussed here: https://github.com/paritytech/polkadot-sdk/issues/701
Built on top of https://github.com/paritytech/polkadot-sdk/pull/1178
v0: https://github.com/paritytech/polkadot/pull/7554,
## Overall idea
When approval-voting checks a candidate and is ready to advertise the
approval, defer it in a per-relay chain block until we either have
MAX_APPROVAL_COALESCE_COUNT candidates to sign or a candidate has stayed
MAX_APPROVALS_COALESCE_TICKS in the queue, in both cases we sign what
candidates we have available.
This should allow us to reduce the number of approvals messages we have
to create/send/verify. The parameters are configurable, so we should
find some values that balance:
- Security of the network: Delaying broadcasting of an approval
shouldn't but the finality at risk and to make sure that never happens
we won't delay sending a vote if we are past 2/3 from the no-show time.
- Scalability of the network: MAX_APPROVAL_COALESCE_COUNT = 1 &
MAX_APPROVALS_COALESCE_TICKS =0, is what we have now and we know from
the measurements we did on versi, it bottlenecks
approval-distribution/approval-voting when increase significantly the
number of validators and parachains
- Block storage: In case of disputes we have to import this votes on
chain and that increase the necessary storage with
MAX_APPROVAL_COALESCE_COUNT * CandidateHash per vote. Given that
disputes are not the normal way of the network functioning and we will
limit MAX_APPROVAL_COALESCE_COUNT in the single digits numbers, this
should be good enough. Alternatively, we could try to create a better
way to store this on-chain through indirection, if that's needed.
## Other fixes:
- Fixed the fact that we were sending random assignments to
non-validators, that was wrong because those won't do anything with it
and they won't gossip it either because they do not have a grid topology
set, so we would waste the random assignments.
- Added metrics to be able to debug potential no-shows and
mis-processing of approvals/assignments.
## TODO:
- [x] Get feedback, that this is moving in the right direction. @ordian
@sandreim @eskimor @burdges, let me know what you think.
- [x] More and more testing.
- [x] Test in versi.
- [x] Make MAX_APPROVAL_COALESCE_COUNT &
MAX_APPROVAL_COALESCE_WAIT_MILLIS a parachain host configuration.
- [x] Make sure the backwards compatibility works correctly
- [x] Make sure this direction is compatible with other streams of work:
https://github.com/paritytech/polkadot-sdk/issues/635 &
https://github.com/paritytech/polkadot-sdk/issues/742
- [x] Final versi burn-in before merging
---------
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Using taplo, fixes all our broken and inconsistent toml formatting and
adds CI to keep them tidy.
If people want we can customise the format rules as described here
https://taplo.tamasfe.dev/configuration/formatter-options.html
@ggwpez, I suggest zepter is used only for checking features are
propagated, and leave formatting for taplo to avoid duplicate work and
conflicts.
TODO
- [x] Use `exclude = [...]` syntax in taplo file to ignore zombienet
tests instead of deleting the dir
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
**_PR migrated from https://github.com/paritytech/polkadot/pull/6782_**
This PR will upgrade the network protocol to version 3 -> VStaging which
will later be renamed to V3. This version introduces a new kind of
assignment certificate that will be used for tranche0 assignments.
Instead of issuing/importing one tranche0 assignment per candidate,
there will be just one certificate per relay chain block per validator.
However, we will not be sending out the new assignment certificates,
yet. So everything should work exactly as before. Once the majority of
the validators have been upgraded to the new protocol version we will
enable the new certificates (starting at a specific relay chain block)
with a new client update.
There are still a few things that need to be done:
- [x] Use bitfield instead of Vec<CandidateIndex>:
https://github.com/paritytech/polkadot/pull/6802
- [x] Fix existing approval-distribution and approval-voting tests
- [x] Fix bitfield-distribution and statement-distribution tests
- [x] Fix network bridge tests
- [x] Implement todos in the code
- [x] Add tests to cover new code
- [x] Update metrics
- [x] Remove the approval distribution aggression levels: TBD PR
- [x] Parachains DB migration
- [x] Test network protocol upgrade on Versi
- [x] Versi Load test
- [x] Add Zombienet test
- [x] Documentation updates
- [x] Fix for sending DistributeAssignment for each candidate claimed by
a v2 assignment (warning: Importing locally an already known assignment)
- [x] Fix AcceptedDuplicate
- [x] Fix DB migration so that we can still keep old data.
- [x] Final Versi burn in
---------
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Bumps the known_good_semver group with 1 update:
[clap](https://github.com/clap-rs/clap).
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.4.6</h2>
<h2>[4.4.6] - 2023-09-28</h2>
<h3>Internal</h3>
<ul>
<li>Upgrade <code>anstream</code></li>
</ul>
<h2>v4.4.5</h2>
<h2>[4.4.5] - 2023-09-25</h2>
<h3>Fixes</h3>
<ul>
<li><em>(parser)</em> When inferring subcommand <code>name</code> or
<code>long_flag</code>, allow ambiguous-looking matches that
unambiguously map back to the same command</li>
<li><em>(parser)</em> When inferring subcommand <code>long_flag</code>,
don't panic</li>
<li><em>(assert)</em> Clarify what action is causing a positional that
doesn't set values which is especially useful for derive users</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.4.6] - 2023-09-28</h2>
<h3>Internal</h3>
<ul>
<li>Upgrade <code>anstream</code></li>
</ul>
<h2>[4.4.5] - 2023-09-25</h2>
<h3>Fixes</h3>
<ul>
<li><em>(parser)</em> When inferring subcommand <code>name</code> or
<code>long_flag</code>, allow ambiguous-looking matches that
unambiguously map back to the same command</li>
<li><em>(parser)</em> When inferring subcommand <code>long_flag</code>,
don't panic</li>
<li><em>(assert)</em> Clarify what action is causing a positional that
doesn't set values which is especially useful for derive users</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/clap-rs/clap/commit/39f5e807af1c08acedbf7343ce9ec379a4308636"><code>39f5e80</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/a5cb6bb988bbacb02e8cf98b6156c860d0801e08"><code>a5cb6bb</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/418c0017a654e9859adfa9b051815f20e4583e31"><code>418c001</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5146">#5146</a>
from epage/update</li>
<li><a
href="https://github.com/clap-rs/clap/commit/485b957c4b90aa010276f813dbd429e1071f8fd9"><code>485b957</code></a>
chore: Upgrade anstream</li>
<li><a
href="https://github.com/clap-rs/clap/commit/a1af8d9ad8f81eb2d71203b50c370a78ce3ec9f3"><code>a1af8d9</code></a>
chore: Update from '_rust/main'</li>
<li><a
href="https://github.com/clap-rs/clap/commit/ac51f0925003597dec21529538597dbd7872d1ac"><code>ac51f09</code></a>
chore(ci): Normalize json5 syntax</li>
<li><a
href="https://github.com/clap-rs/clap/commit/86c29dea384c7392a2b682fa0150f52c0f4c7f00"><code>86c29de</code></a>
chore(ci): Updaet Renovate schema</li>
<li><a
href="https://github.com/clap-rs/clap/commit/204552890d316ec9ae0b21f85298ba1d5d0786f8"><code>2045528</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/55d223001682bc668f5e4db91afd5e76c2a36597"><code>55d2230</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/492ee03b325ff98c7702295e024576b52b71358d"><code>492ee03</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5140">#5140</a>
from epage/dyn</li>
<li>Additional commits viewable in <a
href="https://github.com/clap-rs/clap/compare/v4.4.4...v4.4.6">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
[//]: # (dependabot-start)
⚠️ **Dependabot is rebasing this PR** ⚠️
Rebasing might not happen immediately, so don't worry if this takes some
time.
Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.
---
[//]: # (dependabot-end)
Bumps the known_good_semver group with 2 updates:
[clap](https://github.com/clap-rs/clap) and
[syn](https://github.com/dtolnay/syn).
Updates `clap` from 4.4.3 to 4.4.4
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.4.4</h2>
<h2>[4.4.4] - 2023-09-18</h2>
<h3>Internal</h3>
<ul>
<li>Update <code>terminal_size</code> to 0.3</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.4.4] - 2023-09-18</h2>
<h3>Internal</h3>
<ul>
<li>Update <code>terminal_size</code> to 0.3</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/clap-rs/clap/commit/e6e539660f36487e3521ab6835ef1381a21785a4"><code>e6e5396</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/acbb60c11389d362e3b2d23a7b2a0a0523bd2fa8"><code>acbb60c</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/f09d521450c50f56153f7246489666c71e3b4739"><code>f09d521</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5129">#5129</a>
from cgwalters/widen-terminal-size</li>
<li><a
href="https://github.com/clap-rs/clap/commit/84f99ff979d58e3fe94716144b1800fd81723ed7"><code>84f99ff</code></a>
chore(builder): Bump terminal_size to 0.3</li>
<li><a
href="https://github.com/clap-rs/clap/commit/7f8df272d90afde89e40de086492e1c9f5749897"><code>7f8df27</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5124">#5124</a>
from devinherron/master</li>
<li><a
href="https://github.com/clap-rs/clap/commit/4dff87386a950d8dfa8e89906e9bf54d180d13de"><code>4dff873</code></a>
fix(doc): Fix typo in 03_04_subcommands.md</li>
<li>See full diff in <a
href="https://github.com/clap-rs/clap/compare/v4.4.3...v4.4.4">compare
view</a></li>
</ul>
</details>
<br />
Updates `syn` from 2.0.36 to 2.0.37
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/syn/releases">syn's
releases</a>.</em></p>
<blockquote>
<h2>2.0.37</h2>
<ul>
<li>Work around incorrect future compatibility warning in rustc
1.74.0-nightly</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dtolnay/syn/commit/96810880f3acbb63415cf4684ab42c82edc31e2a"><code>9681088</code></a>
Release 2.0.37</li>
<li><a
href="https://github.com/dtolnay/syn/commit/fbe3bc2ddcee4192f95697a953330d031030f5a1"><code>fbe3bc2</code></a>
Work around unknown_lints warning on rustc older than 1.64</li>
<li><a
href="https://github.com/dtolnay/syn/commit/75cf912e061ef5a1d97c003f4988f43d7639f5a8"><code>75cf912</code></a>
Ignore more repr_transparent_external_private_fields</li>
<li><a
href="https://github.com/dtolnay/syn/commit/299c782439823b868de4aea3682b41d03351d978"><code>299c782</code></a>
Ignore false repr_transparent_external_private_fields FCW in generated
code</li>
<li>See full diff in <a
href="https://github.com/dtolnay/syn/compare/2.0.36...2.0.37">compare
view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Closes#583
After the separation of PVF worker binaries, dedicated puppet workers
are not needed for tests anymore. The production workers can be used
instead, avoiding some code duplication and decreasing complexity.
The changes also make it possible to further refactor the code to
isolate workers completely.
* Rename `polkadot-parachain` to `polkadot-parachain-primitives`
While doing this it also fixes some last `rustdoc` issues and fixes
another Cargo warning related to `pallet-paged-list`.
* Fix compilation
* ".git/.scripts/commands/fmt/fmt.sh"
* Fix XCM docs
---------
Co-authored-by: command-bot <>
* pvf: use test-utils feature to export test only
* adding comment to test-utils feature
* make prepare-worker and execute-worker as optional dependencies and add comments to test-utils
* remove doc hidden from pvf testing
* add prepare worker and execute worker entrypoints to test-utils feature
* pvf: add sp_tracing as optional dependency of test-utils
* add test-utils for polkadot and malus
* add test-utils feature to prepare and execute workers script
* remove required features from prepare and executing
* Try to trigger CI again to fix broken jobs
---------
Co-authored-by: Marcin S <marcin@realemail.net>
* PVF: Refactor workers into separate crates, remove host dependency
* Fix compile error
* Remove some leftover code
* Fix compile errors
* Update Cargo.lock
* Remove worker main.rs files
I accidentally copied these from the other PR. This PR isn't intended to
introduce standalone workers yet.
* Address review comments
* cargo fmt
* Update a couple of comments
* Update log targets
* rust 1.64 enables workspace properties
* add edition, repository and authors.
* of course, update the version in one place.
Co-authored-by: Andronik <write@reusable.software>
* westend: update transaction version
* polkadot: update transaction version
* kusama: update transaction version
* Bump spec_version to 9330
* bump versions to 0.9.33
* Bump crate versions
* Bump spec_version to 9280 for kusama
* Bump spec_version to 9280 for polkadot
* Bump spec_version to 9280 for rococo
* Bump spec_version to 9280 for westend
* update Cargo.lock
Co-authored-by: parity-processbot <>
dependabot[bot]