Commit Graph

1871 Commits

Author SHA1 Message Date
Bradley Olson f7bd3a95a0 Drop timers for new requests of active participations (#6974)
* First pass adding logs

* fmt

* Adjustments

* Get rid of extra timers for running participations

* fmt

* Handling timer discards more elegantly
2023-03-30 20:33:25 +00:00
s0me0ne-unkn0wn a946ca2daf Use SIGTERM instead of SIGKILL on PVF worker version mismatch (#6981) 2023-03-30 21:41:54 +02:00
s0me0ne-unkn0wn 55b4aceb99 Check spawned worker version vs node version before PVF preparation (#6861)
* Check spawned worker version vs node version before PVF preparation

* Address discussions

* Propagate errors and shutdown preparation and execution pipelines properly

* Add logs; Fix execution worker checks

* Revert "Propagate errors and shutdown preparation and execution pipelines properly"

This reverts commit b96cc3160ff58db5ff001d8ca0bfea9bd4bdd0f2.

* Don't try to shut down; report the condition and exit worker

* Get rid of `VersionMismatch` preparation error

* Merge master

* Add docs; Fix tests

* Update Cargo.lock

* Kill again, but only the main node process

* Move unsafe code to a common safe function

* Fix libc dependency error on MacOS

* pvf spawning: Add some logging, add a small integration test

* Minor fixes

* Restart CI

---------

Co-authored-by: Marcin S <marcin@realemail.net>
2023-03-29 22:48:41 +00:00
Oliver Tale-Yazdi faa2d5b09e Fix feature (#6966)
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-03-29 12:00:23 +03:00
Tsvetomir Dimitrov 22f404f092 Histogram support in runtime metrics (#6935)
* Histogram support in runtime metrics

Add support for histograms to the runtime metrics. Additionally add
`polkadot_parachain_verify_dispute_signature` histogram which tracks the
time needed from the runtime to verify a single validator signature of a
dispute statement.

* Add noops

* u64 instead of f64

* Update buckets

* Wrap `get_current_time()` in runtime metrics

* Change the dimension of the Histogram from usec to sec

* Fix a compilation error

* Update buckets

* Fix `on_signature_check_complete` calculation

* Update buckets

* Update buckets

* formatting

* Another weights update

* Adjust buckets again

* Final buckets adjustment

* Revert "Fix a compilation error"

This reverts commit 06290b40a39eeb78de2602d8916a39edf7a8b714.

* Update primitives/src/v4/metrics.rs

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

* Use `saturating_sub` for time difference calculation

* Pass nanoseconds to client instead of seconds (using f64 in runtime is dangerous)

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
2023-03-28 13:37:19 +00:00
Andrei Sandu fc2cba1805 Subsystem channel tweaks (#6905)
* subsystem tweaks for higher scale

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Adjust queue sizes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Orchestra 0.0.5

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-03-24 17:19:37 +00:00
Davide Galassi 260d073658 Companion for #13683 (#6944)
* Companion for #13683

* Wraps trait is not required

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-03-24 14:43:21 +00:00
Tsvetomir Dimitrov 95f0ca271e Bump parity-scale-codec to v 3.4.0 (#6945) 2023-03-23 15:15:03 +00:00
Bastian Köcher c7b3352b79 Ensure candidate-backing and provisioner exit on missing Overseer (#6939)
This ensures that both subsystems exit when the Overseer has exited
because of some error.
2023-03-23 10:07:40 +01:00
Gavin Wood 7beeba5fb8 Introduce OpenGov into Polkadot (#6701)
* OpenGov for Polkadot

* Integrate OpenGov into XCM

* Formatting

* Missing files

* Remove Gov1 from Kusama

* Fixes

* Update runtime/polkadot/src/governance/origins.rs

Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

* Revert scheduler origin changes

* Fixes

* Docs

* Remove todo

* Docs

* Move Fellowship to Collectives  (#6718)

* FellowshipAdmin origin over xcm

* accept Fellows origin from Collectives

* remove Fellowship

* remove unreachable arm

* define benchmarks

* correct comment for DOLLARS constant

* Add OpenGov Calls to Proxy Definitions (#6729)

* add opengov calls to proxy definitions

* fix build

* Update runtime/polkadot/src/governance/mod.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update runtime/polkadot/src/governance/tracks.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update runtime/polkadot/src/lib.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update runtime/polkadot/src/governance/origins.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update runtime/polkadot/src/governance/mod.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update old.rs

* Update old.rs

* weights with new api

* XCM tweaks for OpenGov (#6855)

* pass xcm origin, allow unpaid form Collectives and Fellows, whitelist call

* unpaid execution only for Fellows

* Apply suggestions from code review

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* rename const namespace to system_parachains

* remove prod_or_fast

---------

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* remove unused import

---------

Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: parity-processbot <>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
2023-03-21 13:59:54 +00:00
Arkadiy Paronyan 3264cb7b64 Bump parity-db (#6921) 2023-03-21 09:54:57 +01:00
Bradley Olson b26cf3b7d0 Handling timers for repeat dispute participation requests (#6901)
* Added participation and queue sizes metrics

* First draft of all metric code

* Tests pass

* Changed Metrics to field on participation + queues

* fmt

* Improving naming

* Refactor, placing timer in ParticipationRequest

* fmt

* Final cleanup

* Revert "Final cleanup"

This reverts commit 02e5608df64b2e0f7810905e4508673b2037d351.

* Changing metric names

* Implementing Eq only for unit tests

* fmt

* Removing Clone trait from ParticipationRequest

* fmt

* Moved clone functionality to tests helper

* fmt

* Fixing dropped timers on repeat requests

* Keep older best effort timers

* Removing comment redundency and explaining better

* Updating queue() to use single mem read

* fmt
2023-03-21 00:55:35 +00:00
Davide Galassi e0b8b30288 [Companion #13634] keystore overhaul (iter2) (#6913)
* removed 'remote_keystore' parameter

* Keystore can't be None

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-03-20 19:33:54 +00:00
Mira Ressel 058082fcb0 tweak some pattern matches to address a new clippy warning 2023-03-20 16:36:06 +01:00
Vsevolod Stakhov 6d902ce073 Revert "Update orchestra to the recent version (#6854)" (#6916)
This reverts commit 5565bb9dbc.
2023-03-20 15:08:05 +00:00
Gavin Wood 2e656dcd6e Deprecate Currency: Companion for #12951 (#6780)
* Some renames

* Fix

* Fix build for new APIs

* Remove diener

* Fixes

* Fixes

* Fix integration tests

* Fixes

* fix nis issuance

* Update Cargo.toml

* Polkadot doesn't have freezes/holds yet

* No networks use freezes/holds

* update lockfile for {"substrate"}

* Fix tests

There are more failing tests; just starting with the easy ones.
Reserved balance does not count towards ED anymore, therefore reducing
all the reserves by ED (1).

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fixes for Polkadot pallets

* Fix parachains benchmarks

* Update Substrate

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-03-20 13:20:22 +00:00
Davide Galassi 46c36e5a4f [Companion #13615] Keystore overhaul (#6892)
* Remove not required async calls

* Fixed missing renaming

* make_keystore can be sync

* More fixes

* Trivial nitpicks

* Cherry pick test fix from master

* Fixes after master merge

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-03-17 12:09:15 +00:00
Davide Galassi 163d892d8c Fix approval voting test (#6898) 2023-03-17 08:33:50 +01:00
Bradley Olson 8d92debd44 Testing Reversion Speed on Dispute Concluded Against (#6880)
* First pass adding logs

* fmt

* Adjustments
2023-03-16 18:56:38 +00:00
Bradley Olson 200e9dfa29 Issue 4393: Correcting Unnecessary Use of Arc (#6882)
* Added participation and queue sizes metrics

* First draft of all metric code

* Tests pass

* Changed Metrics to field on participation + queues

* fmt

* Improving naming

* Refactor, placing timer in ParticipationRequest

* fmt

* Final cleanup

* Revert "Final cleanup"

This reverts commit 02e5608df64b2e0f7810905e4508673b2037d351.

* Changing metric names

* Implementing Eq only for unit tests

* fmt

* Removing Clone trait from ParticipationRequest

* fmt

* Moved clone functionality to tests helper
2023-03-16 10:00:56 +01:00
s0me0ne-unkn0wn 27c9124411 Release parachain host API v4 (#6885) 2023-03-15 17:06:18 +00:00
Marcin S ba29d06334 PVF: Document that preparation cannot lead to disputes (#6873)
* PVF: Document that preparation cannot lead to disputes

* Add warning for deterministic errors

* Fix warning
2023-03-15 01:35:38 +01:00
Aaro Altonen 0e1c932c42 Companion for paritytech/substrate#13592 (#6869)
* Move configs

* Start using `NetworkService` traits from `sc-network`

* Fix stuff

* Remove `sc-network-common` as dependency

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-03-14 12:52:15 +00:00
Vsevolod Stakhov 5565bb9dbc Update orchestra to the recent version (#6854)
* Test the recent orchestra changes

* Update orchestra and metered-channel
2023-03-14 09:29:57 +00:00
Bradley Olson c99362e56c Adding Dispute Participation Metrics (#6838)
* Added participation and queue sizes metrics

* First draft of all metric code

* Tests pass

* Changed Metrics to field on participation + queues

* fmt

* Improving naming

* Refactor, placing timer in ParticipationRequest

* fmt

* Final cleanup

* Revert "Final cleanup"

This reverts commit 02e5608df64b2e0f7810905e4508673b2037d351.

* Changing metric names

* Implementing Eq only for unit tests

* fmt
2023-03-11 00:38:24 +00:00
s0me0ne-unkn0wn 03d4af104f Move PVF timeouts to executor environment parameters (#6823)
* Move PVF timeouts to executor environment parameters

* Typo

Co-authored-by: Marcin S. <marcin@realemail.net>

* Fix comments

* Change handle_import_statements to FatalResult (#6820)

* Changing dispute db errors to fatal

* fmt

* Change node-key for bootnodes (#6772)

* Additional tracing in `provisioner`, `vote_selection` and `dispute-coordinator` (#6775)

* Additional tracing in `provisioner`, `vote_selection`

* Add `fetched_onchain_disputes` metric to provisioner

* Some tracelines in dispute-coordinator

TODO: cherry pick this in the initial branch!!!

* Remove spammy logs

* Remove some trace lines

* Rename and fix things

* Fix comments

* Typo

* Minor fixes

* Add codec indexes; Remove macro

---------

Co-authored-by: Marcin S. <marcin@realemail.net>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Petr Mensik <petr.mensik1@gmail.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
2023-03-08 22:43:51 +00:00
Tsvetomir Dimitrov e5248e35dd Remove hardcoded paths from dispute-valid-block zombienet test for malus (#6837) 2023-03-07 15:26:48 +00:00
Aaro Altonen ce9fbe73d9 Companion for paritytech/substrate#12828 (#6380)
* Companion for paritytech/substrate#12828

* Pass sync service

* Fix test

* Fix tests again

* update lockfile for {"substrate"}

* cargo fmt

---------

Co-authored-by: parity-processbot <>
2023-03-06 17:46:27 +00:00
Tsvetomir Dimitrov ed6fa5499c Don't send ActiveLeaves from leaves in db on startup in Overseer (#6727)
* Don't send `ActiveLeaves` from leaves in db on startup in Overseer. Wait for fresh leaves instead.

* Don't pass initial set of leaves to Overseer

* Fix compilation error in subsystem-test-helpers
2023-03-06 15:02:16 +00:00
Tsvetomir Dimitrov 70be7a03d3 Additional tracing in provisioner, vote_selection and dispute-coordinator (#6775)
* Additional tracing in `provisioner`, `vote_selection`

* Add `fetched_onchain_disputes` metric to provisioner

* Some tracelines in dispute-coordinator

TODO: cherry pick this in the initial branch!!!

* Remove spammy logs

* Remove some trace lines
2023-03-06 12:18:30 +00:00
Petr Mensik a934fa5b16 Change node-key for bootnodes (#6772) 2023-03-06 10:21:46 +01:00
Bradley Olson 7e7f72ed74 Change handle_import_statements to FatalResult (#6820)
* Changing dispute db errors to fatal

* fmt
2023-03-04 19:35:04 +01:00
Arkadiy Paronyan d98d948bfc Bump parity-db (#6816)
* Bump parity-db

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-03-03 21:12:28 +01:00
Bradley Olson f2bc0f57b6 Removing Outdated References to Misbehavior Arbitration Subsystem (#6814)
* Provisioner guide entry update

* Removing mentions of misbehavior arbitration sub

* fmt
2023-03-03 12:17:17 +01:00
eskimor 4931eb0a97 Get rid of unnecessary cloning and work. (#6808)
Co-authored-by: eskimor <eskimor@no-such-url.com>
2023-03-02 07:53:32 +01:00
Davide Galassi 0b00e6d5b8 [Companion] BEEFY crates renaming (#6799)
* Temporary commit to make the Substrate CI happy

* Revert "Temporary commit to make the Substrate CI happy"

This reverts commit 9eb2fd223c3e36312242d4fda4ebacf3dd732547.

* Align to substrate master

* Update lock

* Adjust some naming according to the new substrate crates
2023-02-28 17:09:43 +01:00
Davide Galassi 03d8256b6e [Companion] GRANDPA crate renaming (#6776)
* Companion for grandpa crate renaming
* lock update
2023-02-27 17:55:16 +01:00
Bradley Olson 92ee0b7d9f Explicitly Handling ProvisionableData Cases (#6757)
* Initial pass

* fmt

* Lock changes

* Implementers guide edit

* Reasoning changes

* typo

Co-authored-by: Marcin S. <marcin@bytedude.com>

* Revert "Lock changes"

This reverts commit 3131a5245c6349743789270e6d1ad7842dc4c9df.

* Getting rid of cargo.lock changes

* Argumentation update

* fmt

---------

Co-authored-by: Marcin S. <marcin@bytedude.com>
2023-02-24 21:54:14 +01:00
Bastian Köcher 1e291f30f4 Companion for Substrate#12520 (#6730)
* Companion for Substrate#12520

https://github.com/paritytech/substrate/pull/12520

* Fix

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-02-24 12:47:41 +00:00
Paulo Martins 1ae3c81126 Add Turboflakes bootnodes to Polkadot, Kusama and Westend (#6628) 2023-02-22 13:57:15 +01:00
Koute 1b30ce5a9d Companion for substrate#13284 (#6653)
* Align iterator use to new substrate

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-02-22 08:36:16 +00:00
Michal Kucharczyk 0e59d866f3 BlockId removal: BlockBuilderProvider::new_block_at (#6734)
* `BlockId` removal: `BlockBuilderProvider::new_block_at`

It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`

* useless import removed

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-02-21 19:26:46 +00:00
Michal Kucharczyk d091a84a07 BlockId removal: refactor of runtime API (#6721)
* BlockId removal: refactor of runtime API

It changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`

* fmt

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-02-20 23:44:20 +00:00
Oliver Tale-Yazdi 35a0d69af8 Fix auction bench (#6747)
* Adjust command

* Move imports

* Chain-specific blocktime

* update lockfile for {"substrate"}

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix bench

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Export constants from node service

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use constants in CLI

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Piotr Mikołajczyk <piomiko41@gmail.com>
Co-authored-by: parity-processbot <>
2023-02-20 17:02:07 +00:00
s0me0ne-unkn0wn b13ba77a3b Use PVF code paired with executor params wherever possible (#6742) 2023-02-20 13:41:46 +01:00
s0me0ne-unkn0wn bad4afca36 Retire OldV1SessionInfo (#6744) 2023-02-20 10:30:45 +00:00
Bastian Köcher 5ca430b649 Do not call longest chain by default (#6704)
* Do not call longest chain by default

We always called longest chain by default just for some potential
logging. This was probably some oversight for when this select chain
implementation was introduced.

* Fix tests
2023-02-17 08:55:14 +01:00
Tsvetomir Dimitrov 2853f349bf Fix a metric description (#6732) 2023-02-16 18:59:46 +02:00
Bastian Köcher 83eb8453eb Change MaxMemorySize to MaxMemoryPages (#6728)
* Change `MaxMemorySize` to `MaxMemoryPages`

We should set the max memory for the executor in pages (64KiB) and not in bytes.
The wasm memory is always a multiple of a page and we should use the
same terminology.

* FMT

* Fix warning
2023-02-16 17:10:24 +01:00
Marcin S d5a7991429 Use a BoundedVec in ValidationResult (#6603)
* Use a `BoundedVec` in `ValidationResult`

> Use a `BoundedVec` for `upward_messages` and `horizontal_messages` in order to
> limit the number of individual messages/memory allocations right at decoding
> time. The reason for this is that the `ValidationResult` may contain a code
> upgrade (including a full PVF binary), so the total size limit can't be set
> too low and this limit will still allow several millions of upward messages,
> which will (due to the memory allocator overhead) already have a
> non-negligible memory footprint in decoded form.

* List all fields when hashing so we don't miss one

* Define types for  `BoundedVec`s of messages

* Fix test compile errors

* Depend on `bounded-collections` 0.1.4 (fixes allocation issue)

* Fix compilation issue

* Derive `Hash` instead of manual `impl`

* Avoid use of unwrap
2023-02-16 15:08:56 +00:00