Commit Graph

41 Commits

Author SHA1 Message Date
s0me0ne-unkn0wn 0211c4b2f7 Enable changing executor params through governance (#6934)
* Add a pallet call to change executor params

* Use `OptionQuery`; Avoid runtime panic

* Move pending executor params to `configuration`

* Move `ExecutorParams` to `HostConfiguration` structure

* Add executor params to the v5 migration

* Add an `ExecutorParams` benchmark

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::configuration

* Add to `WeightInfo`

* Add dummy weights to other networks

* ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::configuration

* ".git/.scripts/commands/bench/bench.sh" runtime rococo runtime_parachains::configuration

* ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::configuration

* Use real weight

* Fix comment

---------

Co-authored-by: command-bot <>
2023-04-14 11:21:12 +00:00
s0me0ne-unkn0wn 64660ee8d2 Remove years from copyright notes (#7034)
* Happy New Year!

* Remove year entierly

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove years from copyright notice in the entire repo

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-04-08 20:38:35 +00:00
s0me0ne-unkn0wn 27c9124411 Release parachain host API v4 (#6885) 2023-03-15 17:06:18 +00:00
Vivek Pandya 87db25ce01 Remove use of Store trait (#6835)
* Remove use of Store trait from runtime directory

* Remove Store trait usage from xcm directory

* Run cargo fmt

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-03-13 16:11:22 +00:00
s0me0ne-unkn0wn dd0a556665 Executor Environment parameterization (#6161)
* Re-apply changes without Diener, rebase to the lastest master

* Cache pruning

* Bit-pack InstantiationStrategy

* Move ExecutorParams version inside the structure itself

* Rework runtime API and executor parameters storage

* Pass executor parameters through backing subsystem

* Update Cargo.lock

* Introduce `ExecutorParams` to approval voting subsys

* Introduce `ExecutorParams` to dispute coordinator

* `cargo fmt`

* Simplify requests from backing subsys

* Fix tests

* Replace manual config cloning with `.clone()`

* Move constants to module

* Parametrize executor performing PVF pre-check

* Fix Malus

* Fix test runtime

* Introduce session executor params as a constant defined by session info
pallet

* Use Parity SCALE codec instead of hand-crafted binary encoding

* Get rid of constants; Add docs

* Get rid of constants

* Minor typo

* Fix Malus after rebase

* `cargo fmt`

* Use transparent SCALE encoding instead of explicit

* Clean up

* Get rid of relay parent to session index mapping

* Join environment type and version in a single enum element

* Use default execution parameters if running an old runtime

* `unwrap()` -> `expect()`

* Correct API version

* Constants are back in town

* Use constants for execution environment types

* Artifact separation, first try

* Get rid of explicit version

* PVF execution queue worker separation

* Worker handshake

* Global renaming

* Minor fixes resolving discussions

* Two-stage requesting of executor params to make use of runtime API cache

* Proper error handling in pvf-checker

* Executor params storage bootstrapping

* Propagate migration to v3 network runtimes

* Fix storage versioning

* Ensure `ExecutorParams` serialization determinism; Add comments

* Rename constants to make things a bit more deterministic
Get rid of stale code

* Tidy up a structure of active PVFs

* Minor formatting

* Fix comment

* Add try-runtime hooks

* Add storage version write on upgrade

Co-authored-by: Andronik <write@reusable.software>

* Add pre- and post-upgrade assertions

* Require to specify environment type; Remove redundant `impl`s

* Add `ExecutorParamHash` creation from `H256`

* Fix candidate validation subsys tests

* Return splittable error from executor params request fn

* Revert "Return splittable error from executor params request fn"

This reverts commit a0b274177d8bb2f6e13c066741892ecd2e72a456.

* Decompose approval voting metrics

* Use more relevant errors

* Minor formatting fix

* Assert a valid environment type instead of checking

* Fix `try-runtime` hooks

* After-merge fixes

* Add migration logs

* Remove dead code

* Fix tests

* Fix tests

* Back to the strongly typed implementation

* Promote strong types to executor interface

* Remove stale comment

* Move executor params to `SessionInfo`: primitives and runtime

* Move executor params to `SessionInfo`: node

* Try to bump primitives and API version

* Get rid of `MallocSizeOf`

* Bump target API version to v4

* Make use of session index already in place

* Back to v3

* Fix all the tests

* Add migrations to all the runtimes

* Make use of existing `SessionInfo` in approval voting subsys

* Rename `TARGET` -> `LOG_TARGET`

* Bump all the primitives to v3

* Fix Rococo ParachainHost API version

* Use `RollingSessionWindow` to acquire `ExecutorParams` in disputes

* Fix nits from discussions; add comments

* Re-evaluate queue logic

* Rework job assignment in execution queue

* Add documentation

* Use `RuntimeInfo` to obtain `SessionInfo` (with blackjack and caching)

* Couple `Pvf` with `ExecutorParams` wherever possible

* Put members of `PvfWithExecutorParams` under `Arc` for cheap cloning

* Fix comment

* Fix CI tests

* Fix clippy warnings

* Address nits from discussions

* Add a placeholder for raw data

* Fix non exhaustive match

* Remove redundant reexports and fix imports

* Keep only necessary semantic features, as discussed

* Rework `RuntimeInfo` to support mock implementation for tests

* Remove unneeded bound

* `cargo fmt`

* Revert "Remove unneeded bound"

This reverts commit 932463f26b00ce290e1e61848eb9328632ef8a61.

* Fix PVF host tests

* Fix PVF checker tests

* Fix overseer declarations

* Simplify tests

* `MAX_KEEP_WAITING` timeout based on `BACKGING_EXECUTION_TIMEOUT`

* Add a unit test for varying executor parameters

* Minor fixes from discussions

* Add prechecking max. memory parameter (see paritytech/srlabs_findings#110)

* Fix and improve a test

* Remove `ExecutionEnvironment` and `RawData`

* New primitives versioning in parachain host API

* `disputes()` implementation for Kusama and Polkadot

* Move `ExecutorParams` from `vstaging` to stable primitives

* Move disputes from `vstaging` to stable implementation

* Fix `try-runtime`

* Fixes after merge

* Move `ExecutorParams` to the bottom of `SessionInfo`

* Revert "Move executor params to `SessionInfo`: primitives and runtime"

This reverts commit dfcfb85fefd1c5be6c8a8f72dc09fd1809cfa9ce.

* Always use fresh activated live hash in pvf precheck
(re-apply 34b09a4c20de17e7926ed942cd0d657d18f743fa)

* Fixing tests (broken commit)

* Fix candidate validation tests

* Fix PVF host test

* Minor fixes

* Address discussions

* Restore migration

* Fix `use` to only include what is needed instead of `*`

* Add comment to never touch `DEFAULT_CONFIG`

* Update migration to set default `ExecutorParams` for `dispute_period`
sessions back

* Use `earliest_stored_session` instead of calculations

* Nit

* Add logs

* Treat any runtime error as `NotSupported` again

* Always return default executor params if not available

* Revert "Always return default executor params if not available"

This reverts commit b58ac4482ef444c67a9852d5776550d08e312f30.

* Add paritytech/substrate#9997 workaround

* `cargo fmt`

* Remove migration (again!)

* Bump executor params to API v4 (backport from #6698)

---------

Co-authored-by: Andronik <write@reusable.software>
2023-02-15 11:26:09 +00:00
s0me0ne-unkn0wn 1cb1d03c08 Re-export current primitives in crate root (#6487)
* Re-export current primitives in crate root

* Add missing exports

* restart CI
2023-01-11 11:28:12 +00:00
Boluwatife Bakre 8eb1f4617f Use a more typesafe approach for managing indexed data (#6150)
* Fix for issue #2403

* Nightly fmt

* Quick documentation fixes

* Default Implementation

* iter() function integrated

* Implemented iter functionalities

* Fmt

* small change

* updates node-network

* updates in dispute-coordinator

* Updates

* benchmarking fix

* minor fix

* test fixes in runtime api

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* Removal of [index], shorting of FromIterator, Renaming of GroupValidators to ValidatorGroups

* Removal of ops import

* documentation fixes for spell check

* implementation of generic type

* Refactoring

* Test and documentation fixes

* minor test fix

* minor test fix

* minor test fix

* Update node/network/statement-distribution/src/lib.rs

Co-authored-by: Andronik <write@reusable.software>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* removed IterMut

* Update node/core/dispute-coordinator/src/import.rs

Co-authored-by: Andronik <write@reusable.software>

* Update node/core/dispute-coordinator/src/initialized.rs

Co-authored-by: Andronik <write@reusable.software>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* fmt

* IterMut

* documentation update

Co-authored-by: Andronik <write@reusable.software>

* minor adjustments and new TypeIndex trait

* spelling fix

* TypeIndex fix

Co-authored-by: Andronik <write@reusable.software>
2022-10-22 08:39:11 +00:00
Shawn Tabrizi 28e94d97dd Companion for Weight v1.5 (#5943)
* fix to latest substrate pr

* update weights

* cargo build -p polkadot-runtime-parachains

* fix xcm-builder

* fix import

* fix a bunch

* fix a bunch of weight stuff

* kusama compile

* unused

* builds

* maybe fix

* cargo test -p polkadot-runtime-parachains

* xcm simulator example

* fix tests

* xcm sim fuzz

* fix runtime tests

* remove unused

* fix integration tests

* scalar div

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-08-31 11:59:39 +00:00
Andronik 15e7536a4e runtime/session_info: keep track of stash keys (#5473) 2022-05-12 09:36:57 +00:00
Bernhard Schuster b67b7fa355 Avoid an excessive amount of unrelated errors on prior rustc error
`1.into()` -> `ParaId::from(1_u32)` (#5500)
2022-05-11 16:11:48 +02:00
Andronik ff18cabbc5 remove old pallet migrations (#5194)
* remove old pallet migrations

* remove more stuff
2022-04-13 10:01:57 +00:00
Robert Habermeier 49f7e5cce4 Finish migration to v2 primitives (#5037)
* remove v0 primitives from polkadot-primitives

* first pass: remove v0

* fix fallout in erasure-coding

* remove v1 primitives, consolidate to v2

* the great import update

* update runtime_api_impl_v1 to v2 as well

* guide: add `Version` request for runtime API

* add version query to runtime API

* reintroduce OldV1SessionInfo in a limited way
2022-03-09 14:01:13 -06:00
Bernhard Schuster 5b06cc23d6 [chore] runtime split tests (#4834) 2022-02-03 09:12:41 +01:00
Oliver Tale-Yazdi 3d61cc01db co #10662: Require MaxEncodedLen per default (#4746)
* Add `without_storage_info`

The MaxEncodedLen trait is now enforced by default in Substrate.
All pallets missing an implementation need to be marked with
`without_storage_info` now.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove `generate_storage_info`

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more `without_storage_info`

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-01-20 12:36:22 +00:00
Andronik Ordian 9efb223a97 session_info: add dispute_period and random_seed (#4547) 2021-12-27 09:03:50 +00:00
Andronik Ordian b342ae11d3 session-info: add new fields + migration (#4545)
* session_info: v2 + migration

* use primitives::v2

* use polkadot_primitives::v2

* impl primitives::v2

* fix approval-voting tests

* fix other tests

* hook storage migration up

* backwards compat (1)

* backwards compat (2)

* fmt

* fix tests

* FMT

* do not reexport v1 in v2

* fmt

* set storage version to 1

Co-authored-by: Javier Viola <javier@parity.io>
2021-12-27 08:01:30 +00:00
Bernhard Schuster 4adb8466a3 dev-comment spelling mistakes (#4434) 2021-12-06 15:20:29 +01:00
André Silva 9bdded5fe4 Companion for substrate#9448 (#3527)
* Companion for substrate#9448

* fix on_disabled

* use temporary beefy branch

* revert beefy updates

* update Substrate

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: parity-processbot <>
2021-10-06 17:06:53 +00:00
Robert Klotzner 135d9ae931 Put all authorities of a session into SessionInfo. (#3813)
* Make SessionInfo include all authorities

of the current session.

* Add missing import.

* Only take subset for assignment keys.

* Fix typo.

* Make spellcheck happy.

* Really.

* Fix test.

* More clear documentation.

* Update comments in `SessionInfo`.
2021-09-14 13:58:37 +02:00
Georges 0b5117f818 Companion for Generate storage info for pallet authority_discovery #9428 (#3517)
* Fixing Polkadot after substrate changes

moving Vec to BoundedVec

* Removing changes to pallet aura
bumping bitflags version

* update Substrate

Co-authored-by: parity-processbot <>
2021-08-30 06:49:10 +00:00
Shawn Tabrizi ff5d56fb76 cargo +nightly fmt (#3540)
* cargo +nightly fmt

* add cargo-fmt check to ci

* update ci

* fmt

* fmt

* skip macro

* ignore bridges
2021-08-02 10:47:33 +00:00
ferrell-code c615761379 Parachains scheduler.rs to FrameV2 (#3529)
* migration

* migrate runtimes

* storage visability

* Remove unused Call part

* Remove Call part import from parachain mock runtime

* Remove Call part import from test-runtime

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2021-07-31 15:54:01 +00:00
ferrell-code c251e1e0a0 Parachains session_info.rs to FrameV2 (#3521)
* migration

* storage visability

* migrate runtimes

* runtimes

* Update runtime/parachains/src/session_info.rs

* Remove Call part import from all runtimes for ParaSessionInfo

* Remove Call part import from runtime/test-runtime/src/lib.rs

* Remove Call part import from parachain mocks

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2021-07-31 12:17:42 +00:00
ferrell-code 84ca3b2b3b Parachains configuration.rs FrameV2 (#3516)
* migration

* genesis build correctly

* migrate runtimes

* add genesis builds

* expose GenesisBuild

* chainspec migration

* fmt

* check before insert

* Reduce visibility of storage items to crate

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2021-07-28 04:56:35 +00:00
ferrell-code 7788de8164 Parachains shared.rs to Frame V2 (#3425)
* gotta migrate them all

* migrate rococo construct_runtime

* trigger ci

* fix warnings

* get mocks to work

* add pallet to test runtime

* comments

* calm down mr tabrizi lol
2021-07-22 21:59:31 +00:00
Robert Habermeier d4e8f2a3da set session-info availability cores based on scheduler (#3134)
this will account for 'dummy' cores due to maxValidatorsPerCore
2021-05-28 18:06:55 -05:00
Kian Paimani c384f70067 Companion for substrate/pull/8596 (#2895)
* Add weight types

* Add missing types

* update Substrate

* Fix build

* Fix bench build

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_membership --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Co-authored-by: parity-processbot <>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
2021-04-18 09:39:34 +01:00
Robert Habermeier 0fac590b84 avoid reshuffling active validators (#2605) 2021-03-10 17:19:53 -06:00
Robert Habermeier 8b4f46d2df Configurable maximum validators (#2586)
* guide: max_validators

* guide: new approach

* restrict validators based on configurable max

* add some tests

* fix wasm build (Vec)

* clean up warnings

* add logging

* set validator indices in tests as well

* fix common tests

* Update runtime/parachains/src/util.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-03-09 18:38:55 +00:00
Shawn Tabrizi a5defa7c7f Session Delayed Para Changes / Actions Queue (#2406)
* initial implementation of lifecycles and upgrades

* clean up a bit

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* do not include transitioning paras in identifier

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* collapse onboarding state

* starting actions queue

* consolidate actions queue

* schedule para initialize result

* more actions queue for upgrade/downgrade

* clean up with fully implemented actions queue

* fix tests

* fix scheduler tests

* fix hrmp tests

* fix test

* doc fixes

* fix hrmp test w/ valid para

* Update paras.md

* fix paras registrar

* Update propose_parachain.rs

* fix merge

* Introduce "shared" module

* fix rococo build

* fix up and use shared

* guide updates

* add shared config to common tests

* add shared to test-runtime

* remove println

* fix note

Co-authored-by: Gavin Wood <gavin@parity.io>
2021-02-18 21:20:18 -06:00
Andrew Jones 7343c974a2 Use construct_runtime in tests, remove default PalletInfo impl (#2409)
* Claims

* Crowdloan

* Runtime common

* Parachains registrar

* Impls

* Purchase

* Slots

* parachains runtime mock

* Use MockGenesisConfig in tests

* Update runtime/common/src/claims.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-02-09 16:36:15 +00:00
Liu-Cheng Xu d353c76e26 Conpanion for Substrate#7127 (#1865)
* Conpanion for Substrate#7127

* Use sp_session::OneSessionHandler

* .

* Fix pallet_session::OneSessionHandler

* OneSessionHandler is in frame_support now

* "Update Substrate"

Co-authored-by: parity-processbot <>
2021-02-02 17:05:42 +00:00
Andronik Ordian deff43fd30 small cleanup (#2267)
* session_info: fix authorities docstring

* overseer: more consistent metrics naming

* session_info: mention ordering

* use correct bucket sizes

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
2021-01-14 10:41:19 +00:00
Andronik Ordian e1773be63d Companion for substrate#7892 (#2262)
* session_info: use correct authorities set

* bump rococo spec_version to 19

* "Update Substrate"

Co-authored-by: parity-processbot <>
2021-01-13 19:02:03 +00:00
Sergei Shulepov d99dbc496c Fix run_to_block functions (#2258) 2021-01-12 16:45:11 -05:00
Robert Habermeier 15c253117d Add assignment keys to session keys, no separate approvals key (#2092)
* guide: merge backing and approval keys

* bump substrate master & update primitives

* use new SessionInfo struct in session_info

* session keys upgrade for Polkadot

* kusama & westend runtimes

* bump westend, kusama, and polkadot versions

* add session key to rococo & test-runtime

* update prepare-test-net to latest subkey

* update chain specs to support new para_assignment session key

* get cargo.lock from master

* formatting

* update kill_storage based on substrate master

* fix test-service

* assgn -> asgn

* use session info module for assignment session key
2020-12-10 20:30:27 -06:00
Andronik Ordian 04e9489da6 session_ info: small fixes (#2106)
* session_info: use proper timeout in test

* Revert "fix off-by-one error"

This reverts commit 35cb56305a19134acd8a8f881f3aabf999a09d74.

* session_info: use correct EarliestStoredSession when introduced on a live chain

* use saturating_sub

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* session_info: revert the timeout test

* session_info: rust is dumb

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-12-10 15:34:17 -06:00
Andronik Ordian be3e5b3b3b session_info: store the assigment ids (#2102)
* session_info: store the assigment ids

* how about this?
2020-12-10 15:02:36 -06:00
Andronik Ordian 6f1590b253 runtime/session_info: avoid heavy loop when introduced on a live chain (#2099)
* session_info: a heavy loop test

* session_info: fix a typo

* session_info: fix heavy loop

* session_info: crank the iterations all the way up
2020-12-09 09:33:58 -06:00
Guillaume Thiolliere 2d4aa3a42e Companion: Rename pallet trait Trait to Config (#2014)
* rename Trait -> Config

* revert diener changes

* rename HostConfig to ActiveConfig as more meaningful

* fix merge

* "Update Substrate"

* cargo update -p sp-io

Co-authored-by: parity-processbot <>
2020-11-30 15:13:43 +00:00
Andronik Ordian 39a12b68f6 past-session validator discovery APIs (#2009)
* guide: fix formatting for SessionInfo module

* primitives: SessionInfo type

* punt on approval keys

* ah, revert the type alias

* session info runtime module skeleton

* update the guide

* runtime/configuration: sync with the guide

* runtime/configuration: setters for newly added fields

* runtime/configuration: set codec indexes

* runtime/configuration: update test

* primitives: fix SessionInfo definition

* runtime/session_info: initial impl

* runtime/session_info: use initializer for session handling (wip)

* runtime/session_info: mock authority discovery trait

* guide: update the initializer's order

* runtime/session_info: tests skeleton

* runtime/session_info: store n_delay_tranches in Configuration

* runtime/session_info: punt on approval keys

* runtime/session_info: add some basic tests

* Update primitives/src/v1.rs

* small fixes

* remove codec index annotation on structs

* fix off-by-one error

* validator_discovery: accept a session index

* runtime: replace validator_discovery api with session_info

* Update runtime/parachains/src/session_info.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* runtime/session_info: add a comment about missing entries

* runtime/session_info: define the keys

* util: expose connect_to_past_session_validators

* util: allow session_info requests for jobs

* runtime-api: add mock test for session_info

* collator-protocol: add session_index to test state

* util: fix error message for runtime error

* fix compilation

* fix tests after merge with master

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2020-11-26 11:02:50 +00:00