* pvf-precheck: Integrate PVF pre-checking into paras module
Closes#4009
This is the most of the runtime-side change needed for #3211.
Here is how it works.
The PVF pre-checking can be triggered either by an upgrade or by
onboarding (i.e. calling `schedule_para_initialize`). The PVF
pre-checking process is identified by the PVF code hash that is being
voted on. If there is already PVF pre-checking process running, then no
new PVF pre-checking process will be started. Instead, we just subscribe
to the existing one.
If there is no PVF pre-checking process running but the PVF code hash
was already saved in the storage, that necessarily means (I invite the
reviewers to double-check this invariant) that the PVF already passed
pre-checking. This is equivalent to instant approving of the PVF.
The pre-checking process can be concluded either by obtaining a
supermajority or if it expires.
Each validator checks the list of PVFs available for voting. The vote is
binary, i.e. accept or reject a given PVF. As soon as the supermajority
of votes are collected for one of the sides of the vote, the voting is
concluded in that direction and the effects of the voting are enacted.
Only validators from the active set can participate in the vote. The set
of active validators can change each session. That's why we reset the
votes each session. A voting that observed a certain number of sessions
will be rejected.
The effects of the PVF accepting depend on the operations requested it:
1. All onboardings subscribed to the approved PVF pre-checking process will
get scheduled and after passing 2 session boundaries they will be onboarded.
2. All upgrades subscribed to the approved PVF pre-checking process will
get scheduled very similarly to the existing process. Upgrades with
pre-checking are really the same process that is just delayed by the
time required for pre-checking voting. In case of instant approval the
mechanism is exactly the same. This is important from parachains
compatibility standpoint since following the delayed upgrade requires
the parachain to implement
https://github.com/paritytech/cumulus/pull/517.
In case, PVF pre-checking process was concluded with rejection, then all
the requesting operations get cancelled. For onboarding it means it gets
without movement: the lifecycle of such parachain is terminated on the
`Onboarding` state and after rejection the lifecycle is none. That in
turn means that the caller can attempt registering the parachain once
more. For upgrading it means that the upgrade process is aborted: that
flashes go-ahead signal with `Abort` flag.
Rejection leads to removing the allegedly bad validation code from the
chain storage. Among other things, this implies that the operation can
be re-requested. That allows for retrying an operation in case there was
some bug. At the same time it does not look as a DoS vector due to the
caching performed by the nodes.
PVF pre-checking can be enabled and disabled. Initially, according to
the changes in #4420, this mechanism is disabled. Triggering the PVF
pre-checking when it is disabled just means that we insta approve the
requesting operation. This should lead to the behavior being unchanged.
Follow-ups:
- expose runtime APIs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs
* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs
* Review fixes
Co-authored-by: Parity Bot <admin@parity.io>
* rococo-runtime: Switch to latest `construct_runtime!` syntax
Besides that it fixes pallet macro errors in other crates that popped up
because of this switch.
* FMT
* validator-discovery: remove from peer set before inserting
* bump spec versions
* rework into a companion
* fmt
* fix
* fix
* one more time
* one more try
* one more try
* Revert "one more try"
This reverts commit ab6568d3b828a33dc06f5650037597fc88dd06b1.
* one more try
* one more try
* Revert "one more try"
This reverts commit 8d7369f7b78633bd1b1c5ba3e0f2a0544bdd77a5.
* fix a warning
* fix another warn
* correct log
* fix compilation
* ffs
* less cloning
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* add comments and a small refactoring
* use set_reserved_peers
* cargo update -p sp-io
* rename added to num_peers
* update Substrate
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: parity-processbot <>
* add slot offset for slots
* trying things out
* fix test
* improve api to return the first block of a new lease period
* add an integration test with offset
* de-duplicate test
* hide lease period_period_length from public api
* fix benchmarks
* Update runtime/common/src/slots.rs
* support the exact same range of crowdloans
* fix docs
* fix docs again
* introduce offset to runtimes
* fix and check edge case w/ offset and lease period first block
* remove newline
* turn into an option
* fix benchmarks
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Adding MaxSegmentLength and MaxAuthorities
to pallet babe
* Missed a few to_vec
* Removing `MaxSegmentLength` as not needed anymore
* Adding `MaxAuthorities` to couple of missing place
* Adding missing definition of `MaxAuthorities`
* Adding a missing to_vec
* update Substrate
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: parity-processbot <>
* gotta migrate them all
* migrate rococo construct_runtime
* trigger ci
* fix warnings
* get mocks to work
* add pallet to test runtime
* comments
* calm down mr tabrizi lol
* to v2 macro
* line width
* fix mock
* actually compile moxk
* compile tests
* add hooks
* origin back in mocks
* assimilate_storage
* add generic
* maybe mock compiles now
* origin between parachain and system causing problem
* change origin
* type alias as origin
* keep origin
* add aliases to tests
* remove unnecessary imports
* Parachain -> Paras
* paras test
* nvm
* use genesis build in mock
* Make ParaId value of public paras non-discretionary
* Fixes
* Fixes
* fix tests
* fix benchmark tests
* dont use hardcoded number
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* lock substrate version for testing
* `ModuleId` to `PalletId`
* release substrate version
* Update Cargo.lock
Co-authored-by: Xavier Lau <c.estlavie@icloud.com>
* Add many sanity checks to Crowdloans
* fix tests
* test contribution block in integration test
* fix rococo build
* remove leaser from crowdloans
* fix docs and terms
* fix compile
* TODOs
* Add auctions.rs, comment on changes needed.
* Remove cruft from slots
* Remove more from auctions.rs
* More logic drafting in slots.
* More logic in slots.rs
* patch some errors
* more fixes
* last nit
* Cleanups in slots.rs
* Cleanups in slots.rs
* patches
* make build
* crowdloan to new api
* auction compile
* Use ParaId instead of FundIndex in Crowdloan (#2303)
* use paraid instead of fundindex
* Update crowdloan.rs
* check caller is manager
* Auction tests and fix build warnings.
* Configurable origin for initiating auctions
* Remove on_finalize
* #2303 (manual merge)
* Tests for Slots
* some registrar tests
* Apply suggestions from code review
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update runtime/common/src/slots.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Slots uses Registrar for CurrentChains
* swap works test
* on swap impl
* traitify parachain cleanup
* explicit lifecycle tracking for paras
* initial implementation of lifecycles and upgrades
* clean up a bit
* Update runtime/common/src/slots.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* fix doc comment
* more rigid lifecycle checks
* include paras which are transitioning, and lifecycle query
* format guide
* update api
* update guide
* explicit outgoing state, fix genesis
* handle outgoing with transitioning paras
* Revert "explicit lifecycle tracking for paras"
This reverts commit 4177af7ba473bbd9c26bccd861793f25265b6657.
* remove lifecycle tracking from registrar
* do not include transitioning paras in identifier
* Update paras_registrar.rs
* final patches to registrar
* Fix test
* use noop in test
* clean up pending swap on deregistration
* finish registrar tests
* Update roadmap/implementers-guide/src/runtime/paras.md
* Update roadmap/implementers-guide/src/runtime/paras.md
* Update roadmap/implementers-guide/src/runtime/paras.md
* Apply suggestions from code review
* Use matches macro
* Correct terms
* Apply suggestions from code review
* Remove direct need for Slots and Registrar from Crowdloan
* Rejig things slightly
* actions queue
* Revert "actions queue"
This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.
* Traitify Auction interface.
* Mockups and initial code for Crowdloan testing
* One test...
* collapse onboarding state
* fix some crowdloan tests
* one more
* start benchmarks for auctions
* benchmark bid
* fix more crowdloan tests
* onboard and begin retirement no longer exist
* Revert "onboard and begin retirement no longer exist"
This reverts commit 2e100fd94e3540bff5f172328b5d917896f1c6fc.
* Simplify crowdloan and make it work.
* Fixes
* fix some
* finish merge fixes
* fix refund bug in auctions
* Add traits to Registrar for tests and benchmarks
* fix more auction benchmarks
* Fix TestAuctioneer
* finish crowdloan benchmarks
* start setting up full integration tests
* expand integration tests
* finish basic integration test
* add more integration tests
* begin slots benchmarks
* start paras registrar benchmarks
* fix merge
* fix tests
* clean up paras registrar
* remove println
* remove outdated cleanup config
* update benchmarks
* Add WeightInfo
* enable runtime-benchmarks feature flag
* complete swap benchmark
* add parachains and onboarding into westend
* add benchmarks and genesis
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* fix benchmark execution
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* Use `new_raise_len` in crowdloan on_initialize
* Update paras_registrar.rs
* fix westend merge
* impl on_swap for crowdloan
* Check fund exists before create
* update for crowdloan sig
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* slots on_initialize
* use integration tests environment for benchmarks
* fix hrmp event
* auction on_initialize
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* fix storage name in auctions
* add auction_index to winning data
* winning data takes into account current auction index
* remove println
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* Revert "add auction_index to winning data"
* PastRandomness.
* Fixes
* Use new randomness
* fix use of randomness in auctions and runtime config
* expose consts
* fix auction test
* add deposit per byte for para registration
* basic swap integration test
* make swap test more comprehensive
* Add WinningVec for easier retrieval in the front-end.
* clean up `WinningVec` at the end
* Add event for when a new best bid comes in
* Fix propagation of winners in ending period
* fix benchmarks, refund weight in dissolve
* fix unused
* remove some TODOs
* setup opaque keys for paras in westend
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* remove unused
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* back to regular runtime config
* use saturating math where user input can be
* better first slot check
* Update runtime/common/src/claims.rs
* update westend onswap impl
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Sergei Shulepov