The PVF host is designed to avoid spawning tasks to minimize knowledge
of outer code. Using `async_std::task::spawn` (or Tokio's counterpart)
deemed unacceptable, `SpawnNamed` undesirable. Instead there is only one
task returned that is spawned by the candidate-validation subsystem.
The tasks from the sub-components are polled by that root task.
However, the way the tasks are bundled was incorrect. There was a giant
select that was polling those tasks. Particularly, that implies that as soon as
one of the arms of that select goes into await those sub-tasks stop
getting polled. This is a recipe for a deadlock which indeed happened
here.
Specifically, the deadlock happened during sending messages to the
execute queue by calling
[`send_execute`](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L601).
When the channel to the queue reaches the capacity, the control flow is
suspended until the queue handles those messages. Since this code is
essentially reached from [one of the select
arms](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L371),
the queue won't be given the control and thus no further progress can be
made.
This problem is solved by bundling the tasks one level higher instead,
by `selecting` over those long-running tasks.
We also stop treating returning from those long-running tasks as error
conditions, since that can happen during legit shutdown.
* paras: `include_pvf_check_statement` rt bench
Resolves#4933
This PR adds a benchmark for the `include_pvf_check_statement`
dispatchable. This is a necessary step to make it work without
modifications. That enables us to proceed with testing on Versi.
This introduces 5 new benchmarks. Those measure performance of the
`include_pvf_check_statement` under 2 different conditions:
1. regular vote submission. That's the common case.
2. submission of the last vote. That happens only once and leads to a
heavy finalization stage.
There are 2 different types of finalization (one for onboarding, one for
upgrading) and there are two outcomes: accepted and rejected. Those 4
are similar but I decided to cover them all and assign the maximum of
all 4. This is to avoid a situation when one of those paths becomes more
heavier than others and opens up an attack venue.
The regular vote submission weight is drastically different from the
submission last vote weight. That's why in case during runtime
finalization was not executed the weight consumed value will be lowered
down to the regular vote submission.
The finalization weight is proportional to the number of "causes", i.e.
the events that caused the PVF pre-checking vote in the first place, and
here we assume that the maximum number of causes is 100.
Theoretically, there is nothing that prevents an adversary to
register/upgrade to more than 100 parachains. In that case, the consumed
weight will be lower than the actual time consumed by the finalization
process. That can enable a DoS vector.
However, practically, it is not very possible. Right now it is very
expensive to call `schedule_para_initialize` because it requires a very
large lock up of funds. Moreover, finalizing a vote with 100 causes
leads to around 31ms time spent. Finalizing more will require more time.
However, finalizing with 200 causes will cause ≈62ms delay. This is not
that bad since even though we had a full block and the adversary tried
to finalize 200 causes it won't be able to even exceed the operational
extrinsic boundary of 250ms and even if so it won't make big difference.
That said, this should be addressed later on, esp. when we enable
parathreads, which will make creating causes easier. One of potential
solutions will be shifting the logic of finalization into
`on_initialize`/`on_finalize`. Another is to create a maximum number of
causes and then reject upgrades or onboardings if that was reached.
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs
* cargo run --quiet --profile=production --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs
* Fix import error
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
This commit adds a new primitive called `ParachainsCache` to manipulate
the `Parachains` storage entry in a more convenient way.
Then, on top of that, this commit changes the logic of
`initialize_para_now` so that it is identical to what is used for
initialization of onboarding.
* decl_module and extract runtime mod
* Invert dependency staking←→contract
* Remove CodeOf
* Remove StorageOf and move double_map
* Comment staking test
* Clean
* Add gas_price and gas_limit
* Commit.
* Renames
* Params
* WIP
* Rename transfer to call
* WIP
* Rebuild binaries.
* WIP
* Backport ctro changes
* Call wiring
* Commit overlay.
* Rename merge → commit, into_state → ..._change_set
* WIP
* Contract creation routines
* Set code of the created account.
* Fix the ID of `create` Call
* Fix most of the warning.
* Add the simplest test in the contract crate
* Transfers work!
* Add contract_create test.
* Clean
* Add top-level create test
* Clean a bit.
* Pass gas_limit and data via create.
* Introduce OnAccountKill callback in staking
* Hook up OnAccountKill
* Comments
* Pay for gas.
* Refund unused gas in call
* Tests for zero call and zero endownment.
* Add todo about rewriting docs
* Pay for gas in create transactions
* Fix refunds
* Clean unrelevant comments
* fixup! Fix refunds
* fixup! Clean unrelevant comments
* Move DetermineContractAddress to contract
Also restore account removal test
* fixup! Clean unrelevant comments
* Inline effect_transfer, remove effect_create
Remove account_db!
* Use own new_test_ext.
* Don't account for liability
* Add some docs
* Move contract_fee into contract module
* Take GasMeter in vm::execute
* Use GasMeter throughout contract module for meter
* gas module refactoring
* Clean
* Add base call fee
* note about gas price should be taken from storage
* Add base fee for create
* Rename send → call
* Clean
* Take fee expressed in dots in gas
* Add Checked{Add,Sub,Mul,Div} to SimpleArithmetic
* Make Gas generic
* Store {call,create}_base_fee in storage
* Clean
* Rename buy_gas
* Store gas_price in the storage
* Remove unneeded comment.
* Bail out if contract already has code.
* Todos
* Refund even if top-level contract fails.
* Fix error msg
* Fix caller issue
* Extract tests module
* Add max_depth var in storage
* Remove left over gas_left
* Refactor exec
* Add test oog test.
* set_free_balance_creating
* Docs and comments.
* Update storage roots because of ContractFee move
* Rebuild binaries.
* Simplify vm code.
* Wrapping.
* Refactor a bit.
* Typo
* UpdateBalanceOutcome enum
* Style grumbles.
* Rebuild binaries.
* Always consume the given amount of gas.
* [skip ci] endownment → endowment
* Rename `AccountId` generic in on_account_kill
* Fix Cargo.lock
* Refine docs for gas meter.
* [skip ci] Add comments for gas module
* Directly assign to `return_data` at declaration
* Use slices instead of vecs to pass the input data
* Add todo about passing return data without copy
* Use checked_add instead of add with overflow
* Use return_data directly.
* Rebuild binaries.
* Rebuild binaries.
* WIP
* Iteration over all keys with the specified prefix
* Add clear_prefix in runtime-io
* Introduce a custom storage impl: Double Map
* Remove prefix
* Impl for_keys_with_prefix for light client
* Fix wasm_executor
* Test storage removal leads to removal of stroage
* Check for ok result in storage tests.
* Add docs.
* Remove commented code under decl_storage!
* Add clear_prefix test in runtime-io
* Add test for wasm_executor
* Prefix walking test.
* Rebuild binaries.
* Add primitives for sandboxing.
* Add sandbox module.
* Implement the runtime part of the sandbox.
* Rebuild binaries.
* Implement smart-contract execution.
* Add more documentation.
* Reserve pages for heap at initialization time
Since it is freshly allocated memory it won't collide with stack, data or whatever else.
* Compile all wasm files with LTO.
Modules compiled without LTO contain undesired imports and outright invalid (e.g. type mismatches inside the wasm).
