paras: include_pvf_check_statement rt bench (#4938)

* paras: `include_pvf_check_statement` rt bench

Resolves #4933

This PR adds a benchmark for the `include_pvf_check_statement`
dispatchable. This is a necessary step to make it work without
modifications. That enables us to proceed with testing on Versi.

This introduces 5 new benchmarks. Those measure performance of the
`include_pvf_check_statement` under 2 different conditions:

1. regular vote submission. That's the common case.
2. submission of the last vote. That happens only once and leads to a
   heavy finalization stage.

There are 2 different types of finalization (one for onboarding, one for
upgrading) and there are two outcomes: accepted and rejected. Those 4
are similar but I decided to cover them all and assign the maximum of
all 4. This is to avoid a situation when one of those paths becomes more
heavier than others and opens up an attack venue.

The regular vote submission weight is drastically different from the
submission last vote weight. That's why in case during runtime
finalization was not executed the weight consumed value will be lowered
down to the regular vote submission.

The finalization weight is proportional to the number of "causes", i.e.
the events that caused the PVF pre-checking vote in the first place, and
here we assume that the maximum number of causes is 100.

Theoretically, there is nothing that prevents an adversary to
register/upgrade to more than 100 parachains. In that case, the consumed
weight will be lower than the actual time consumed by the finalization
process. That can enable a DoS vector.

However, practically, it is not very possible. Right now it is very
expensive to call `schedule_para_initialize` because it requires a very
large lock up of funds. Moreover, finalizing a vote with 100 causes
leads to around 31ms time spent. Finalizing more will require more time.
However, finalizing with 200 causes will cause ≈62ms delay. This is not
that bad since even though we had a full block and the adversary tried
to finalize 200 causes it won't be able to even exceed the operational
extrinsic boundary of 250ms and even if so it won't make big difference.

That said, this should be addressed later on, esp. when we enable
parathreads, which will make creating causes easier. One of potential
solutions will be shifting the logic of finalization into
`on_initialize`/`on_finalize`. Another is to create a maximum number of
causes and then reject upgrades or onboardings if that was reached.

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Fix import error

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>

polkadot/runtime/parachains/src/paras/benchmarking.rs

@@ -15,12 +15,16 @@
 // along with Polkadot.  If not, see <http://www.gnu.org/licenses/>.
 
 use super::*;
-use crate::{configuration::HostConfiguration, shared};
+use crate::configuration::HostConfiguration;
 use frame_benchmarking::benchmarks;
 use frame_system::RawOrigin;
 use primitives::v2::{HeadData, Id as ParaId, ValidationCode, MAX_CODE_SIZE, MAX_HEAD_DATA_SIZE};
 use sp_runtime::traits::{One, Saturating};
 
+mod pvf_check;
+
+use self::pvf_check::{VoteCause, VoteOutcome};
+
 // 2 ^ 10, because binary search time complexity is O(log(2, n)) and n = 1024 gives us a big and
 // round number.
 // Due to the limited number of parachains, the number of pruning, upcoming upgrades and cooldowns
@@ -139,6 +143,48 @@ benchmarks! {
 		let code_hash = [0; 32].into();
 	}: _(RawOrigin::Root, code_hash)
 
+	include_pvf_check_statement {
+		let (stmt, signature) = pvf_check::prepare_inclusion_bench::<T>();
+	}: {
+		let _ = Pallet::<T>::include_pvf_check_statement(RawOrigin::None.into(), stmt, signature);
+	}
+
+	include_pvf_check_statement_finalize_upgrade_accept {
+		let (stmt, signature) = pvf_check::prepare_finalization_bench::<T>(
+			VoteCause::Upgrade,
+			VoteOutcome::Accept,
+		);
+	}: {
+		let _ = Pallet::<T>::include_pvf_check_statement(RawOrigin::None.into(), stmt, signature);
+	}
+
+	include_pvf_check_statement_finalize_upgrade_reject {
+		let (stmt, signature) = pvf_check::prepare_finalization_bench::<T>(
+			VoteCause::Upgrade,
+			VoteOutcome::Reject,
+		);
+	}: {
+		let _ = Pallet::<T>::include_pvf_check_statement(RawOrigin::None.into(), stmt, signature);
+	}
+
+	include_pvf_check_statement_finalize_onboarding_accept {
+		let (stmt, signature) = pvf_check::prepare_finalization_bench::<T>(
+			VoteCause::Onboarding,
+			VoteOutcome::Accept,
+		);
+	}: {
+		let _ = Pallet::<T>::include_pvf_check_statement(RawOrigin::None.into(), stmt, signature);
+	}
+
+	include_pvf_check_statement_finalize_onboarding_reject {
+		let (stmt, signature) = pvf_check::prepare_finalization_bench::<T>(
+			VoteCause::Onboarding,
+			VoteOutcome::Reject,
+		);
+	}: {
+		let _ = Pallet::<T>::include_pvf_check_statement(RawOrigin::None.into(), stmt, signature);
+	}
+
 	impl_benchmark_test_suite!(
 		Pallet,
 		crate::mock::new_test_ext(Default::default()),

polkadot/runtime/parachains/src/paras/benchmarking/pvf_check.rs

@@ -0,0 +1,195 @@
+// Copyright 2022 Parity Technologies (UK) Ltd.
+// This file is part of Polkadot.
+
+// Polkadot is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+
+// Polkadot is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with Polkadot.  If not, see <http://www.gnu.org/licenses/>.
+
+//! This module focuses on the benchmarking of the `include_pvf_check_statement` dispatchable.
+
+use crate::{configuration, paras::*, shared::Pallet as ParasShared};
+use frame_system::RawOrigin;
+use primitives::v2::{HeadData, Id as ParaId, ValidationCode, ValidatorId, ValidatorIndex};
+use sp_application_crypto::RuntimeAppPublic;
+
+// Constants for the benchmarking
+const SESSION_INDEX: SessionIndex = 1;
+const VALIDATOR_NUM: usize = 800;
+const CAUSES_NUM: usize = 100;
+fn validation_code() -> ValidationCode {
+	ValidationCode(vec![0])
+}
+fn old_validation_code() -> ValidationCode {
+	ValidationCode(vec![1])
+}
+
+/// Prepares the PVF check statement and the validator signature to pass into
+/// `include_pvf_check_statement` during benchmarking phase.
+///
+/// It won't trigger finalization, so we expect the benchmarking will only measure the performance
+/// of only vote accounting.
+pub fn prepare_inclusion_bench<T>() -> (PvfCheckStatement, ValidatorSignature)
+where
+	T: Config + shared::Config,
+{
+	initialize::<T>();
+	// we do not plan to trigger finalization, thus the cause is inconsequential.
+	initialize_pvf_active_vote::<T>(VoteCause::Onboarding);
+
+	// `unwrap` cannot panic here since the `initialize` function should initialize validators count
+	// to be more than 0.
+	//
+	// VoteDirection doesn't matter here as well.
+	let stmt_n_sig = generate_statements::<T>(VoteOutcome::Accept).next().unwrap();
+
+	stmt_n_sig
+}
+
+/// Prepares conditions for benchmarking of the finalization part of `include_pvf_check_statement`.
+///
+/// This function will initialize a PVF pre-check vote, then submit a number of PVF pre-checking
+/// statements so that to achieve the quorum only one statement is left. This statement is returned
+/// from this function and is expected to be passed into `include_pvf_check_statement` during the
+/// benchmarking phase.
+pub fn prepare_finalization_bench<T>(
+	cause: VoteCause,
+	outcome: VoteOutcome,
+) -> (PvfCheckStatement, ValidatorSignature)
+where
+	T: Config + shared::Config,
+{
+	initialize::<T>();
+	initialize_pvf_active_vote::<T>(cause);
+
+	let mut stmts = generate_statements::<T>(outcome).collect::<Vec<_>>();
+	// this should be ensured by the `initialize` function.
+	assert!(stmts.len() > 2);
+
+	// stash the last statement to be used in the benchmarking phase.
+	let stmt_n_sig = stmts.pop().unwrap();
+
+	for (stmt, sig) in stmts {
+		let r = Pallet::<T>::include_pvf_check_statement(RawOrigin::None.into(), stmt, sig);
+		assert!(r.is_ok());
+	}
+
+	stmt_n_sig
+}
+
+/// What caused the PVF pre-checking vote?
+#[derive(PartialEq, Eq, Copy, Clone, Debug)]
+pub enum VoteCause {
+	Onboarding,
+	Upgrade,
+}
+
+/// The outcome of the PVF pre-checking vote.
+#[derive(PartialEq, Eq, Copy, Clone, Debug)]
+pub enum VoteOutcome {
+	Accept,
+	Reject,
+}
+
+fn initialize<T>()
+where
+	T: Config + shared::Config,
+{
+	// 0. generate a list of validators
+	let validators = (0..VALIDATOR_NUM)
+		.map(|_| <ValidatorId as RuntimeAppPublic>::generate_pair(None))
+		.collect::<Vec<_>>();
+
+	// 1. Make sure PVF pre-checking is enabled in the config.
+	let mut config = configuration::Pallet::<T>::config();
+	config.pvf_checking_enabled = true;
+	configuration::Pallet::<T>::force_set_active_config(config.clone());
+
+	// 2. initialize a new session with deterministic validator set.
+	ParasShared::<T>::set_active_validators_ascending(validators.clone());
+	ParasShared::<T>::set_session_index(SESSION_INDEX);
+}
+
+/// Creates a new PVF pre-checking active vote.
+///
+/// The subject of the vote (i.e. validation code) and the cause (upgrade/onboarding) is specified
+/// by the test setup.
+fn initialize_pvf_active_vote<T>(vote_cause: VoteCause)
+where
+	T: Config + shared::Config,
+{
+	for i in 0..CAUSES_NUM {
+		let id = ParaId::from(i as u32);
+
+		if vote_cause == VoteCause::Upgrade {
+			// we do care about validation code being actually different, since there is a check
+			// that prevents upgrading to the same code.
+			let old_validation_code = old_validation_code();
+			let validation_code = validation_code();
+
+			let mut parachains = ParachainsCache::new();
+			Pallet::<T>::initialize_para_now(
+				&mut parachains,
+				id,
+				&ParaGenesisArgs {
+					parachain: true,
+					genesis_head: HeadData(vec![1, 2, 3, 4]),
+					validation_code: old_validation_code,
+				},
+			);
+			// don't care about performance here, but we do care about robustness. So dump the cache
+			// asap.
+			drop(parachains);
+
+			Pallet::<T>::schedule_code_upgrade(
+				id,
+				validation_code,
+				/* relay_parent_number */ 1u32.into(),
+				&configuration::Pallet::<T>::config(),
+			);
+		} else {
+			let r = Pallet::<T>::schedule_para_initialize(
+				id,
+				ParaGenesisArgs {
+					parachain: true,
+					genesis_head: HeadData(vec![1, 2, 3, 4]),
+					validation_code: validation_code(),
+				},
+			);
+			assert!(r.is_ok());
+		}
+	}
+}
+
+/// Generates a list of votes combined with signatures for the active validator set. The number of
+/// votes is equal to the minimum number of votes required to reach the supermajority.
+fn generate_statements<T>(
+	vote_outcome: VoteOutcome,
+) -> impl Iterator<Item = (PvfCheckStatement, ValidatorSignature)>
+where
+	T: Config + shared::Config,
+{
+	let validators = ParasShared::<T>::active_validator_keys();
+
+	let required_votes = primitives::v2::supermajority_threshold(validators.len());
+	(0..required_votes).map(move |validator_index| {
+		let stmt = PvfCheckStatement {
+			accept: vote_outcome == VoteOutcome::Accept,
+			subject: validation_code().hash(),
+			session_index: SESSION_INDEX,
+
+			validator_index: ValidatorIndex(validator_index as u32),
+		};
+		let signature = validators[validator_index].sign(&stmt.signing_payload()).unwrap();
+
+		(stmt, signature)
+	})
+}

polkadot/runtime/parachains/src/paras/mod.rs

@@ -404,6 +404,12 @@ pub trait WeightInfo {
 	fn force_queue_action() -> Weight;
 	fn add_trusted_validation_code(c: u32) -> Weight;
 	fn poke_unused_validation_code() -> Weight;
+
+	fn include_pvf_check_statement_finalize_upgrade_accept() -> Weight;
+	fn include_pvf_check_statement_finalize_upgrade_reject() -> Weight;
+	fn include_pvf_check_statement_finalize_onboarding_accept() -> Weight;
+	fn include_pvf_check_statement_finalize_onboarding_reject() -> Weight;
+	fn include_pvf_check_statement() -> Weight;
 }
 
 pub struct TestWeightInfo;
@@ -429,6 +435,22 @@ impl WeightInfo for TestWeightInfo {
 	fn poke_unused_validation_code() -> Weight {
 		Weight::MAX
 	}
+	fn include_pvf_check_statement_finalize_upgrade_accept() -> Weight {
+		Weight::MAX
+	}
+	fn include_pvf_check_statement_finalize_upgrade_reject() -> Weight {
+		Weight::MAX
+	}
+	fn include_pvf_check_statement_finalize_onboarding_accept() -> Weight {
+		Weight::MAX
+	}
+	fn include_pvf_check_statement_finalize_onboarding_reject() -> Weight {
+		Weight::MAX
+	}
+	fn include_pvf_check_statement() -> Weight {
+		// This special value is to distinguish from the finalizing variants above in tests.
+		Weight::MAX - 1
+	}
 }
 
 #[frame_support::pallet]
@@ -855,12 +877,23 @@ pub mod pallet {
 
 		/// Includes a statement for a PVF pre-checking vote. Potentially, finalizes the vote and
 		/// enacts the results if that was the last vote before achieving the supermajority.
-		#[pallet::weight(Weight::MAX)]
+		#[pallet::weight(
+			sp_std::cmp::max(
+				sp_std::cmp::max(
+					<T as Config>::WeightInfo::include_pvf_check_statement_finalize_upgrade_accept(),
+					<T as Config>::WeightInfo::include_pvf_check_statement_finalize_upgrade_reject(),
+				),
+				sp_std::cmp::max(
+					<T as Config>::WeightInfo::include_pvf_check_statement_finalize_onboarding_accept(),
+					<T as Config>::WeightInfo::include_pvf_check_statement_finalize_onboarding_reject(),
+				)
+			)
+		)]
 		pub fn include_pvf_check_statement(
 			origin: OriginFor<T>,
 			stmt: PvfCheckStatement,
 			signature: ValidatorSignature,
-		) -> DispatchResult {
+		) -> DispatchResultWithPostInfo {
 			ensure_none(origin)?;
 
 			// Make sure that PVF pre-checking is enabled.
@@ -931,13 +964,17 @@ pub mod pallet {
 						Self::enact_pvf_rejected(&stmt.subject, active_vote.causes);
 					},
 				}
-			} else {
-				// No quorum has been achieved. So just store the updated state back into the
-				// storage.
-				PvfActiveVoteMap::<T>::insert(&stmt.subject, active_vote);
-			}
 
-			Ok(())
+				// No weight refund since this statement was the last one and lead to finalization.
+				Ok(().into())
+			} else {
+				// No quorum has been achieved.
+				//
+				// - So just store the updated state back into the storage.
+				// - Only charge weight for simple vote inclusion.
+				PvfActiveVoteMap::<T>::insert(&stmt.subject, active_vote);
+				Ok(Some(<T as Config>::WeightInfo::include_pvf_check_statement()).into())
+			}
 		}
 	}
 

polkadot/runtime/parachains/src/paras/tests.rs

@@ -1277,7 +1277,8 @@ fn pvf_check_submit_vote() {
 			<Paras as ValidateUnsigned>::validate_unsigned(TransactionSource::InBlock, &call)
 				.map(|_| ());
 		let dispatch_result =
-			Paras::include_pvf_check_statement(None.into(), stmt.clone(), signature.clone());
+			Paras::include_pvf_check_statement(None.into(), stmt.clone(), signature.clone())
+				.map(|_| ());
 
 		(validate_unsigned, dispatch_result)
 	};
@@ -1375,6 +1376,69 @@ fn pvf_check_submit_vote() {
 	});
 }
 
+#[test]
+fn include_pvf_check_statement_refunds_weight() {
+	let a = ParaId::from(111);
+	let old_code: ValidationCode = vec![1, 2, 3].into();
+	let new_code: ValidationCode = vec![3, 2, 1].into();
+
+	let paras = vec![(
+		a,
+		ParaGenesisArgs {
+			parachain: false,
+			genesis_head: Default::default(),
+			validation_code: old_code,
+		},
+	)];
+
+	let genesis_config = MockGenesisConfig {
+		paras: GenesisConfig { paras, ..Default::default() },
+		configuration: crate::configuration::GenesisConfig {
+			config: HostConfiguration { pvf_checking_enabled: true, ..Default::default() },
+			..Default::default()
+		},
+		..Default::default()
+	};
+
+	new_test_ext(genesis_config).execute_with(|| {
+		// At this point `a` is already onboarded. Run to block 1 performing session change at
+		// the end of block #0.
+		run_to_block(2, Some(vec![1]));
+
+		// Relay parent of the block that schedules the upgrade.
+		const RELAY_PARENT: BlockNumber = 1;
+		// Expected current session index.
+		const EXPECTED_SESSION: SessionIndex = 1;
+
+		Paras::schedule_code_upgrade(a, new_code.clone(), RELAY_PARENT, &Configuration::config());
+
+		let mut stmts = IntoIterator::into_iter([0, 1, 2, 3])
+			.map(|i| {
+				let stmt = PvfCheckStatement {
+					accept: true,
+					subject: new_code.hash(),
+					session_index: EXPECTED_SESSION,
+					validator_index: (i as u32).into(),
+				};
+				let sig = VALIDATORS[i].sign(&stmt.signing_payload());
+				(stmt, sig)
+			})
+			.collect::<Vec<_>>();
+		let last_one = stmts.pop().unwrap();
+
+		// Verify that just vote submission is priced accordingly.
+		for (stmt, sig) in stmts {
+			let r = Paras::include_pvf_check_statement(None.into(), stmt, sig.into()).unwrap();
+			assert_eq!(r.actual_weight, Some(TestWeightInfo::include_pvf_check_statement()));
+		}
+
+		// Verify that the last statement is priced maximally.
+		let (stmt, sig) = last_one;
+		let r = Paras::include_pvf_check_statement(None.into(), stmt, sig.into()).unwrap();
+		assert_eq!(r.actual_weight, None);
+	});
+}
+
 #[test]
 fn add_trusted_validation_code_inserts_with_no_users() {
 	// This test is to ensure that trusted validation code is inserted into the storage

polkadot/runtime/parachains/src/shared.rs

@@ -124,7 +124,7 @@ impl<T: Config> Pallet<T> {
 		CurrentSessionIndex::<T>::set(index);
 	}
 
-	#[cfg(test)]
+	#[cfg(any(feature = "runtime-benchmarks", test))]
 	pub(crate) fn set_active_validators_ascending(active: Vec<ValidatorId>) {
 		ActiveValidatorIndices::<T>::set(
 			(0..active.len()).map(|i| ValidatorIndex(i as _)).collect(),