Do not re-prepare PVFs if not needed (#4211)

Currently, PVFs are re-prepared if any execution environment parameter
changes. As we've recently seen on Kusama and Polkadot, that may lead to
a severe finality lag because every validator has to re-prepare every
PVF. That cannot be avoided altogether; however, we could cease
re-preparing PVFs when a change in the execution environment can't lead
to a change in the artifact itself. For example, it's clear that
changing the execution timeout cannot affect the artifact.

In this PR, I'm introducing a separate hash for the subset of execution
environment parameters that changes only if a preparation-related
parameter changes. It introduces some minor code duplication, but
without that, the scope of changes would be much bigger.

TODO:
- [x] Add a test to ensure the artifact is not re-prepared if
non-preparation-related parameter is changed
- [x] Add a test to ensure the artifact is re-prepared if a
preparation-related parameter is changed
- [x] Add comments, warnings, and, possibly, a test to ensure a new
parameter ever added to the executor environment parameters will be
evaluated by the author of changes with respect to its artifact
preparation impact and added to the new hash preimage if needed.

Closes #4132
This commit is contained in:
s0me0ne-unkn0wn
2024-04-25 12:16:12 +02:00
committed by GitHub
parent 239a23d9cc
commit c26cf3f6f2
6 changed files with 201 additions and 10 deletions
+12 -7
View File
@@ -58,7 +58,7 @@ use crate::{host::PrecheckResultSender, worker_interface::WORKER_DIR_PREFIX};
use always_assert::always;
use polkadot_node_core_pvf_common::{error::PrepareError, prepare::PrepareStats, pvf::PvfPrepData};
use polkadot_parachain_primitives::primitives::ValidationCodeHash;
use polkadot_primitives::ExecutorParamsHash;
use polkadot_primitives::ExecutorParamsPrepHash;
use std::{
collections::HashMap,
fs,
@@ -85,22 +85,27 @@ pub fn generate_artifact_path(cache_path: &Path) -> PathBuf {
artifact_path
}
/// Identifier of an artifact. Encodes a code hash of the PVF and a hash of executor parameter set.
/// Identifier of an artifact. Encodes a code hash of the PVF and a hash of preparation-related
/// executor parameter set.
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct ArtifactId {
pub(crate) code_hash: ValidationCodeHash,
pub(crate) executor_params_hash: ExecutorParamsHash,
pub(crate) executor_params_prep_hash: ExecutorParamsPrepHash,
}
impl ArtifactId {
/// Creates a new artifact ID with the given hash.
pub fn new(code_hash: ValidationCodeHash, executor_params_hash: ExecutorParamsHash) -> Self {
Self { code_hash, executor_params_hash }
pub fn new(
code_hash: ValidationCodeHash,
executor_params_prep_hash: ExecutorParamsPrepHash,
) -> Self {
Self { code_hash, executor_params_prep_hash }
}
/// Returns an artifact ID that corresponds to the PVF with given executor params.
/// Returns an artifact ID that corresponds to the PVF with given preparation-related
/// executor parameters.
pub fn from_pvf_prep_data(pvf: &PvfPrepData) -> Self {
Self::new(pvf.code_hash(), pvf.executor_params().hash())
Self::new(pvf.code_hash(), pvf.executor_params().prep_hash())
}
}