Files
pezkuwi-subxt/polkadot/node/malus/src/variants/suggest_garbage_candidate.rs
T
s0me0ne-unkn0wn dd0a556665 Executor Environment parameterization (#6161)
* Re-apply changes without Diener, rebase to the lastest master

* Cache pruning

* Bit-pack InstantiationStrategy

* Move ExecutorParams version inside the structure itself

* Rework runtime API and executor parameters storage

* Pass executor parameters through backing subsystem

* Update Cargo.lock

* Introduce `ExecutorParams` to approval voting subsys

* Introduce `ExecutorParams` to dispute coordinator

* `cargo fmt`

* Simplify requests from backing subsys

* Fix tests

* Replace manual config cloning with `.clone()`

* Move constants to module

* Parametrize executor performing PVF pre-check

* Fix Malus

* Fix test runtime

* Introduce session executor params as a constant defined by session info
pallet

* Use Parity SCALE codec instead of hand-crafted binary encoding

* Get rid of constants; Add docs

* Get rid of constants

* Minor typo

* Fix Malus after rebase

* `cargo fmt`

* Use transparent SCALE encoding instead of explicit

* Clean up

* Get rid of relay parent to session index mapping

* Join environment type and version in a single enum element

* Use default execution parameters if running an old runtime

* `unwrap()` -> `expect()`

* Correct API version

* Constants are back in town

* Use constants for execution environment types

* Artifact separation, first try

* Get rid of explicit version

* PVF execution queue worker separation

* Worker handshake

* Global renaming

* Minor fixes resolving discussions

* Two-stage requesting of executor params to make use of runtime API cache

* Proper error handling in pvf-checker

* Executor params storage bootstrapping

* Propagate migration to v3 network runtimes

* Fix storage versioning

* Ensure `ExecutorParams` serialization determinism; Add comments

* Rename constants to make things a bit more deterministic
Get rid of stale code

* Tidy up a structure of active PVFs

* Minor formatting

* Fix comment

* Add try-runtime hooks

* Add storage version write on upgrade

Co-authored-by: Andronik <write@reusable.software>

* Add pre- and post-upgrade assertions

* Require to specify environment type; Remove redundant `impl`s

* Add `ExecutorParamHash` creation from `H256`

* Fix candidate validation subsys tests

* Return splittable error from executor params request fn

* Revert "Return splittable error from executor params request fn"

This reverts commit a0b274177d8bb2f6e13c066741892ecd2e72a456.

* Decompose approval voting metrics

* Use more relevant errors

* Minor formatting fix

* Assert a valid environment type instead of checking

* Fix `try-runtime` hooks

* After-merge fixes

* Add migration logs

* Remove dead code

* Fix tests

* Fix tests

* Back to the strongly typed implementation

* Promote strong types to executor interface

* Remove stale comment

* Move executor params to `SessionInfo`: primitives and runtime

* Move executor params to `SessionInfo`: node

* Try to bump primitives and API version

* Get rid of `MallocSizeOf`

* Bump target API version to v4

* Make use of session index already in place

* Back to v3

* Fix all the tests

* Add migrations to all the runtimes

* Make use of existing `SessionInfo` in approval voting subsys

* Rename `TARGET` -> `LOG_TARGET`

* Bump all the primitives to v3

* Fix Rococo ParachainHost API version

* Use `RollingSessionWindow` to acquire `ExecutorParams` in disputes

* Fix nits from discussions; add comments

* Re-evaluate queue logic

* Rework job assignment in execution queue

* Add documentation

* Use `RuntimeInfo` to obtain `SessionInfo` (with blackjack and caching)

* Couple `Pvf` with `ExecutorParams` wherever possible

* Put members of `PvfWithExecutorParams` under `Arc` for cheap cloning

* Fix comment

* Fix CI tests

* Fix clippy warnings

* Address nits from discussions

* Add a placeholder for raw data

* Fix non exhaustive match

* Remove redundant reexports and fix imports

* Keep only necessary semantic features, as discussed

* Rework `RuntimeInfo` to support mock implementation for tests

* Remove unneeded bound

* `cargo fmt`

* Revert "Remove unneeded bound"

This reverts commit 932463f26b00ce290e1e61848eb9328632ef8a61.

* Fix PVF host tests

* Fix PVF checker tests

* Fix overseer declarations

* Simplify tests

* `MAX_KEEP_WAITING` timeout based on `BACKGING_EXECUTION_TIMEOUT`

* Add a unit test for varying executor parameters

* Minor fixes from discussions

* Add prechecking max. memory parameter (see paritytech/srlabs_findings#110)

* Fix and improve a test

* Remove `ExecutionEnvironment` and `RawData`

* New primitives versioning in parachain host API

* `disputes()` implementation for Kusama and Polkadot

* Move `ExecutorParams` from `vstaging` to stable primitives

* Move disputes from `vstaging` to stable implementation

* Fix `try-runtime`

* Fixes after merge

* Move `ExecutorParams` to the bottom of `SessionInfo`

* Revert "Move executor params to `SessionInfo`: primitives and runtime"

This reverts commit dfcfb85fefd1c5be6c8a8f72dc09fd1809cfa9ce.

* Always use fresh activated live hash in pvf precheck
(re-apply 34b09a4c20de17e7926ed942cd0d657d18f743fa)

* Fixing tests (broken commit)

* Fix candidate validation tests

* Fix PVF host test

* Minor fixes

* Address discussions

* Restore migration

* Fix `use` to only include what is needed instead of `*`

* Add comment to never touch `DEFAULT_CONFIG`

* Update migration to set default `ExecutorParams` for `dispute_period`
sessions back

* Use `earliest_stored_session` instead of calculations

* Nit

* Add logs

* Treat any runtime error as `NotSupported` again

* Always return default executor params if not available

* Revert "Always return default executor params if not available"

This reverts commit b58ac4482ef444c67a9852d5776550d08e312f30.

* Add paritytech/substrate#9997 workaround

* `cargo fmt`

* Remove migration (again!)

* Bump executor params to API v4 (backport from #6698)

---------

Co-authored-by: Andronik <write@reusable.software>
2023-02-15 11:26:09 +00:00

286 lines
9.3 KiB
Rust

// Copyright 2021 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! A malicious node that stores bogus availability chunks, preventing others from
//! doing approval voting. This should lead to disputes depending if the validator
//! has fetched a malicious chunk.
//!
//! Attention: For usage with `zombienet` only!
#![allow(missing_docs)]
use polkadot_cli::{
prepared_overseer_builder,
service::{
AuthorityDiscoveryApi, AuxStore, BabeApi, Block, Error, HeaderBackend, Overseer,
OverseerConnector, OverseerGen, OverseerGenArgs, OverseerHandle, ParachainHost,
ProvideRuntimeApi,
},
Cli,
};
use polkadot_node_core_candidate_validation::find_validation_data;
use polkadot_node_primitives::{AvailableData, BlockData, PoV};
use polkadot_primitives::{CandidateDescriptor, CandidateReceipt};
use polkadot_node_subsystem_util::request_validators;
use sp_core::traits::SpawnNamed;
use rand::distributions::{Bernoulli, Distribution};
// Filter wrapping related types.
use crate::{
interceptor::*,
shared::{MALICIOUS_POV, MALUS},
variants::{
create_fake_candidate_commitments, FakeCandidateValidation, FakeCandidateValidationError,
ReplaceValidationResult,
},
};
// Import extra types relevant to the particular
// subsystem.
use polkadot_node_subsystem::{messages::CandidateBackingMessage, SpawnGlue};
use std::sync::Arc;
/// Replace outgoing approval messages with disputes.
#[derive(Clone)]
struct NoteCandidate<Spawner> {
spawner: Spawner,
percentage: f64,
}
impl<Sender, Spawner> MessageInterceptor<Sender> for NoteCandidate<Spawner>
where
Sender: overseer::CandidateBackingSenderTrait + Clone + Send + 'static,
Spawner: overseer::gen::Spawner + Clone + 'static,
{
type Message = CandidateBackingMessage;
/// Intercept incoming `Second` requests from the `collator-protocol` subsystem.
fn intercept_incoming(
&self,
subsystem_sender: &mut Sender,
msg: FromOrchestra<Self::Message>,
) -> Option<FromOrchestra<Self::Message>> {
match msg {
FromOrchestra::Communication {
msg: CandidateBackingMessage::Second(relay_parent, ref candidate, ref _pov),
} => {
gum::debug!(
target: MALUS,
candidate_hash = ?candidate.hash(),
?relay_parent,
"Received request to second candidate",
);
// Need to draw value from Bernoulli distribution with given probability of success defined by the clap parameter.
// Note that clap parameter must be f64 since this is expected by the Bernoulli::new() function.
// It must be converted from u8, due to the lack of support for the .range() call on u64 in the clap crate.
let distribution = Bernoulli::new(self.percentage / 100.0)
.expect("Invalid probability! Percentage must be in range [0..=100].");
// Draw a random boolean from the Bernoulli distribution with probability of true equal to `p`.
// We use `rand::thread_rng` as the source of randomness.
let generate_malicious_candidate = distribution.sample(&mut rand::thread_rng());
if generate_malicious_candidate == true {
gum::debug!(target: MALUS, "😈 Suggesting malicious candidate.",);
let pov = PoV { block_data: BlockData(MALICIOUS_POV.into()) };
let (sender, receiver) = std::sync::mpsc::channel();
let mut new_sender = subsystem_sender.clone();
let _candidate = candidate.clone();
self.spawner.spawn_blocking(
"malus-get-validation-data",
Some("malus"),
Box::pin(async move {
gum::trace!(target: MALUS, "Requesting validators");
let n_validators = request_validators(relay_parent, &mut new_sender)
.await
.await
.unwrap()
.unwrap()
.len();
gum::trace!(target: MALUS, "Validators {}", n_validators);
match find_validation_data(&mut new_sender, &_candidate.descriptor())
.await
{
Ok(Some((validation_data, validation_code))) => {
sender
.send((validation_data, validation_code, n_validators))
.expect("channel is still open");
},
_ => {
panic!("Unable to fetch validation data");
},
}
}),
);
let (validation_data, validation_code, n_validators) = receiver.recv().unwrap();
let validation_data_hash = validation_data.hash();
let validation_code_hash = validation_code.hash();
let validation_data_relay_parent_number = validation_data.relay_parent_number;
gum::trace!(
target: MALUS,
candidate_hash = ?candidate.hash(),
?relay_parent,
?n_validators,
?validation_data_hash,
?validation_code_hash,
?validation_data_relay_parent_number,
"Fetched validation data."
);
let malicious_available_data =
AvailableData { pov: Arc::new(pov.clone()), validation_data };
let pov_hash = pov.hash();
let erasure_root = {
let chunks = erasure::obtain_chunks_v1(
n_validators as usize,
&malicious_available_data,
)
.unwrap();
let branches = erasure::branches(chunks.as_ref());
branches.root()
};
let (collator_id, collator_signature) = {
use polkadot_primitives::CollatorPair;
use sp_core::crypto::Pair;
let collator_pair = CollatorPair::generate().0;
let signature_payload = polkadot_primitives::collator_signature_payload(
&relay_parent,
&candidate.descriptor().para_id,
&validation_data_hash,
&pov_hash,
&validation_code_hash,
);
(collator_pair.public(), collator_pair.sign(&signature_payload))
};
let malicious_commitments = create_fake_candidate_commitments(
&malicious_available_data.validation_data,
);
let malicious_candidate = CandidateReceipt {
descriptor: CandidateDescriptor {
para_id: candidate.descriptor().para_id,
relay_parent,
collator: collator_id,
persisted_validation_data_hash: validation_data_hash,
pov_hash,
erasure_root,
signature: collator_signature,
para_head: malicious_commitments.head_data.hash(),
validation_code_hash,
},
commitments_hash: malicious_commitments.hash(),
};
let malicious_candidate_hash = malicious_candidate.hash();
let message = FromOrchestra::Communication {
msg: CandidateBackingMessage::Second(
relay_parent,
malicious_candidate,
pov,
),
};
gum::info!(
target: MALUS,
candidate_hash = ?candidate.hash(),
"😈 Intercepted CandidateBackingMessage::Second and created malicious candidate with hash: {:?}",
&malicious_candidate_hash
);
Some(message)
} else {
Some(msg)
}
},
FromOrchestra::Communication { msg } => Some(FromOrchestra::Communication { msg }),
FromOrchestra::Signal(signal) => Some(FromOrchestra::Signal(signal)),
}
}
}
#[derive(Debug, clap::Parser)]
#[clap(rename_all = "kebab-case")]
#[allow(missing_docs)]
pub struct SuggestGarbageCandidateOptions {
/// Determines the percentage of malicious candidates that are suggested by malus,
/// based on the total number of intercepted CandidateBacking
/// Must be in the range [0..=100].
#[clap(short, long, ignore_case = true, default_value_t = 100, value_parser = clap::value_parser!(u8).range(0..=100))]
pub percentage: u8,
#[clap(flatten)]
pub cli: Cli,
}
/// Garbage candidate implementation wrapper which implements `OverseerGen` glue.
pub(crate) struct SuggestGarbageCandidates {
/// The probability of behaving maliciously.
pub percentage: u8,
}
impl OverseerGen for SuggestGarbageCandidates {
fn generate<'a, Spawner, RuntimeClient>(
&self,
connector: OverseerConnector,
args: OverseerGenArgs<'a, Spawner, RuntimeClient>,
) -> Result<(Overseer<SpawnGlue<Spawner>, Arc<RuntimeClient>>, OverseerHandle), Error>
where
RuntimeClient: 'static + ProvideRuntimeApi<Block> + HeaderBackend<Block> + AuxStore,
RuntimeClient::Api: ParachainHost<Block> + BabeApi<Block> + AuthorityDiscoveryApi<Block>,
Spawner: 'static + SpawnNamed + Clone + Unpin,
{
gum::info!(
target: MALUS,
"😈 Started Malus node with a {:?} percent chance of behaving maliciously for a given candidate.",
&self.percentage,
);
let note_candidate = NoteCandidate {
spawner: SpawnGlue(args.spawner.clone()),
percentage: f64::from(self.percentage),
};
let fake_valid_probability = 100.0;
let validation_filter = ReplaceValidationResult::new(
FakeCandidateValidation::BackingAndApprovalValid,
FakeCandidateValidationError::InvalidOutputs,
fake_valid_probability,
SpawnGlue(args.spawner.clone()),
);
prepared_overseer_builder(args)?
.replace_candidate_backing(move |cb| InterceptedSubsystem::new(cb, note_candidate))
.replace_candidate_validation(move |cb| {
InterceptedSubsystem::new(cb, validation_filter)
})
.build_with_connector(connector)
.map_err(|e| e.into())
}
}