mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-15 02:11:07 +00:00
Introduce approval-voting/distribution benchmark (#2621)
## Summary Built on top of the tooling and ideas introduced in https://github.com/paritytech/polkadot-sdk/pull/2528, this PR introduces a synthetic benchmark for measuring and assessing the performance characteristics of the approval-voting and approval-distribution subsystems. Currently this allows, us to simulate the behaviours of these systems based on the following dimensions: ``` TestConfiguration: # Test 1 - objective: !ApprovalsTest last_considered_tranche: 89 min_coalesce: 1 max_coalesce: 6 enable_assignments_v2: true send_till_tranche: 60 stop_when_approved: false coalesce_tranche_diff: 12 workdir_prefix: "/tmp" num_no_shows_per_candidate: 0 approval_distribution_expected_tof: 6.0 approval_distribution_cpu_ms: 3.0 approval_voting_cpu_ms: 4.30 n_validators: 500 n_cores: 100 n_included_candidates: 100 min_pov_size: 1120 max_pov_size: 5120 peer_bandwidth: 524288000000 bandwidth: 524288000000 latency: min_latency: secs: 0 nanos: 1000000 max_latency: secs: 0 nanos: 100000000 error: 0 num_blocks: 10 ``` ## The approach 1. We build a real overseer with the real implementations for approval-voting and approval-distribution subsystems. 2. For a given network size, for each validator we pre-computed all potential assignments and approvals it would send, because this a computation heavy operation this will be cached on a file on disk and be re-used if the generation parameters don't change. 3. The messages will be sent accordingly to the configured parameters and those are split into 3 main benchmarking scenarios. ## Benchmarking scenarios ### Best case scenario *approvals_throughput_best_case.yaml* It send to the approval-distribution only the minimum required tranche to gathered the needed_approvals, so that a candidate is approved. ### Behaviour in the presence of no-shows *approvals_no_shows.yaml* It sends the tranche needed to approve a candidate when we have a maximum of *num_no_shows_per_candidate* tranches with no-shows for each candidate. ### Maximum throughput *approvals_throughput.yaml* It sends all the tranches for each block and measures the used CPU and necessary network bandwidth. by the approval-voting and approval-distribution subsystem. ## How to run it ``` cargo run -p polkadot-subsystem-bench --release -- test-sequence --path polkadot/node/subsystem-bench/examples/approvals_throughput.yaml ``` ## Evaluating performance ### Use the real subsystems metrics If you follow the steps in https://github.com/paritytech/polkadot-sdk/tree/master/polkadot/node/subsystem-bench#install-grafana for installing locally prometheus and grafana, all real metrics for the `approval-distribution`, `approval-voting` and overseer are available. E.g: <img width="2149" alt="Screenshot 2023-12-05 at 11 07 46" src="https://github.com/paritytech/polkadot-sdk/assets/49718502/cb8ae2dd-178b-4922-bfa4-dc37e572ed38"> <img width="2551" alt="Screenshot 2023-12-05 at 11 09 42" src="https://github.com/paritytech/polkadot-sdk/assets/49718502/8b4542ba-88b9-46f9-9b70-cc345366081b"> <img width="2154" alt="Screenshot 2023-12-05 at 11 10 15" src="https://github.com/paritytech/polkadot-sdk/assets/49718502/b8874d8d-632e-443a-9840-14ad8e90c54f"> <img width="2535" alt="Screenshot 2023-12-05 at 11 10 52" src="https://github.com/paritytech/polkadot-sdk/assets/49718502/779a439f-fd18-4985-bb80-85d5afad78e2"> ### Profile with pyroscope 1. Setup pyroscope following the steps in https://github.com/paritytech/polkadot-sdk/tree/master/polkadot/node/subsystem-bench#install-pyroscope, then run any of the benchmark scenario with `--profile` as the arguments. 2. Open the pyroscope dashboard in grafana, e.g: <img width="2544" alt="Screenshot 2024-01-09 at 17 09 58" src="https://github.com/paritytech/polkadot-sdk/assets/49718502/58f50c99-a910-4d20-951a-8b16639303d9"> ### Useful logs 1. Network bandwidth requirements: ``` Payload bytes received from peers: 503993 KiB total, 50399 KiB/block Payload bytes sent to peers: 629971 KiB total, 62997 KiB/block ``` 2. Cpu usage by the approval-distribution/approval-voting subsystems. ``` approval-distribution CPU usage 84.061s approval-distribution CPU usage per block 8.406s approval-voting CPU usage 96.532s approval-voting CPU usage per block 9.653s ``` 3. Time passed until a given block is approved ``` Chain selection approved after 3500 ms hash=0x0101010101010101010101010101010101010101010101010101010101010101 Chain selection approved after 4500 ms hash=0x0202020202020202020202020202020202020202020202020202020202020202 ``` ### Using benchmark to quantify improvements from https://github.com/paritytech/polkadot-sdk/pull/1178 + https://github.com/paritytech/polkadot-sdk/pull/1191 Using a versi-node we compare the scenarios where all new optimisations are disabled with a scenarios where tranche0 assignments are sent in a single message and a conservative simulation where the coalescing of approvals gives us just 50% reduction in the number of messages we send. Overall, what we see is a speedup of around 30-40% in the time it takes to process the necessary messages and a 30-40% reduction in the necessary bandwidth. #### Best case scenario comparison(minimum required tranches sent). Unoptimised ``` Number of blocks: 10 Payload bytes received from peers: 53289 KiB total, 5328 KiB/block Payload bytes sent to peers: 52489 KiB total, 5248 KiB/block approval-distribution CPU usage 6.732s approval-distribution CPU usage per block 0.673s approval-voting CPU usage 9.523s approval-voting CPU usage per block 0.952s ``` vs Optimisation enabled ``` Number of blocks: 10 Payload bytes received from peers: 32141 KiB total, 3214 KiB/block Payload bytes sent to peers: 37314 KiB total, 3731 KiB/block approval-distribution CPU usage 4.658s approval-distribution CPU usage per block 0.466s approval-voting CPU usage 6.236s approval-voting CPU usage per block 0.624s ``` #### Worst case all tranches sent, very unlikely happens when sharding breaks. Unoptimised ``` Number of blocks: 10 Payload bytes received from peers: 746393 KiB total, 74639 KiB/block Payload bytes sent to peers: 729151 KiB total, 72915 KiB/block approval-distribution CPU usage 118.681s approval-distribution CPU usage per block 11.868s approval-voting CPU usage 124.118s approval-voting CPU usage per block 12.412s ``` vs optimised ``` Number of blocks: 10 Payload bytes received from peers: 503993 KiB total, 50399 KiB/block Payload bytes sent to peers: 629971 KiB total, 62997 KiB/block approval-distribution CPU usage 84.061s approval-distribution CPU usage per block 8.406s approval-voting CPU usage 96.532s approval-voting CPU usage per block 9.653s ``` ## TODOs [x] Polish implementation. [x] Use what we have so far to evaluate https://github.com/paritytech/polkadot-sdk/pull/1191 before merging. [x] List of features and additional dimensions we want to use for benchmarking. [x] Run benchmark on hardware similar with versi and kusama nodes. [ ] Add benchmark to be run in CI for catching regression in performance. [ ] Rebase on latest changes for network emulation. --------- Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
This commit is contained in:
Alexandru Gheorghe
committed by
GitHub
GitHub
parent
90849b66b9
commit
f9f886886b
@@ -0,0 +1,686 @@
|
||||
// Copyright (C) Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
use std::{
|
||||
cmp::max,
|
||||
collections::{BTreeMap, HashSet},
|
||||
fs,
|
||||
io::Write,
|
||||
path::{Path, PathBuf},
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
use futures::SinkExt;
|
||||
use itertools::Itertools;
|
||||
use parity_scale_codec::Encode;
|
||||
use polkadot_node_core_approval_voting::{
|
||||
criteria::{compute_assignments, Config},
|
||||
time::tranche_to_tick,
|
||||
};
|
||||
use polkadot_node_network_protocol::grid_topology::{
|
||||
GridNeighbors, RandomRouting, RequiredRouting, SessionGridTopology,
|
||||
};
|
||||
use polkadot_node_primitives::approval::{
|
||||
self,
|
||||
v2::{CoreBitfield, IndirectAssignmentCertV2, IndirectSignedApprovalVoteV2},
|
||||
};
|
||||
use polkadot_primitives::{
|
||||
vstaging::ApprovalVoteMultipleCandidates, CandidateEvent, CandidateHash, CandidateIndex,
|
||||
CoreIndex, SessionInfo, Slot, ValidatorId, ValidatorIndex, ASSIGNMENT_KEY_TYPE_ID,
|
||||
};
|
||||
use rand::{seq::SliceRandom, RngCore, SeedableRng};
|
||||
use rand_chacha::ChaCha20Rng;
|
||||
use rand_distr::{Distribution, Normal};
|
||||
use sc_keystore::LocalKeystore;
|
||||
use sc_network::PeerId;
|
||||
use sha1::Digest;
|
||||
use sp_application_crypto::AppCrypto;
|
||||
use sp_consensus_babe::SlotDuration;
|
||||
use sp_keystore::Keystore;
|
||||
use sp_timestamp::Timestamp;
|
||||
|
||||
use super::{
|
||||
test_message::{MessagesBundle, TestMessageInfo},
|
||||
ApprovalTestState, ApprovalsOptions, BlockTestData,
|
||||
};
|
||||
use crate::{
|
||||
approval::{
|
||||
helpers::{generate_babe_epoch, generate_topology},
|
||||
GeneratedState, BUFFER_FOR_GENERATION_MILLIS, LOG_TARGET, SLOT_DURATION_MILLIS,
|
||||
},
|
||||
core::{
|
||||
configuration::{TestAuthorities, TestConfiguration, TestObjective},
|
||||
mock::session_info_for_peers,
|
||||
NODE_UNDER_TEST,
|
||||
},
|
||||
};
|
||||
use polkadot_node_network_protocol::v3 as protocol_v3;
|
||||
use polkadot_primitives::Hash;
|
||||
use sc_service::SpawnTaskHandle;
|
||||
/// A generator of messages coming from a given Peer/Validator
|
||||
pub struct PeerMessagesGenerator {
|
||||
/// The grid neighbors of the node under test.
|
||||
pub topology_node_under_test: GridNeighbors,
|
||||
/// The topology of the network for the epoch under test.
|
||||
pub topology: SessionGridTopology,
|
||||
/// The validator index for this object generates the messages.
|
||||
pub validator_index: ValidatorIndex,
|
||||
/// An array of pre-generated random samplings, that is used to determine, which nodes would
|
||||
/// send a given assignment, to the node under test because of the random samplings.
|
||||
/// As an optimization we generate this sampling at the begining of the test and just pick
|
||||
/// one randomly, because always taking the samples would be too expensive for benchamrk.
|
||||
pub random_samplings: Vec<Vec<ValidatorIndex>>,
|
||||
/// Channel for sending the generated messages to the aggregator
|
||||
pub tx_messages: futures::channel::mpsc::UnboundedSender<(Hash, Vec<MessagesBundle>)>,
|
||||
/// The list of test authorities
|
||||
pub test_authorities: TestAuthorities,
|
||||
//// The session info used for the test.
|
||||
pub session_info: SessionInfo,
|
||||
/// The blocks used for testing
|
||||
pub blocks: Vec<BlockTestData>,
|
||||
/// Approval options params.
|
||||
pub options: ApprovalsOptions,
|
||||
}
|
||||
|
||||
impl PeerMessagesGenerator {
|
||||
/// Generates messages by spawning a blocking task in the background which begins creating
|
||||
/// the assignments/approvals and peer view changes at the begining of each block.
|
||||
pub fn generate_messages(mut self, spawn_task_handle: &SpawnTaskHandle) {
|
||||
spawn_task_handle.spawn("generate-messages", "generate-messages", async move {
|
||||
for block_info in &self.blocks {
|
||||
let assignments = self.generate_assignments(block_info);
|
||||
|
||||
let bytes = self.validator_index.0.to_be_bytes();
|
||||
let seed = [
|
||||
bytes[0], bytes[1], bytes[2], bytes[3], 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
];
|
||||
|
||||
let mut rand_chacha = ChaCha20Rng::from_seed(seed);
|
||||
let approvals = issue_approvals(
|
||||
assignments,
|
||||
block_info.hash,
|
||||
&self.test_authorities.validator_public,
|
||||
block_info.candidates.clone(),
|
||||
&self.options,
|
||||
&mut rand_chacha,
|
||||
self.test_authorities.keyring.keystore_ref(),
|
||||
);
|
||||
|
||||
self.tx_messages
|
||||
.send((block_info.hash, approvals))
|
||||
.await
|
||||
.expect("Should not fail");
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Builds the messages finger print corresponding to this configuration.
|
||||
// When the finger print exists already on disk the messages are not re-generated.
|
||||
fn messages_fingerprint(
|
||||
configuration: &TestConfiguration,
|
||||
options: &ApprovalsOptions,
|
||||
) -> String {
|
||||
let mut fingerprint = options.fingerprint();
|
||||
let mut exclude_objective = configuration.clone();
|
||||
// The objective contains the full content of `ApprovalOptions`, we don't want to put all of
|
||||
// that in fingerprint, so execlute it because we add it manually see above.
|
||||
exclude_objective.objective = TestObjective::Unimplemented;
|
||||
let configuration_bytes = bincode::serialize(&exclude_objective).unwrap();
|
||||
fingerprint.extend(configuration_bytes);
|
||||
let mut sha1 = sha1::Sha1::new();
|
||||
sha1.update(fingerprint);
|
||||
let result = sha1.finalize();
|
||||
hex::encode(result)
|
||||
}
|
||||
|
||||
/// Generate all messages(Assignments & Approvals) needed for approving `blocks``.
|
||||
pub fn generate_messages_if_needed(
|
||||
configuration: &TestConfiguration,
|
||||
test_authorities: &TestAuthorities,
|
||||
options: &ApprovalsOptions,
|
||||
spawn_task_handle: &SpawnTaskHandle,
|
||||
) -> PathBuf {
|
||||
let path_name = format!(
|
||||
"{}/{}",
|
||||
options.workdir_prefix,
|
||||
Self::messages_fingerprint(configuration, options)
|
||||
);
|
||||
|
||||
let path = Path::new(&path_name);
|
||||
if path.exists() {
|
||||
return path.to_path_buf();
|
||||
}
|
||||
|
||||
gum::info!("Generate message because file does not exist");
|
||||
let delta_to_first_slot_under_test = Timestamp::new(BUFFER_FOR_GENERATION_MILLIS);
|
||||
let initial_slot = Slot::from_timestamp(
|
||||
(*Timestamp::current() - *delta_to_first_slot_under_test).into(),
|
||||
SlotDuration::from_millis(SLOT_DURATION_MILLIS),
|
||||
);
|
||||
|
||||
let babe_epoch = generate_babe_epoch(initial_slot, test_authorities.clone());
|
||||
let session_info = session_info_for_peers(configuration, test_authorities);
|
||||
let blocks = ApprovalTestState::generate_blocks_information(
|
||||
configuration,
|
||||
&babe_epoch,
|
||||
initial_slot,
|
||||
);
|
||||
|
||||
gum::info!(target: LOG_TARGET, "Generate messages");
|
||||
let topology = generate_topology(test_authorities);
|
||||
|
||||
let random_samplings = random_samplings_to_node(
|
||||
ValidatorIndex(NODE_UNDER_TEST),
|
||||
test_authorities.validator_public.len(),
|
||||
test_authorities.validator_public.len() * 2,
|
||||
);
|
||||
|
||||
let topology_node_under_test =
|
||||
topology.compute_grid_neighbors_for(ValidatorIndex(NODE_UNDER_TEST)).unwrap();
|
||||
|
||||
let (tx, mut rx) = futures::channel::mpsc::unbounded();
|
||||
|
||||
// Spawn a thread to generate the messages for each validator, so that we speed up the
|
||||
// generation.
|
||||
for current_validator_index in 1..test_authorities.validator_public.len() {
|
||||
let peer_message_source = PeerMessagesGenerator {
|
||||
topology_node_under_test: topology_node_under_test.clone(),
|
||||
topology: topology.clone(),
|
||||
validator_index: ValidatorIndex(current_validator_index as u32),
|
||||
test_authorities: test_authorities.clone(),
|
||||
session_info: session_info.clone(),
|
||||
blocks: blocks.clone(),
|
||||
tx_messages: tx.clone(),
|
||||
random_samplings: random_samplings.clone(),
|
||||
options: options.clone(),
|
||||
};
|
||||
|
||||
peer_message_source.generate_messages(spawn_task_handle);
|
||||
}
|
||||
|
||||
std::mem::drop(tx);
|
||||
|
||||
let seed = [0x32; 32];
|
||||
let mut rand_chacha = ChaCha20Rng::from_seed(seed);
|
||||
|
||||
let mut all_messages: BTreeMap<u64, Vec<MessagesBundle>> = BTreeMap::new();
|
||||
// Receive all messages and sort them by Tick they have to be sent.
|
||||
loop {
|
||||
match rx.try_next() {
|
||||
Ok(Some((block_hash, messages))) =>
|
||||
for message in messages {
|
||||
let block_info = blocks
|
||||
.iter()
|
||||
.find(|val| val.hash == block_hash)
|
||||
.expect("Should find blocks");
|
||||
let tick_to_send = tranche_to_tick(
|
||||
SLOT_DURATION_MILLIS,
|
||||
block_info.slot,
|
||||
message.tranche_to_send(),
|
||||
);
|
||||
let to_add = all_messages.entry(tick_to_send).or_default();
|
||||
to_add.push(message);
|
||||
},
|
||||
Ok(None) => break,
|
||||
Err(_) => {
|
||||
std::thread::sleep(Duration::from_millis(50));
|
||||
},
|
||||
}
|
||||
}
|
||||
let all_messages = all_messages
|
||||
.into_iter()
|
||||
.flat_map(|(_, mut messages)| {
|
||||
// Shuffle the messages inside the same tick, so that we don't priorites messages
|
||||
// for older nodes. we try to simulate the same behaviour as in real world.
|
||||
messages.shuffle(&mut rand_chacha);
|
||||
messages
|
||||
})
|
||||
.collect_vec();
|
||||
|
||||
gum::info!("Generated a number of {:} unique messages", all_messages.len());
|
||||
|
||||
let generated_state = GeneratedState { all_messages: Some(all_messages), initial_slot };
|
||||
|
||||
let mut messages_file = fs::OpenOptions::new()
|
||||
.write(true)
|
||||
.create(true)
|
||||
.truncate(true)
|
||||
.open(path)
|
||||
.unwrap();
|
||||
|
||||
messages_file
|
||||
.write_all(&generated_state.encode())
|
||||
.expect("Could not update message file");
|
||||
path.to_path_buf()
|
||||
}
|
||||
|
||||
/// Generates assignments for the given `current_validator_index`
|
||||
/// Returns a list of assignments to be sent sorted by tranche.
|
||||
fn generate_assignments(&self, block_info: &BlockTestData) -> Vec<TestMessageInfo> {
|
||||
let config = Config::from(&self.session_info);
|
||||
|
||||
let leaving_cores = block_info
|
||||
.candidates
|
||||
.clone()
|
||||
.into_iter()
|
||||
.map(|candidate_event| {
|
||||
if let CandidateEvent::CandidateIncluded(candidate, _, core_index, group_index) =
|
||||
candidate_event
|
||||
{
|
||||
(candidate.hash(), core_index, group_index)
|
||||
} else {
|
||||
todo!("Variant is never created in this benchmark")
|
||||
}
|
||||
})
|
||||
.collect_vec();
|
||||
|
||||
let mut assignments_by_tranche = BTreeMap::new();
|
||||
|
||||
let bytes = self.validator_index.0.to_be_bytes();
|
||||
let seed = [
|
||||
bytes[0], bytes[1], bytes[2], bytes[3], 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
];
|
||||
let mut rand_chacha = ChaCha20Rng::from_seed(seed);
|
||||
|
||||
let to_be_sent_by = neighbours_that_would_sent_message(
|
||||
&self.test_authorities.peer_ids,
|
||||
self.validator_index.0,
|
||||
&self.topology_node_under_test,
|
||||
&self.topology,
|
||||
);
|
||||
|
||||
let leaving_cores = leaving_cores
|
||||
.clone()
|
||||
.into_iter()
|
||||
.filter(|(_, core_index, _group_index)| core_index.0 != self.validator_index.0)
|
||||
.collect_vec();
|
||||
|
||||
let store = LocalKeystore::in_memory();
|
||||
let _public = store
|
||||
.sr25519_generate_new(
|
||||
ASSIGNMENT_KEY_TYPE_ID,
|
||||
Some(self.test_authorities.key_seeds[self.validator_index.0 as usize].as_str()),
|
||||
)
|
||||
.expect("should not fail");
|
||||
let assignments = compute_assignments(
|
||||
&store,
|
||||
block_info.relay_vrf_story.clone(),
|
||||
&config,
|
||||
leaving_cores.clone(),
|
||||
self.options.enable_assignments_v2,
|
||||
);
|
||||
|
||||
let random_sending_nodes = self
|
||||
.random_samplings
|
||||
.get(rand_chacha.next_u32() as usize % self.random_samplings.len())
|
||||
.unwrap();
|
||||
let random_sending_peer_ids = random_sending_nodes
|
||||
.iter()
|
||||
.map(|validator| (*validator, self.test_authorities.peer_ids[validator.0 as usize]))
|
||||
.collect_vec();
|
||||
|
||||
let mut unique_assignments = HashSet::new();
|
||||
for (core_index, assignment) in assignments {
|
||||
let assigned_cores = match &assignment.cert().kind {
|
||||
approval::v2::AssignmentCertKindV2::RelayVRFModuloCompact { core_bitfield } =>
|
||||
core_bitfield.iter_ones().map(|val| CoreIndex::from(val as u32)).collect_vec(),
|
||||
approval::v2::AssignmentCertKindV2::RelayVRFDelay { core_index } =>
|
||||
vec![*core_index],
|
||||
approval::v2::AssignmentCertKindV2::RelayVRFModulo { sample: _ } =>
|
||||
vec![core_index],
|
||||
};
|
||||
|
||||
let bitfiled: CoreBitfield = assigned_cores.clone().try_into().unwrap();
|
||||
|
||||
// For the cases where tranch0 assignments are in a single certificate we need to make
|
||||
// sure we create a single message.
|
||||
if unique_assignments.insert(bitfiled) {
|
||||
let this_tranche_assignments =
|
||||
assignments_by_tranche.entry(assignment.tranche()).or_insert_with(Vec::new);
|
||||
|
||||
this_tranche_assignments.push((
|
||||
IndirectAssignmentCertV2 {
|
||||
block_hash: block_info.hash,
|
||||
validator: self.validator_index,
|
||||
cert: assignment.cert().clone(),
|
||||
},
|
||||
block_info
|
||||
.candidates
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_index, candidate)| {
|
||||
if let CandidateEvent::CandidateIncluded(_, _, core, _) = candidate {
|
||||
assigned_cores.contains(core)
|
||||
} else {
|
||||
panic!("Should not happen");
|
||||
}
|
||||
})
|
||||
.map(|(index, _)| index as u32)
|
||||
.collect_vec()
|
||||
.try_into()
|
||||
.unwrap(),
|
||||
to_be_sent_by
|
||||
.iter()
|
||||
.chain(random_sending_peer_ids.iter())
|
||||
.copied()
|
||||
.collect::<HashSet<(ValidatorIndex, PeerId)>>(),
|
||||
assignment.tranche(),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
assignments_by_tranche
|
||||
.into_values()
|
||||
.flat_map(|assignments| assignments.into_iter())
|
||||
.map(|assignment| {
|
||||
let msg = protocol_v3::ApprovalDistributionMessage::Assignments(vec![(
|
||||
assignment.0,
|
||||
assignment.1,
|
||||
)]);
|
||||
TestMessageInfo {
|
||||
msg,
|
||||
sent_by: assignment
|
||||
.2
|
||||
.into_iter()
|
||||
.map(|(validator_index, _)| validator_index)
|
||||
.collect_vec(),
|
||||
tranche: assignment.3,
|
||||
block_hash: block_info.hash,
|
||||
}
|
||||
})
|
||||
.collect_vec()
|
||||
}
|
||||
}
|
||||
|
||||
/// A list of random samplings that we use to determine which nodes should send a given message to
|
||||
/// the node under test.
|
||||
/// We can not sample every time for all the messages because that would be too expensive to
|
||||
/// perform, so pre-generate a list of samples for a given network size.
|
||||
/// - result[i] give us as a list of random nodes that would send a given message to the node under
|
||||
/// test.
|
||||
fn random_samplings_to_node(
|
||||
node_under_test: ValidatorIndex,
|
||||
num_validators: usize,
|
||||
num_samplings: usize,
|
||||
) -> Vec<Vec<ValidatorIndex>> {
|
||||
let seed = [7u8; 32];
|
||||
let mut rand_chacha = ChaCha20Rng::from_seed(seed);
|
||||
|
||||
(0..num_samplings)
|
||||
.map(|_| {
|
||||
(0..num_validators)
|
||||
.filter(|sending_validator_index| {
|
||||
*sending_validator_index != NODE_UNDER_TEST as usize
|
||||
})
|
||||
.flat_map(|sending_validator_index| {
|
||||
let mut validators = (0..num_validators).collect_vec();
|
||||
validators.shuffle(&mut rand_chacha);
|
||||
|
||||
let mut random_routing = RandomRouting::default();
|
||||
validators
|
||||
.into_iter()
|
||||
.flat_map(|validator_to_send| {
|
||||
if random_routing.sample(num_validators, &mut rand_chacha) {
|
||||
random_routing.inc_sent();
|
||||
if validator_to_send == node_under_test.0 as usize {
|
||||
Some(ValidatorIndex(sending_validator_index as u32))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
} else {
|
||||
None
|
||||
}
|
||||
})
|
||||
.collect_vec()
|
||||
})
|
||||
.collect_vec()
|
||||
})
|
||||
.collect_vec()
|
||||
}
|
||||
|
||||
/// Helper function to randomly determine how many approvals we coalesce together in a single
|
||||
/// message.
|
||||
fn coalesce_approvals_len(
|
||||
coalesce_mean: f32,
|
||||
coalesce_std_dev: f32,
|
||||
rand_chacha: &mut ChaCha20Rng,
|
||||
) -> usize {
|
||||
max(
|
||||
1,
|
||||
Normal::new(coalesce_mean, coalesce_std_dev)
|
||||
.expect("normal distribution parameters are good")
|
||||
.sample(rand_chacha)
|
||||
.round() as i32,
|
||||
) as usize
|
||||
}
|
||||
|
||||
/// Helper function to create approvals signatures for all assignments passed as arguments.
|
||||
/// Returns a list of Approvals messages that need to be sent.
|
||||
fn issue_approvals(
|
||||
assignments: Vec<TestMessageInfo>,
|
||||
block_hash: Hash,
|
||||
validator_ids: &[ValidatorId],
|
||||
candidates: Vec<CandidateEvent>,
|
||||
options: &ApprovalsOptions,
|
||||
rand_chacha: &mut ChaCha20Rng,
|
||||
store: &LocalKeystore,
|
||||
) -> Vec<MessagesBundle> {
|
||||
let mut queued_to_sign: Vec<TestSignInfo> = Vec::new();
|
||||
let mut num_coalesce =
|
||||
coalesce_approvals_len(options.coalesce_mean, options.coalesce_std_dev, rand_chacha);
|
||||
let result = assignments
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(_index, message)| match &message.msg {
|
||||
protocol_v3::ApprovalDistributionMessage::Assignments(assignments) => {
|
||||
let mut approvals_to_create = Vec::new();
|
||||
|
||||
let current_validator_index = queued_to_sign
|
||||
.first()
|
||||
.map(|msg| msg.validator_index)
|
||||
.unwrap_or(ValidatorIndex(99999));
|
||||
|
||||
// Invariant for this benchmark.
|
||||
assert_eq!(assignments.len(), 1);
|
||||
|
||||
let assignment = assignments.first().unwrap();
|
||||
|
||||
let earliest_tranche = queued_to_sign
|
||||
.first()
|
||||
.map(|val| val.assignment.tranche)
|
||||
.unwrap_or(message.tranche);
|
||||
|
||||
if queued_to_sign.len() >= num_coalesce ||
|
||||
(!queued_to_sign.is_empty() &&
|
||||
current_validator_index != assignment.0.validator) ||
|
||||
message.tranche - earliest_tranche >= options.coalesce_tranche_diff
|
||||
{
|
||||
approvals_to_create.push(TestSignInfo::sign_candidates(
|
||||
&mut queued_to_sign,
|
||||
validator_ids,
|
||||
block_hash,
|
||||
num_coalesce,
|
||||
store,
|
||||
));
|
||||
num_coalesce = coalesce_approvals_len(
|
||||
options.coalesce_mean,
|
||||
options.coalesce_std_dev,
|
||||
rand_chacha,
|
||||
);
|
||||
}
|
||||
|
||||
// If more that one candidate was in the assignment queue all of them for issuing
|
||||
// approvals
|
||||
for candidate_index in assignment.1.iter_ones() {
|
||||
let candidate = candidates.get(candidate_index).unwrap();
|
||||
if let CandidateEvent::CandidateIncluded(candidate, _, _, _) = candidate {
|
||||
queued_to_sign.push(TestSignInfo {
|
||||
candidate_hash: candidate.hash(),
|
||||
candidate_index: candidate_index as CandidateIndex,
|
||||
validator_index: assignment.0.validator,
|
||||
assignment: message.clone(),
|
||||
});
|
||||
} else {
|
||||
todo!("Other enum variants are not used in this benchmark");
|
||||
}
|
||||
}
|
||||
approvals_to_create
|
||||
},
|
||||
_ => {
|
||||
todo!("Other enum variants are not used in this benchmark");
|
||||
},
|
||||
})
|
||||
.collect_vec();
|
||||
|
||||
let mut messages = result.into_iter().flatten().collect_vec();
|
||||
|
||||
if !queued_to_sign.is_empty() {
|
||||
messages.push(TestSignInfo::sign_candidates(
|
||||
&mut queued_to_sign,
|
||||
validator_ids,
|
||||
block_hash,
|
||||
num_coalesce,
|
||||
store,
|
||||
));
|
||||
}
|
||||
messages
|
||||
}
|
||||
|
||||
/// Helper struct to gather information about more than one candidate an sign it in a single
|
||||
/// approval message.
|
||||
struct TestSignInfo {
|
||||
/// The candidate hash
|
||||
candidate_hash: CandidateHash,
|
||||
/// The candidate index
|
||||
candidate_index: CandidateIndex,
|
||||
/// The validator sending the assignments
|
||||
validator_index: ValidatorIndex,
|
||||
/// The assignments convering this candidate
|
||||
assignment: TestMessageInfo,
|
||||
}
|
||||
|
||||
impl TestSignInfo {
|
||||
/// Helper function to create a signture for all candidates in `to_sign` parameter.
|
||||
/// Returns a TestMessage
|
||||
fn sign_candidates(
|
||||
to_sign: &mut Vec<TestSignInfo>,
|
||||
validator_ids: &[ValidatorId],
|
||||
block_hash: Hash,
|
||||
num_coalesce: usize,
|
||||
store: &LocalKeystore,
|
||||
) -> MessagesBundle {
|
||||
let current_validator_index = to_sign.first().map(|val| val.validator_index).unwrap();
|
||||
let tranche_approval_can_be_sent =
|
||||
to_sign.iter().map(|val| val.assignment.tranche).max().unwrap();
|
||||
let validator_id = validator_ids.get(current_validator_index.0 as usize).unwrap().clone();
|
||||
|
||||
let unique_assignments: HashSet<TestMessageInfo> =
|
||||
to_sign.iter().map(|info| info.assignment.clone()).collect();
|
||||
|
||||
let mut to_sign = to_sign
|
||||
.drain(..)
|
||||
.sorted_by(|val1, val2| val1.candidate_index.cmp(&val2.candidate_index))
|
||||
.peekable();
|
||||
|
||||
let mut bundle = MessagesBundle {
|
||||
assignments: unique_assignments.into_iter().collect_vec(),
|
||||
approvals: Vec::new(),
|
||||
};
|
||||
|
||||
while to_sign.peek().is_some() {
|
||||
let to_sign = to_sign.by_ref().take(num_coalesce).collect_vec();
|
||||
|
||||
let hashes = to_sign.iter().map(|val| val.candidate_hash).collect_vec();
|
||||
let candidate_indices = to_sign.iter().map(|val| val.candidate_index).collect_vec();
|
||||
|
||||
let sent_by = to_sign
|
||||
.iter()
|
||||
.flat_map(|val| val.assignment.sent_by.iter())
|
||||
.copied()
|
||||
.collect::<HashSet<ValidatorIndex>>();
|
||||
|
||||
let payload = ApprovalVoteMultipleCandidates(&hashes).signing_payload(1);
|
||||
|
||||
let signature = store
|
||||
.sr25519_sign(ValidatorId::ID, &validator_id.clone().into(), &payload[..])
|
||||
.unwrap()
|
||||
.unwrap()
|
||||
.into();
|
||||
let indirect = IndirectSignedApprovalVoteV2 {
|
||||
block_hash,
|
||||
candidate_indices: candidate_indices.try_into().unwrap(),
|
||||
validator: current_validator_index,
|
||||
signature,
|
||||
};
|
||||
let msg = protocol_v3::ApprovalDistributionMessage::Approvals(vec![indirect]);
|
||||
|
||||
bundle.approvals.push(TestMessageInfo {
|
||||
msg,
|
||||
sent_by: sent_by.into_iter().collect_vec(),
|
||||
tranche: tranche_approval_can_be_sent,
|
||||
block_hash,
|
||||
});
|
||||
}
|
||||
bundle
|
||||
}
|
||||
}
|
||||
|
||||
/// Determine what neighbours would send a given message to the node under test.
|
||||
fn neighbours_that_would_sent_message(
|
||||
peer_ids: &[PeerId],
|
||||
current_validator_index: u32,
|
||||
topology_node_under_test: &GridNeighbors,
|
||||
topology: &SessionGridTopology,
|
||||
) -> Vec<(ValidatorIndex, PeerId)> {
|
||||
let topology_originator = topology
|
||||
.compute_grid_neighbors_for(ValidatorIndex(current_validator_index))
|
||||
.unwrap();
|
||||
|
||||
let originator_y = topology_originator.validator_indices_y.iter().find(|validator| {
|
||||
topology_node_under_test.required_routing_by_index(**validator, false) ==
|
||||
RequiredRouting::GridY
|
||||
});
|
||||
|
||||
assert!(originator_y != Some(&ValidatorIndex(NODE_UNDER_TEST)));
|
||||
|
||||
let originator_x = topology_originator.validator_indices_x.iter().find(|validator| {
|
||||
topology_node_under_test.required_routing_by_index(**validator, false) ==
|
||||
RequiredRouting::GridX
|
||||
});
|
||||
|
||||
assert!(originator_x != Some(&ValidatorIndex(NODE_UNDER_TEST)));
|
||||
|
||||
let is_neighbour = topology_originator
|
||||
.validator_indices_x
|
||||
.contains(&ValidatorIndex(NODE_UNDER_TEST)) ||
|
||||
topology_originator
|
||||
.validator_indices_y
|
||||
.contains(&ValidatorIndex(NODE_UNDER_TEST));
|
||||
|
||||
let mut to_be_sent_by = originator_y
|
||||
.into_iter()
|
||||
.chain(originator_x)
|
||||
.map(|val| (*val, peer_ids[val.0 as usize]))
|
||||
.collect_vec();
|
||||
|
||||
if is_neighbour {
|
||||
to_be_sent_by.push((ValidatorIndex(current_validator_index), peer_ids[0]));
|
||||
}
|
||||
|
||||
to_be_sent_by
|
||||
}
|
||||
Reference in New Issue
Block a user