mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-07-10 11:07:30 +00:00
f0e589d72e
### What's been done - `subsystem-bench` has been split into two parts: a cli benchmark runner and a library. - The cli runner is quite simple. It just allows us to run `.yaml` based test sequences. Now it should only be used to run benchmarks during development. - The library is used in the cli runner and in regression tests. Some code is changed to make the library independent of the runner. - Added first regression tests for availability read and write that replicate existing test sequences. ### How we run regression tests - Regression tests are simply rust integration tests without the harnesses. - They should only be compiled under the `subsystem-benchmarks` feature to prevent them from running with other tests. - This doesn't work when running tests with `nextest` in CI, so additional filters have been added to the `nextest` runs. - Each benchmark run takes a different time in the beginning, so we "warm up" the tests until their CPU usage differs by only 1%. - After the warm-up, we run the benchmarks a few more times and compare the average with the exception using a precision. ### What is still wrong? - I haven't managed to set up approval voting tests. The spread of their results is too large and can't be narrowed down in a reasonable amount of time in the warm-up phase. - The tests start an unconfigurable prometheus endpoint inside, which causes errors because they use the same 9999 port. I disable it with a flag, but I think it's better to extract the endpoint launching outside the test, as we already do with `valgrind` and `pyroscope`. But we still use `prometheus` inside the tests. ### Future work * https://github.com/paritytech/polkadot-sdk/issues/3528 * https://github.com/paritytech/polkadot-sdk/issues/3529 * https://github.com/paritytech/polkadot-sdk/issues/3530 * https://github.com/paritytech/polkadot-sdk/issues/3531 --------- Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
101 lines
3.6 KiB
Rust
101 lines
3.6 KiB
Rust
// Copyright (C) Parity Technologies (UK) Ltd.
|
|
// This file is part of Polkadot.
|
|
|
|
// Polkadot is free software: you can redistribute it and/or modify
|
|
// it under the terms of the GNU General Public License as published by
|
|
// the Free Software Foundation, either version 3 of the License, or
|
|
// (at your option) any later version.
|
|
|
|
// Polkadot is distributed in the hope that it will be useful,
|
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
// GNU General Public License for more details.
|
|
|
|
// You should have received a copy of the GNU General Public License
|
|
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
//! Dummy subsystem mocks.
|
|
|
|
use futures::FutureExt;
|
|
use paste::paste;
|
|
use polkadot_node_subsystem::{overseer, SpawnedSubsystem, SubsystemError};
|
|
use std::time::Duration;
|
|
use tokio::time::sleep;
|
|
|
|
const LOG_TARGET: &str = "subsystem-bench::mockery";
|
|
|
|
macro_rules! mock {
|
|
// Just query by relay parent
|
|
($subsystem_name:ident) => {
|
|
paste! {
|
|
pub struct [<Mock $subsystem_name >] {}
|
|
#[overseer::subsystem($subsystem_name, error=SubsystemError, prefix=self::overseer)]
|
|
impl<Context> [<Mock $subsystem_name >] {
|
|
fn start(self, ctx: Context) -> SpawnedSubsystem {
|
|
let future = self.run(ctx).map(|_| Ok(())).boxed();
|
|
|
|
// The name will appear in substrate CPU task metrics as `task_group`.`
|
|
SpawnedSubsystem { name: "test-environment", future }
|
|
}
|
|
}
|
|
|
|
#[overseer::contextbounds($subsystem_name, prefix = self::overseer)]
|
|
impl [<Mock $subsystem_name >] {
|
|
async fn run<Context>(self, mut ctx: Context) {
|
|
let mut count_total_msg = 0;
|
|
loop {
|
|
futures::select!{
|
|
msg = ctx.recv().fuse() => {
|
|
match msg.unwrap() {
|
|
orchestra::FromOrchestra::Signal(signal) => {
|
|
match signal {
|
|
polkadot_node_subsystem_types::OverseerSignal::Conclude => {return},
|
|
_ => {}
|
|
}
|
|
},
|
|
orchestra::FromOrchestra::Communication { msg } => {
|
|
gum::debug!(target: LOG_TARGET, msg = ?msg, "mocked subsystem received message");
|
|
}
|
|
}
|
|
|
|
count_total_msg +=1;
|
|
}
|
|
_ = sleep(Duration::from_secs(6)).fuse() => {
|
|
if count_total_msg > 0 {
|
|
gum::trace!(target: LOG_TARGET, "Subsystem {} processed {} messages since last time", stringify!($subsystem_name), count_total_msg);
|
|
}
|
|
count_total_msg = 0;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
};
|
|
}
|
|
|
|
// Generate dummy implementation for all subsystems
|
|
mock!(AvailabilityStore);
|
|
mock!(StatementDistribution);
|
|
mock!(BitfieldSigning);
|
|
mock!(BitfieldDistribution);
|
|
mock!(Provisioner);
|
|
mock!(NetworkBridgeRx);
|
|
mock!(CollationGeneration);
|
|
mock!(CollatorProtocol);
|
|
mock!(GossipSupport);
|
|
mock!(DisputeDistribution);
|
|
mock!(DisputeCoordinator);
|
|
mock!(ProspectiveParachains);
|
|
mock!(PvfChecker);
|
|
mock!(CandidateBacking);
|
|
mock!(AvailabilityDistribution);
|
|
mock!(CandidateValidation);
|
|
mock!(AvailabilityRecovery);
|
|
mock!(NetworkBridgeTx);
|
|
mock!(ChainApi);
|
|
mock!(ChainSelection);
|
|
mock!(ApprovalVoting);
|
|
mock!(ApprovalDistribution);
|
|
mock!(RuntimeApi);
|