mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-04-25 22:17:58 +00:00
Andrei Eres
f0e589d72e
### What's been done - `subsystem-bench` has been split into two parts: a cli benchmark runner and a library. - The cli runner is quite simple. It just allows us to run `.yaml` based test sequences. Now it should only be used to run benchmarks during development. - The library is used in the cli runner and in regression tests. Some code is changed to make the library independent of the runner. - Added first regression tests for availability read and write that replicate existing test sequences. ### How we run regression tests - Regression tests are simply rust integration tests without the harnesses. - They should only be compiled under the `subsystem-benchmarks` feature to prevent them from running with other tests. - This doesn't work when running tests with `nextest` in CI, so additional filters have been added to the `nextest` runs. - Each benchmark run takes a different time in the beginning, so we "warm up" the tests until their CPU usage differs by only 1%. - After the warm-up, we run the benchmarks a few more times and compare the average with the exception using a precision. ### What is still wrong? - I haven't managed to set up approval voting tests. The spread of their results is too large and can't be narrowed down in a reasonable amount of time in the warm-up phase. - The tests start an unconfigurable prometheus endpoint inside, which causes errors because they use the same 9999 port. I disable it with a flag, but I think it's better to extract the endpoint launching outside the test, as we already do with `valgrind` and `pyroscope`. But we still use `prometheus` inside the tests. ### Future work * https://github.com/paritytech/polkadot-sdk/issues/3528 * https://github.com/paritytech/polkadot-sdk/issues/3529 * https://github.com/paritytech/polkadot-sdk/issues/3530 * https://github.com/paritytech/polkadot-sdk/issues/3531 --------- Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
19 lines
443 B
YAML
19 lines
443 B
YAML
TestConfiguration:
|
|
# Test 1
|
|
- objective: !ApprovalVoting
|
|
coalesce_mean: 3.0
|
|
coalesce_std_dev: 1.0
|
|
enable_assignments_v2: true
|
|
last_considered_tranche: 89
|
|
stop_when_approved: true
|
|
coalesce_tranche_diff: 12
|
|
num_no_shows_per_candidate: 10
|
|
workdir_prefix: "/tmp/"
|
|
n_validators: 500
|
|
n_cores: 100
|
|
min_pov_size: 1120
|
|
max_pov_size: 5120
|
|
peer_bandwidth: 524288000000
|
|
bandwidth: 524288000000
|
|
num_blocks: 10
|