mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-04-27 06:57:58 +00:00
subsystem-bench: add regression tests for availability read and write (#3311)
### What's been done - `subsystem-bench` has been split into two parts: a cli benchmark runner and a library. - The cli runner is quite simple. It just allows us to run `.yaml` based test sequences. Now it should only be used to run benchmarks during development. - The library is used in the cli runner and in regression tests. Some code is changed to make the library independent of the runner. - Added first regression tests for availability read and write that replicate existing test sequences. ### How we run regression tests - Regression tests are simply rust integration tests without the harnesses. - They should only be compiled under the `subsystem-benchmarks` feature to prevent them from running with other tests. - This doesn't work when running tests with `nextest` in CI, so additional filters have been added to the `nextest` runs. - Each benchmark run takes a different time in the beginning, so we "warm up" the tests until their CPU usage differs by only 1%. - After the warm-up, we run the benchmarks a few more times and compare the average with the exception using a precision. ### What is still wrong? - I haven't managed to set up approval voting tests. The spread of their results is too large and can't be narrowed down in a reasonable amount of time in the warm-up phase. - The tests start an unconfigurable prometheus endpoint inside, which causes errors because they use the same 9999 port. I disable it with a flag, but I think it's better to extract the endpoint launching outside the test, as we already do with `valgrind` and `pyroscope`. But we still use `prometheus` inside the tests. ### Future work * https://github.com/paritytech/polkadot-sdk/issues/3528 * https://github.com/paritytech/polkadot-sdk/issues/3529 * https://github.com/paritytech/polkadot-sdk/issues/3530 * https://github.com/paritytech/polkadot-sdk/issues/3531 --------- Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
This commit is contained in:
+1
-1
@@ -7,8 +7,8 @@ TestConfiguration:
|
||||
last_considered_tranche: 89
|
||||
stop_when_approved: false
|
||||
coalesce_tranche_diff: 12
|
||||
workdir_prefix: "/tmp/"
|
||||
num_no_shows_per_candidate: 0
|
||||
workdir_prefix: "/tmp/"
|
||||
n_validators: 500
|
||||
n_cores: 100
|
||||
min_pov_size: 1120
|
||||
|
||||
Reference in New Issue
Block a user