mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-05-31 11:01:01 +00:00
Zombienet: add new test collator and integration test (#4797)
* WIP: Wasm compilation perf Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Fix Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Undying collator WIP Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Fix build Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * more fixes Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * update test with undying Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Correctly compute post hash Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * update helper Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * squash bugs Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Add --pov-size cli parameter Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * refactor Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix strings Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Add pov-size param to export genesis state Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Fix graveyard size Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * docs + fmt Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Fix PVF bug and switch to u8 graves Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Update tests Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Build/publish undying collator as colander img Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * add undying-collator to colander image * Fix transaction overflow Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * warn fix Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * add parachain id for export genesis cli Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix the build Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * scale test up Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * default parachain id Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Add PVF complexity param Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Hash on each iteration Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Update pvf metric histogram buckets Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Ladi attempt to fix tests * Fix test * Fix typos * Fix pvf typo * CI integration Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * cargo lock missing Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix clap merge damage Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * add zombienet image back Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Use collator image from env Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * bump zombienet version * update test to check pvf prep/exec Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * delete file Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * use default bootnode in upgrade test * FIx tests Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * add some stress - pvf exec times up to 2s Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix name Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Dial down pvf execution time < 2s Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * 1100 Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * bump Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * try again Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Add connectivity check Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Fixes and refactor folder struct Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * change toml name to match Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * bump zombienet Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * impl Feedback for Review 😎 Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * spell check Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Zombienet: add disputes test (#4859) * Zombienet disputes test Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * CI: add zombienet-parachain-disputes Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Bump zombienet Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * reduce duration Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Update zombienet_tests/functional/0002-parachains-disputes.feature fix test * Update zombienet_tests/functional/0002-parachains-disputes.feature fix fix * more logs and set collator image Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * spellcheck Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix cargo lock damage Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * merge fixes Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * bump zombienet image Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * cargo lock Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * debugging CI run - scale down test Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * fix cargo merge damage Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * default command fix Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> * Revert "debugging CI run - scale down test" This reverts commit eec2ba7e65ede3f929e2f0a8fe44930df2119450. * more review feedback Signed-off-by: Andrei Sandu <andrei-mihail@parity.io> Co-authored-by: Javier Viola <javier@parity.io> Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
This commit is contained in:
Andrei Sandu
@@ -0,0 +1,102 @@
|
||||
Description: PVF preparation & execution time
|
||||
Network: ./0001-parachains-pvf.toml
|
||||
Creds: config
|
||||
|
||||
# Some sanity checks
|
||||
alice: is up
|
||||
bob: is up
|
||||
charlie: is up
|
||||
dave: is up
|
||||
eve: is up
|
||||
ferdie: is up
|
||||
one: is up
|
||||
two: is up
|
||||
|
||||
# Check authority status.
|
||||
alice: reports node_roles is 4
|
||||
bob: reports node_roles is 4
|
||||
charlie: reports node_roles is 4
|
||||
dave: reports node_roles is 4
|
||||
eve: reports node_roles is 4
|
||||
ferdie: reports node_roles is 4
|
||||
one: reports node_roles is 4
|
||||
two: reports node_roles is 4
|
||||
|
||||
# Ensure parachains are registered.
|
||||
alice: parachain 2000 is registered within 60 seconds
|
||||
bob: parachain 2001 is registered within 60 seconds
|
||||
charlie: parachain 2002 is registered within 60 seconds
|
||||
dave: parachain 2003 is registered within 60 seconds
|
||||
ferdie: parachain 2004 is registered within 60 seconds
|
||||
eve: parachain 2005 is registered within 60 seconds
|
||||
one: parachain 2006 is registered within 60 seconds
|
||||
two: parachain 2007 is registered within 60 seconds
|
||||
|
||||
# Check if network is fully connected.
|
||||
alice: reports peers count is at least 15 within 15 seconds
|
||||
bob: reports peers count is at least 15 within 15 seconds
|
||||
charlie: reports peers count is at least 15 within 15 seconds
|
||||
dave: reports peers count is at least 15 within 15 seconds
|
||||
ferdie: reports peers count is at least 15 within 15 seconds
|
||||
eve: reports peers count is at least 15 within 15 seconds
|
||||
one: reports peers count is at least 15 within 15 seconds
|
||||
two: reports peers count is at least 15 within 15 seconds
|
||||
|
||||
# Ensure parachains made progress.
|
||||
alice: parachain 2000 block height is at least 10 within 300 seconds
|
||||
alice: parachain 2001 block height is at least 10 within 300 seconds
|
||||
alice: parachain 2002 block height is at least 10 within 300 seconds
|
||||
alice: parachain 2003 block height is at least 10 within 300 seconds
|
||||
alice: parachain 2004 block height is at least 10 within 300 seconds
|
||||
alice: parachain 2005 block height is at least 10 within 300 seconds
|
||||
alice: parachain 2006 block height is at least 10 within 300 seconds
|
||||
alice: parachain 2007 block height is at least 10 within 300 seconds
|
||||
|
||||
# Check preparation time is under 10s.
|
||||
# Check all buckets <= 10.
|
||||
alice: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
bob: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
charlie: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
dave: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
ferdie: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
eve: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
one: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
two: reports histogram polkadot_pvf_preparation_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2", "3", "10"] within 10 seconds
|
||||
|
||||
# Check all buckets >= 20.
|
||||
alice: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
bob: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
charlie: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
dave: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
ferdie: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
eve: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
one: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
two: reports histogram polkadot_pvf_preparation_time has 0 samples in buckets ["20", "30", "60", "+Inf"] within 10 seconds
|
||||
|
||||
# Check execution time.
|
||||
# There are two different timeout conditions: BACKING_EXECUTION_TIMEOUT(2s) and
|
||||
# APPROVAL_EXECUTION_TIMEOUT(6s). Currently these are not differentiated by metrics
|
||||
# because the metrics are defined in `polkadot-node-core-pvf` which is a level below
|
||||
# the relevant subsystems.
|
||||
# That being said, we will take the simplifying assumption of testing only the
|
||||
# 2s timeout.
|
||||
# We do this check by ensuring all executions fall into bucket le="2" or lower.
|
||||
# First, check if we have at least 1 sample, but we should have many more.
|
||||
alice: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
bob: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
charlie: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
dave: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
ferdie: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
eve: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
one: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
two: reports histogram polkadot_pvf_execution_time has at least 1 samples in buckets ["0.1", "0.5", "1", "2"] within 10 seconds
|
||||
|
||||
# Check if we have no samples > 2s.
|
||||
alice: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
bob: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
charlie: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
dave: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
ferdie: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
eve: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
one: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
two: reports histogram polkadot_pvf_execution_time has 0 samples in buckets ["3", "4", "5", "6", "+Inf"] within 10 seconds
|
||||
Reference in New Issue
Block a user