Files

T

pezkuwichain ee389beb8c feat: Add rebrand CI/CD workflows to main branch

- Add 72 rebrand workflow files (polkadot→pezkuwi, substrate→bizinikiwi, cumulus→pezcumulus)
- Add GitHub actions, issue templates, and configs
- Removed unnecessary workflows (fork-sync, gitspiegel, upstream-tracker, sync-templates, backport)
- Renamed zombienet test files to match new naming convention

2025-12-19 22:51:57 +03:00

3.3 KiB

Raw Permalink Blame History

Zombienet Flaky Tests

This document explains how to manage flaky or temporarily disabled zombienet tests in the Pezkuwi SDK repository.

Overview

The .github/zombienet-flaky-tests file contains a list of zombienet tests that are currently disabled due to flakiness or known issues. These tests are automatically skipped during CI runs but are tracked for future re-enabling.

File Format

Each line in the zombienet-flaky-tests file follows this format:

<test-job-name>:<issue-number>

Example:

zombienet-pezkuwi-functional-0014-chunk-fetching-network-compatibility:9980
zombienet-pezcumulus-0009-elastic_scaling_pov_recovery:8986

test-job-name: The exact job name as defined in the zombienet test YAML files
issue-number: GitHub issue number tracking the flaky test (for reference and follow-up)

How It Works

Test Discovery: The zombienet workflows read the test definitions from:
- .github/zombienet-tests/zombienet_pezkuwi_tests.yml
- .github/zombienet-tests/zombienet_cumulus_tests.yml
- .github/zombienet-tests/zombienet_substrate_tests.yml
- .github/zombienet-tests/zombienet_teyrchain-template_tests.yml
Filtering: During the preflight job, tests listed in zombienet-flaky-tests are filtered out from the test matrix.
Execution: Only non-flaky tests are executed in the CI pipeline.

Adding a Flaky Test

If you encounter a flaky test that needs to be temporarily disabled:

Create or find a GitHub issue tracking the flaky behavior

Add an entry to .github/zombienet-flaky-tests:

zombienet-<suite>-<test-name>:<issue-number>

Commit and push the change
The CI will automatically validate that:
- The entry follows the correct format
- The referenced GitHub issue exists
- (Warning if the issue is closed)
The test will be automatically skipped in subsequent CI runs

Re-enabling a Test

Once a flaky test has been fixed:

Verify the fix by running the test locally or in a test branch
Remove the entry from .github/zombienet-flaky-tests
Close the associated GitHub issue (or update it with the fix)
Commit and push the change
The test will be automatically included in subsequent CI runs

Validation

The .github/zombienet-flaky-tests file is automatically validated in CI whenever it's modified. The validation checks:

Format: Each entry must follow the <test-name>:<issue-number> format
Issue vs PR: The referenced number must be a GitHub Issue, not a Pull Request
Issue existence: The referenced GitHub issue must exist in the repository
Issue state: A warning is shown if the referenced issue is closed (suggesting the entry might be outdated)

The validation workflow runs on pull requests that modify:

.github/zombienet-flaky-tests
.github/scripts/check-zombienet-flaky-tests.sh
.github/workflows/check-zombienet-flaky-tests.yml

Monitoring

The number of currently disabled tests is displayed in the CI logs during zombienet test runs
You can view the current list at: .github/zombienet-flaky-tests
Each disabled test should have an associated GitHub issue for tracking
The validation script can be run locally: .github/scripts/check-zombienet-flaky-tests.sh .github/zombienet-flaky-tests

3.3 KiB Raw Permalink Blame History