mirror of
https://github.com/pezkuwichain/revive.git
synced 2026-10-21 03:06:17 +00:00
Code coverage (#529)
Adds `make coverage` (cargo-llvm-cov over the full workspace including revive-llvm-builder), a new "Code coverage" book chapter that briefly explains how to run it, and a PR workflow that runs coverage and comments on the PR (it updates the same comment if it already exists) when the `measure code coverage` label is applied (see e.g. [this comment](https://github.com/paritytech/revive/pull/529#issuecomment-4966136441)). Building instrumented LLVM and generating an LLVM C++ coverage report is also possible, see accompanying [PR description #575](https://github.com/paritytech/revive/pull/575) for more details. --------- Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Signed-off-by: xermicus <bigcyrill@hotmail.com> Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: xermicus <cyrill@parity.io> Co-authored-by: elle-j <linnea@parity.io> Co-authored-by: LJ <81748770+elle-j@users.noreply.github.com>
This commit is contained in:
40 files changed
+882
-123
No files matched your search
@@ -0,0 +1,53 @@
|
||||
# Code coverage
|
||||
|
||||
Revive measures Rust and LLVM C++ line and region coverage with [`cargo-llvm-cov`](https://github.com/taiki-e/cargo-llvm-cov) over the same tests as `make test-workspace`.
|
||||
|
||||
Two ways to obtain a report:
|
||||
|
||||
1. **Locally:** `make coverage` and `make coverage-llvm-report` (described below).
|
||||
2. **Per PR:** apply the `measure code coverage` label and read the bot-generated PR comment once the triggered workflow completes.
|
||||
|
||||
## Running locally
|
||||
|
||||
```bash
|
||||
# Install LLVM.
|
||||
make install-llvm
|
||||
|
||||
# Or install instrumented LLVM.
|
||||
make install-llvm-coverage
|
||||
|
||||
# ..set LLVM_SYS_221_PREFIX to a compatible LLVM build..
|
||||
|
||||
# Run coverage and generate revive Rust report.
|
||||
make coverage
|
||||
|
||||
# Generate a separate LLVM C++ report if needed, if
|
||||
# instrumented LLVM was used for `make coverage`.
|
||||
make coverage-llvm-report
|
||||
|
||||
# Print clickable links to all accumulated HTML reports detected
|
||||
# (they're grouped by timestamps) to easily view them in the browser.
|
||||
make coverage-browse
|
||||
```
|
||||
|
||||
The target:
|
||||
|
||||
1. Builds and tests the workspace under cargo-llvm-cov with `--ignore-run-fail` so partial coverage lands even when test binaries exit non-zero.
|
||||
2. Writes a browsable HTML report into a timestamped run directory (`coverage-reports/<stamp>/revive/` or `coverage-reports/<stamp>/llvm/`).
|
||||
|
||||
## Resource requirements and troubleshooting
|
||||
|
||||
`make coverage` bakes in two defaults that keep instrumented runs tractable:
|
||||
* It drops DWARF with `CARGO_PROFILE_DEV_DEBUG=0` since coverage line data lives in the binaries' `__llvm_covmap` sections.
|
||||
* It bounds profile output with `LLVM_PROFILE_FILE_NAME=revive-%4m.profraw`. The default pattern contains `%p`, writing one raw profile per process. With an instrumented LLVM this could exceed tens of gigabytes of raw profiles in a full run. The `%4m` merge pool makes processes merge their counters into a handful of files instead.
|
||||
|
||||
Even with those defaults, plan for:
|
||||
|
||||
* Disk space:
|
||||
* Building instrumented LLVM and running an instrumented `make coverage` still needs tens of gigabytes of disk space.
|
||||
* LLVM link memory (`make install-llvm-coverage`):
|
||||
* If a link step gets OOM-killed, rerun with capped parallelism (e.g. `JOBS=1`) to serialize the links. The rerun resumes incrementally.
|
||||
* Test-binary link memory (`make coverage`):
|
||||
* If a link gets OOM-killed, rerun with `RUSTFLAGS="-C link-arg=-fuse-ld=lld"` (or _additionally_ with capped parallelism, e.g. `CARGO_BUILD_JOBS=1`). GNU ld holds every input object and the coverage mapping of the entire dependency closure in memory at once. lld links with a smaller peak.
|
||||
* Test-run memory (`make coverage`):
|
||||
* Each concurrently running test spawns an instrumented resolc. On memory-constrained machines cap it with `RUST_TEST_THREADS=4`. This throttles test execution only, not compilation.
|
||||
Reference in new issue
Block a user