mirror of
https://github.com/pezkuwichain/revive.git
synced 2026-10-21 03:06:17 +00:00
LJ
3bcb62ac0b
## Description
Adds a workflow executing the resolc and yul benchmarks.
* The workflow can be manually triggered via the Actions UI.
* User input:
* Branch, tag, or SHA before changes (e.g. `main`) _(required)_
* Branch, tag, or SHA after changes (e.g. `lj/ci-benchmarks`)
_(required)_
* PR number _(optional)_
* If provided, the benchmarking result will be posted as a PR comment
* The comment is updated if it exists, otherwise it is created
* If omitted, the result is uploaded as an artifact
This PR originally implemented the workflow to automatically run on each
PR once its tests passed successfully. As we likely do not need to run
this workflow for each PR, the workflow has been rewritten to manually
trigger it.
Providing user input also allows for comparisons between any commits,
such as releases.
## Notes
### File must exist on the default branch
* This workflow is triggered by a `workflow_dispatch` event
[only](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#workflow_dispatch)
if the workflow file exists on the default branch. **Therefore, the
workflow cannot be run for this PR.** (See section below on how it has
still been tested.)
* Once merged, for any subsequent PRs that modify this workflow, those
changes will be testable before merge (by choosing to run the workflow
from the PR branch).
## Testing the Workflow
I've tested a similar enough workflow in a personal repo in order to
push it to `main` and trigger it. Will verify again once this PR is
merged.
- [x] Runs when provided with branch names
- [x] Runs when provided with SHAs
- [x] A PR comment is created if a PR number is provided and the comment
does not exist
- [x] A PR comment is updated if a PR number is provided and the comment
already exists
- [x] An artifact is created if a PR number is omitted
- [x] An in-progress run is cancelled if another one is triggered
comparing the same refs
## Example PR Comment
<details>
<summary>Expand to see an example PR comment</summary>
# Benchmarks
* **Ref Before:** `main`
* **Ref After:** `lj/ci-benchmarks`
## Compile E2E
<details>
<summary>Results</summary>
```
group resolc_before resolc_changes
----- ------------- --------------
Dependency/resolc 1.00 144.0±6.94ms ? ?/sec 1.00 144.6±5.97ms ? ?/sec
Dependency/solc 1.04 65.9±10.73ms ? ?/sec 1.00 63.5±10.10ms ? ?/sec
Empty/resolc 1.03 68.8±4.40ms ? ?/sec 1.00 66.6±4.61ms ? ?/sec
Empty/solc 1.13 11.2±2.01ms ? ?/sec 1.00 10.0±2.28ms ? ?/sec
LargeDivRem/resolc 1.00 119.3±5.62ms ? ?/sec 1.02 121.3±17.03ms ? ?/sec
LargeDivRem/solc 1.15 24.7±3.15ms ? ?/sec 1.00 21.5±1.03ms ? ?/sec
Memset (`--yul`)/resolc 1.02 62.7±3.92ms ? ?/sec 1.00 61.7±3.76ms ? ?/sec
Memset (`--yul`)/solc 1.00 9.0±0.84ms ? ?/sec 1.12 10.1±2.23ms ? ?/sec
Multiple Contracts (`--standard-json`)/resolc 1.01 1559.2±68.95ms ? ?/sec 1.00 1539.3±64.92ms ? ?/sec
Multiple Contracts (`--standard-json`)/solc 1.01 652.0±19.38ms ? ?/sec 1.00 648.4±31.83ms ? ?/sec
Return (`--yul`)/resolc 1.00 60.1±17.02ms ? ?/sec 1.02 61.2±14.50ms ? ?/sec
Return (`--yul`)/solc 1.09 8.8±2.37ms ? ?/sec 1.00 8.0±3.08ms ? ?/sec
```
</details>
## Parse Yul
<details>
<summary>Results</summary>
```
group yul_parse_before yul_parse_changes
----- ---------------- -----------------
Baseline/parse 1.07 8.9±3.64µs ? ?/sec 1.00 8.4±0.09µs ? ?/sec
ERC20/parse 1.00 162.5±34.25µs ? ?/sec 1.07 173.3±123.25µs ? ?/sec
SHA1/parse 1.01 80.8±33.04µs ? ?/sec 1.00 80.3±52.82µs ? ?/sec
Storage/parse 1.00 16.9±8.15µs ? ?/sec 1.01 17.2±7.54µs ? ?/sec
Transfer/parse 1.00 18.5±0.23µs ? ?/sec 1.23 22.7±18.50µs ? ?/sec
```
</details>
## Lower Yul
<details>
<summary>Results</summary>
```
group yul_lower_before yul_lower_changes
----- ---------------- -----------------
Baseline/lower 1.00 122.1±5.95µs ? ?/sec 1.09 133.6±35.12µs ? ?/sec
ERC20/lower 1.00 641.7±13.24µs ? ?/sec 1.15 735.6±665.92µs ? ?/sec
SHA1/lower 1.00 361.4±78.47µs ? ?/sec 1.09 392.3±114.48µs ? ?/sec
Storage/lower 1.00 167.2±50.34µs ? ?/sec 1.01 168.2±32.45µs ? ?/sec
Transfer/lower 1.00 167.5±8.70µs ? ?/sec 1.03 173.1±16.32µs ? ?/sec
```
</details>
</details>
## Resolved Issues
Closes #405