This PR introduces powerful code size optimizations. This is achieved
with an additional IR layer in between Yul and LLVM IR. It works quite
well (see below for a comparison) because it captures optimization gains
that neither solc nor LLVM can realize.
The feature is gated behind `--newyork` or a corresponding standard JSON
input setting and **off by default**.
### How it works
We implemented an SSA IR modelled after or inspired by the [MLIR SCF
dialect](https://mlir.llvm.org/docs/Dialects/SCFDialect/). Yul is
translated into this IR and the IR implements mainly the following
optimizations:
- **Type narrowing**: narrows 256-bit values to I1..I160 where provable
(bidirectional + interprocedural), so LLVM emits native 32/64-bit ops
instead
of software 256-bit arithmetic.
- **FMP range proof**: exposes the `mload(0x40) < heap_size` bound to
LLVM
IPSCCP, eliminating trailing overflow checks on FMP-derived offsets.
Single
largest code-size contributor.
- **Heap analysis**: skips big-endian byte-swaps where native
little-endian
layout is safe.
- **mem_opt / keccak fusion+folding**, **compound outlining** (mapping
sload/sstore), **guard narrowing**, **inlining** tuned for PolkaVM, and
**exact + fuzzy function dedup**.
### Results
The optimizations implemented with the newyork IR bring substantial code
size improvements.
- Integration contracts: approximately **-20% to -50%** code size.
- OpenZeppelin Wizard corpus: around **-47%** code size.
This table shows the code size improvements on various OpenZeppelin ERC
implementations, taken from their [contracts
wizard](https://wizard.openzeppelin.com/), with almost all features
enabled.
| Contract | Default: (bytes) | With `--newyork`: (bytes) | Reduction |
|---|---:|---:|---:|
| erc1155 | 59,206 | 30,649 | -48.2% |
| erc20 | 81,697 | 42,893 | -47.5% |
| erc721 | 92,052 | 49,423 | -46.3% |
| oz_gov | 141,861 | 81,159 | -42.8% |
| oz_rwa | 77,274 | 37,975 | -50.9% |
| oz_simple_erc20 | 29,950 | 16,554 | -44.7% |
| oz_stable | 82,117 | 39,032 | -52.5% |
| **Total** | **564,157** | **297,685** | **-47.2%** |
We believe this to be a reasonably representative dataset and conclude
that the newyork IR achieves drastic code size reductions on real world
dApp implementations.
### Methodology, security and runtime performance
Since the main contributors to these improvements are the FMP range
proofs and the type narrowing, we don't expect big runtime performance
regressions (it may even benefit runtime performance in some cases).
The PR also includes docs
(`book/src/developer_guide/newyork_{optimizer,ir}.md`) and many new
integration tests, including regression tests for known FMP soundness
traps.
The PR was authored collaboratively and reviewed extensively by the
`revive` maintainers, as well as Claude Opus, Claude Fable, Qwen,
Minimax and Deepseek LLMs, over a span of many months.
The `newyork` compiler pipeline introduces no new regressions over our
integration and differential test suites (containing over 30000 test
cases). This was achieved by careful manual reviews and many LLM bughunt
loops. Additionally, a final security review by Anthropic's Fable 5 LLM
found no remaining soundness issues. Nonetheless, like any brand new
software, please consider this an experimental feature upon release.
---------
Signed-off-by: xermicus <bigcyrill@hotmail.com>
Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: neo <me@90f9.org>
Co-authored-by: LJ <81748770+elle-j@users.noreply.github.com>
# Description
Compilation times for resolc were significantly slower than solc when:
* Multiple files were included in the JSON input file via
`--standard-json`
* And none of the files requested code generation
* (If only a subset of files requested code generation, both resolc and
solc were very slow)
Reason:
* Resolc unconditionally generated bytecode for all contracts, even if
not requested.
* When compiling to PolkaVM bytecode, we need to pass additional flags
(e.g. `irOptimized`) to solc. If these flags were not explicitly
requested, resolc also added these to the requested output since PolkaVM
bytecode was always generated.
## Breaking Changes
The following changes are breaking, but fix the semantics to match its
intended behavior:
* Compiling with `--standard-json` now compiles to PolkaVM bytecode
***only*** for the contracts explicitly requested in the
`outputSelection`.
* Compiling with `--standard-json` now outputs ***only*** the explicitly
requested output (e.g. `evm.assembly`, `evm.bytecode`, `ast`, etc.).
```jsonc
// Example:
{
"settings": {
// ...
"outputSelection": {
"src/common/GasService.sol": {
// Contracts in this file will generate bytecode.
// Only these fields of the JSON output selection will be in the `contracts` output.
"*": ["abi", "evm.methodIdentifiers", "metadata", "evm.bytecode"],
// Only this field of the JSON output selection will be in the `sources` output.
"": ["ast"]
},
"src/common/Gateway.sol": {
// No contracts in this file will generate bytecode.
"*": ["abi", "evm.methodIdentifiers", "metadata"],
// No `ast` will be in the `sources` output (only the automatically added `id`,
// similar to solc as this is not a configurable output selection).
"": []
},
}
}
}
```
## Performance Improvements
When e.g. the standard JSON input file includes multiple files and
requests output for every contract as per this setting:
```jsonc
{
"settings": {
// ...
"optimizer": { "enabled": true, "runs": 1 },
"viaIR": true,
"outputSelection": {
"src/common/GasService.sol": {
"*": ["abi", "evm.methodIdentifiers", "metadata", "evm.bytecode"],
},
// Rest of files..
}
}
}
```
The differences in compilation times depending on whether `evm.bytecode`
is selected in every file, one file, or no files can be seen below:
| Solc Optimizer (runs=1) | ViaIR | Request `abi`,
`evm.methodIdentifiers`, `metadata` | Request `evm.bytecode` | resolc
before | resolc after | solc v0.8.33 |
|:-----------------------:|:-----:|:---------:|:---------:|:-------------:|:------------:|:----------:|
| ✅ | ✅ | All files | All files | `~14.5s` | `~14.5s` | `~13.5s` |
| ✅ | ✅ | All files | One file | `~14.5s` | `~450ms` | `~13.5s` |
| ✅ | ✅ | All files | No files | `~14.5s` | `~390ms` | `~350ms` |
**Observations:**
* `resolc before` is roughly the same no matter the request
* This is due to unconditionally generating bytecode
* When not requesting bytecode for any contracts:
* `resolc after` now compiles in ~390ms (vs ~14.5s), similar to `solc`
* Neither `solc` nor `resolc after` generates bytecode when none of the
contracts request it
* When requesting bytecode for all contracts in _only one_ file:
* `resolc after` now compiles in ~450ms (vs ~14.5s)
* `solc` still compiles in the same time as when requesting bytecode for
all contracts in all files
* This is something they have tried to improve (see
[Solidity#15373](https://github.com/argotorg/solidity/issues/15373) and
[Solidity#15615](https://github.com/argotorg/solidity/issues/15615)),
and the intended behavior seems to now be that bytecode should only be
generated when explicitly requested for each contract.
* Even though PRs have merged to try to fix it, their issue seems to
persist as of v0.8.33.
# Open Questions
* Foundry:
* [x] Pruning of the AST:
* Just saw [this past
commit](https://github.com/paritytech/revive/pull/385) to not prune the
AST for Foundry compatibility.
* Can Foundry not explicitly request `"": "ast"`?
* **Answer:**
* Pruning the AST when not requested should be fine, as long as Foundry
explicitly requests it when needed.
* Will be tested by Foundry maintainer.
# Review Notes
Changes in `docs/` (~25 files) can be skipped as they are automatically
updated after adding content to `book/` and rebuilding it.
# Resolved Issues
Closes#326
---------
Co-authored-by: xermicus <cyrill@parity.io>
Fixes https://github.com/paritytech/contract-issues/issues/149
A few test cases from v2-periphery-polkadot are hitting the PVM stack
limit when the contracts are compiled using resolc version 0.5.0. The
same tests pass if the contracts are compiled with a resolc version that
includes commit:
https://github.com/paritytech/revive/commit/878ca91689b99220f7ffd40df760613b7314de86.
After a brief discussion with @xermicus, we propose to set the default
PolkaVM stack and heap size values to 128KB, since we have already
encountered tests that hit the stack size limit and, without real world
data, it is difficult to determine the optimal values.
