mirror of
https://github.com/pezkuwichain/revive.git
synced 2026-10-24 12:45:07 +00:00
This PR introduces powerful code size optimizations. This is achieved with an additional IR layer in between Yul and LLVM IR. It works quite well (see below for a comparison) because it captures optimization gains that neither solc nor LLVM can realize. The feature is gated behind `--newyork` or a corresponding standard JSON input setting and **off by default**. ### How it works We implemented an SSA IR modelled after or inspired by the [MLIR SCF dialect](https://mlir.llvm.org/docs/Dialects/SCFDialect/). Yul is translated into this IR and the IR implements mainly the following optimizations: - **Type narrowing**: narrows 256-bit values to I1..I160 where provable (bidirectional + interprocedural), so LLVM emits native 32/64-bit ops instead of software 256-bit arithmetic. - **FMP range proof**: exposes the `mload(0x40) < heap_size` bound to LLVM IPSCCP, eliminating trailing overflow checks on FMP-derived offsets. Single largest code-size contributor. - **Heap analysis**: skips big-endian byte-swaps where native little-endian layout is safe. - **mem_opt / keccak fusion+folding**, **compound outlining** (mapping sload/sstore), **guard narrowing**, **inlining** tuned for PolkaVM, and **exact + fuzzy function dedup**. ### Results The optimizations implemented with the newyork IR bring substantial code size improvements. - Integration contracts: approximately **-20% to -50%** code size. - OpenZeppelin Wizard corpus: around **-47%** code size. This table shows the code size improvements on various OpenZeppelin ERC implementations, taken from their [contracts wizard](https://wizard.openzeppelin.com/), with almost all features enabled. | Contract | Default: (bytes) | With `--newyork`: (bytes) | Reduction | |---|---:|---:|---:| | erc1155 | 59,206 | 30,649 | -48.2% | | erc20 | 81,697 | 42,893 | -47.5% | | erc721 | 92,052 | 49,423 | -46.3% | | oz_gov | 141,861 | 81,159 | -42.8% | | oz_rwa | 77,274 | 37,975 | -50.9% | | oz_simple_erc20 | 29,950 | 16,554 | -44.7% | | oz_stable | 82,117 | 39,032 | -52.5% | | **Total** | **564,157** | **297,685** | **-47.2%** | We believe this to be a reasonably representative dataset and conclude that the newyork IR achieves drastic code size reductions on real world dApp implementations. ### Methodology, security and runtime performance Since the main contributors to these improvements are the FMP range proofs and the type narrowing, we don't expect big runtime performance regressions (it may even benefit runtime performance in some cases). The PR also includes docs (`book/src/developer_guide/newyork_{optimizer,ir}.md`) and many new integration tests, including regression tests for known FMP soundness traps. The PR was authored collaboratively and reviewed extensively by the `revive` maintainers, as well as Claude Opus, Claude Fable, Qwen, Minimax and Deepseek LLMs, over a span of many months. The `newyork` compiler pipeline introduces no new regressions over our integration and differential test suites (containing over 30000 test cases). This was achieved by careful manual reviews and many LLM bughunt loops. Additionally, a final security review by Anthropic's Fable 5 LLM found no remaining soundness issues. Nonetheless, like any brand new software, please consider this an experimental feature upon release. --------- Signed-off-by: xermicus <bigcyrill@hotmail.com> Signed-off-by: xermicus <cyrill@parity.io> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: neo <me@90f9.org> Co-authored-by: LJ <81748770+elle-j@users.noreply.github.com>
199 lines
6.4 KiB
Rust
199 lines
6.4 KiB
Rust
//! The `solc --standard-json` output.
|
|
|
|
use std::collections::BTreeMap;
|
|
|
|
use serde::Deserialize;
|
|
use serde::Serialize;
|
|
|
|
#[cfg(feature = "resolc")]
|
|
use crate::standard_json::input::settings::warning::Warning;
|
|
use crate::standard_json::output::error::error_handler::ErrorHandler;
|
|
#[cfg(feature = "resolc")]
|
|
use crate::SolcStandardJsonInputSettingsSelection;
|
|
#[cfg(feature = "resolc")]
|
|
use crate::SolcStandardJsonInputSource;
|
|
#[cfg(all(feature = "parallel", feature = "resolc"))]
|
|
use rayon::iter::{IntoParallelRefIterator, ParallelIterator};
|
|
|
|
use self::contract::Contract;
|
|
use self::error::Error as SolcStandardJsonOutputError;
|
|
use self::source::Source;
|
|
|
|
pub mod contract;
|
|
pub mod error;
|
|
pub mod source;
|
|
|
|
/// The `solc --standard-json` output.
|
|
#[derive(Debug, Serialize, Deserialize, Clone, Default)]
|
|
pub struct Output {
|
|
/// The file-contract hashmap.
|
|
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
|
|
pub contracts: BTreeMap<String, BTreeMap<String, Contract>>,
|
|
/// The source code mapping data.
|
|
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
|
|
pub sources: BTreeMap<String, Source>,
|
|
/// The compilation errors and warnings.
|
|
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
|
pub errors: Vec<SolcStandardJsonOutputError>,
|
|
/// The `solc` compiler version.
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub version: Option<String>,
|
|
/// The `solc` compiler long version.
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub long_version: Option<String>,
|
|
/// The `resolc` compiler version.
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub revive_version: Option<String>,
|
|
/// The IR pipeline that produced the output: `"newyork"` or `"yul"`.
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub resolc_pipeline: Option<String>,
|
|
}
|
|
|
|
#[cfg(feature = "resolc")]
|
|
impl Output {
|
|
/// Initializes a standard JSON output.
|
|
///
|
|
/// Is used for projects compiled without `solc`.
|
|
pub fn new(
|
|
sources: &BTreeMap<String, SolcStandardJsonInputSource>,
|
|
messages: &mut Vec<SolcStandardJsonOutputError>,
|
|
) -> Self {
|
|
let sources = sources
|
|
.iter()
|
|
.enumerate()
|
|
.map(|(index, (path, source))| {
|
|
(
|
|
path.to_owned(),
|
|
Source {
|
|
id: index,
|
|
ast: source
|
|
.content()
|
|
.map(|x| serde_json::to_value(x).unwrap())
|
|
.unwrap_or_default(),
|
|
},
|
|
)
|
|
})
|
|
.collect::<BTreeMap<String, Source>>();
|
|
|
|
Self {
|
|
contracts: BTreeMap::new(),
|
|
sources: sources.clone(),
|
|
errors: std::mem::take(messages),
|
|
|
|
version: None,
|
|
long_version: None,
|
|
revive_version: None,
|
|
resolc_pipeline: None,
|
|
}
|
|
}
|
|
|
|
/// Initializes a standard JSON output with messages.
|
|
///
|
|
/// Is used to emit errors in standard JSON mode.
|
|
pub fn new_with_messages(messages: Vec<SolcStandardJsonOutputError>) -> Self {
|
|
Self {
|
|
contracts: BTreeMap::new(),
|
|
sources: BTreeMap::new(),
|
|
errors: messages,
|
|
|
|
version: None,
|
|
long_version: None,
|
|
revive_version: None,
|
|
resolc_pipeline: None,
|
|
}
|
|
}
|
|
|
|
/// Prunes the output JSON and prints it to stdout.
|
|
pub fn write_and_exit(
|
|
mut self,
|
|
selection_to_prune: SolcStandardJsonInputSettingsSelection,
|
|
) -> ! {
|
|
for (path, source) in self.sources.iter_mut() {
|
|
if selection_to_prune.contains(
|
|
path,
|
|
crate::SolcStandardJsonInputSettingsSelectionFileFlag::AST,
|
|
) {
|
|
source.ast = Default::default();
|
|
}
|
|
}
|
|
|
|
for (path, contracts) in self.contracts.iter_mut() {
|
|
for contract in contracts.values_mut() {
|
|
for &flag in crate::SolcStandardJsonInputSettingsSelectionFileFlag::all() {
|
|
if selection_to_prune.contains(path, flag) {
|
|
contract.reset_field_by_flag(flag);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
self.contracts.retain(|_, contracts| {
|
|
contracts.retain(|_, contract| !contract.is_empty());
|
|
!contracts.is_empty()
|
|
});
|
|
|
|
serde_json::to_writer(std::io::stdout(), &self).expect("Stdout writing error");
|
|
std::process::exit(revive_common::EXIT_CODE_SUCCESS);
|
|
}
|
|
|
|
/// Traverses the AST and returns the list of additional errors and warnings.
|
|
pub fn preprocess_ast(
|
|
&mut self,
|
|
sources: &BTreeMap<String, SolcStandardJsonInputSource>,
|
|
suppressed_warnings: &[Warning],
|
|
) -> anyhow::Result<()> {
|
|
let id_paths: BTreeMap<usize, &String> = self
|
|
.sources
|
|
.iter()
|
|
.map(|(path, source)| (source.id, path))
|
|
.collect();
|
|
|
|
#[cfg(feature = "parallel")]
|
|
let iter = self.sources.par_iter();
|
|
#[cfg(not(feature = "parallel"))]
|
|
let iter = self.sources.iter();
|
|
|
|
let messages: Vec<SolcStandardJsonOutputError> = iter
|
|
.flat_map(|(_path, source)| {
|
|
Source::get_messages(&source.ast, &id_paths, sources, suppressed_warnings)
|
|
})
|
|
.collect();
|
|
self.errors.extend(messages);
|
|
|
|
Ok(())
|
|
}
|
|
|
|
/// Pushes an arbitrary error with path.
|
|
///
|
|
/// Please do not push project-general errors without paths here.
|
|
pub fn push_error(&mut self, path: Option<String>, error: anyhow::Error) {
|
|
use crate::standard_json::output::error::source_location::SourceLocation;
|
|
|
|
self.errors.push(SolcStandardJsonOutputError::new_error(
|
|
error,
|
|
path.map(SourceLocation::new),
|
|
None,
|
|
));
|
|
}
|
|
}
|
|
|
|
impl ErrorHandler for Output {
|
|
fn errors(&self) -> Vec<&SolcStandardJsonOutputError> {
|
|
self.errors
|
|
.iter()
|
|
.filter(|error| error.is_error())
|
|
.collect()
|
|
}
|
|
|
|
fn take_warnings(&mut self) -> Vec<SolcStandardJsonOutputError> {
|
|
let warnings = self
|
|
.errors
|
|
.iter()
|
|
.filter(|message| message.is_warning())
|
|
.cloned()
|
|
.collect();
|
|
self.errors.retain(|message| !message.is_warning());
|
|
warnings
|
|
}
|
|
}
|