mirror of
https://github.com/pezkuwichain/revive.git
synced 2026-10-20 18:56:17 +00:00
The new Yul optimizer toolkit (#519)
This PR introduces powerful code size optimizations. This is achieved with an additional IR layer in between Yul and LLVM IR. It works quite well (see below for a comparison) because it captures optimization gains that neither solc nor LLVM can realize. The feature is gated behind `--newyork` or a corresponding standard JSON input setting and **off by default**. ### How it works We implemented an SSA IR modelled after or inspired by the [MLIR SCF dialect](https://mlir.llvm.org/docs/Dialects/SCFDialect/). Yul is translated into this IR and the IR implements mainly the following optimizations: - **Type narrowing**: narrows 256-bit values to I1..I160 where provable (bidirectional + interprocedural), so LLVM emits native 32/64-bit ops instead of software 256-bit arithmetic. - **FMP range proof**: exposes the `mload(0x40) < heap_size` bound to LLVM IPSCCP, eliminating trailing overflow checks on FMP-derived offsets. Single largest code-size contributor. - **Heap analysis**: skips big-endian byte-swaps where native little-endian layout is safe. - **mem_opt / keccak fusion+folding**, **compound outlining** (mapping sload/sstore), **guard narrowing**, **inlining** tuned for PolkaVM, and **exact + fuzzy function dedup**. ### Results The optimizations implemented with the newyork IR bring substantial code size improvements. - Integration contracts: approximately **-20% to -50%** code size. - OpenZeppelin Wizard corpus: around **-47%** code size. This table shows the code size improvements on various OpenZeppelin ERC implementations, taken from their [contracts wizard](https://wizard.openzeppelin.com/), with almost all features enabled. | Contract | Default: (bytes) | With `--newyork`: (bytes) | Reduction | |---|---:|---:|---:| | erc1155 | 59,206 | 30,649 | -48.2% | | erc20 | 81,697 | 42,893 | -47.5% | | erc721 | 92,052 | 49,423 | -46.3% | | oz_gov | 141,861 | 81,159 | -42.8% | | oz_rwa | 77,274 | 37,975 | -50.9% | | oz_simple_erc20 | 29,950 | 16,554 | -44.7% | | oz_stable | 82,117 | 39,032 | -52.5% | | **Total** | **564,157** | **297,685** | **-47.2%** | We believe this to be a reasonably representative dataset and conclude that the newyork IR achieves drastic code size reductions on real world dApp implementations. ### Methodology, security and runtime performance Since the main contributors to these improvements are the FMP range proofs and the type narrowing, we don't expect big runtime performance regressions (it may even benefit runtime performance in some cases). The PR also includes docs (`book/src/developer_guide/newyork_{optimizer,ir}.md`) and many new integration tests, including regression tests for known FMP soundness traps. The PR was authored collaboratively and reviewed extensively by the `revive` maintainers, as well as Claude Opus, Claude Fable, Qwen, Minimax and Deepseek LLMs, over a span of many months. The `newyork` compiler pipeline introduces no new regressions over our integration and differential test suites (containing over 30000 test cases). This was achieved by careful manual reviews and many LLM bughunt loops. Additionally, a final security review by Anthropic's Fable 5 LLM found no remaining soundness issues. Nonetheless, like any brand new software, please consider this an experimental feature upon release. --------- Signed-off-by: xermicus <bigcyrill@hotmail.com> Signed-off-by: xermicus <cyrill@parity.io> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: neo <me@90f9.org> Co-authored-by: LJ <81748770+elle-j@users.noreply.github.com>
This commit is contained in:
183 files changed
+49503
-561
No files matched your search
@@ -1,7 +1,8 @@
|
||||
//! The debug IR type.
|
||||
|
||||
use revive_common::{
|
||||
EXTENSION_LLVM_SOURCE, EXTENSION_OBJECT, EXTENSION_POLKAVM_ASSEMBLY, EXTENSION_YUL,
|
||||
EXTENSION_LLVM_SOURCE, EXTENSION_NEWYORK, EXTENSION_OBJECT, EXTENSION_POLKAVM_ASSEMBLY,
|
||||
EXTENSION_YUL,
|
||||
};
|
||||
|
||||
/// The debug IR type.
|
||||
@@ -16,6 +17,8 @@ pub enum IRType {
|
||||
Assembly,
|
||||
/// Whether to dump the ELF shared object
|
||||
Object,
|
||||
/// Whether to dump the newyork IR.
|
||||
NewYork,
|
||||
}
|
||||
|
||||
impl IRType {
|
||||
@@ -26,6 +29,7 @@ impl IRType {
|
||||
Self::LLVM => EXTENSION_LLVM_SOURCE,
|
||||
Self::Assembly => EXTENSION_POLKAVM_ASSEMBLY,
|
||||
Self::Object => EXTENSION_OBJECT,
|
||||
Self::NewYork => EXTENSION_NEWYORK,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -115,6 +115,29 @@ impl DebugConfig {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Dumps a newyork IR debug artifact for the current contract.
|
||||
///
|
||||
/// The file name is derived from the contract path (set via [`Self::set_contract_path`]) so it
|
||||
/// matches the other dumps and maps back to the source. `suffix` distinguishes artifacts
|
||||
/// (`"snapshot"`, `"heap"`, `"mem"`); `None` is the final optimized IR. A no-op when no output
|
||||
/// directory or contract path is configured.
|
||||
pub fn dump_newyork(&self, suffix: Option<&str>, text: &str) -> anyhow::Result<()> {
|
||||
if let Some(contract_path) = self.contract_path.as_ref() {
|
||||
let stem = contract_path
|
||||
.file_stem()
|
||||
.and_then(|stem| stem.to_str())
|
||||
.unwrap_or_default();
|
||||
let extension = IRType::NewYork.file_extension();
|
||||
let file_name = match suffix {
|
||||
Some(suffix) => format!("{stem}.{suffix}.{extension}"),
|
||||
None => format!("{stem}.{extension}"),
|
||||
};
|
||||
std::fs::write(contract_path.with_file_name(file_name), text)?;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Creates a full file name, given the contract full path, suffix, and extension.
|
||||
fn full_file_name(contract_path: &str, suffix: Option<&str>, ir_type: IRType) -> String {
|
||||
let mut full_file_name = contract_path.replace('/', "_").replace(':', ".");
|
||||
|
||||
@@ -14,6 +14,7 @@ pub use self::polkavm::build as polkavm_build;
|
||||
pub use self::polkavm::context::address_space::AddressSpace as PolkaVMAddressSpace;
|
||||
pub use self::polkavm::context::argument::Argument as PolkaVMArgument;
|
||||
pub use self::polkavm::context::attribute::Attribute as PolkaVMAttribute;
|
||||
pub use self::polkavm::context::attribute::MemoryEffect as PolkaVMMemoryEffect;
|
||||
pub use self::polkavm::context::build::Build as PolkaVMBuild;
|
||||
pub use self::polkavm::context::code_type::CodeType as PolkaVMCodeType;
|
||||
pub use self::polkavm::context::debug_info::DebugInfo;
|
||||
@@ -27,7 +28,14 @@ pub use self::polkavm::context::function::runtime::arithmetics::SignedDivision a
|
||||
pub use self::polkavm::context::function::runtime::arithmetics::SignedRemainder as PolkaVMSignedRemainderFunction;
|
||||
pub use self::polkavm::context::function::runtime::deploy_code::DeployCode as PolkaVMDeployCodeFunction;
|
||||
pub use self::polkavm::context::function::runtime::entry::Entry as PolkaVMEntryFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::CallDataLoad as PolkaVMCallDataLoadFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::CallValue as PolkaVMCallValueFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::CallValueNonzero as PolkaVMCallValueNonzeroFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::Caller as PolkaVMCallerFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::Exit as PolkaVMExitFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::Revert as PolkaVMRevertFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::RevertEmpty as PolkaVMRevertEmptyFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::RevertPanic as PolkaVMRevertPanicFunction;
|
||||
pub use self::polkavm::context::function::runtime::revive::WordToPointer as PolkaVMWordToPointerFunction;
|
||||
pub use self::polkavm::context::function::runtime::runtime_code::RuntimeCode as PolkaVMRuntimeCodeFunction;
|
||||
pub use self::polkavm::context::function::runtime::sbrk::Sbrk as PolkaVMSbrkFunction;
|
||||
@@ -37,8 +45,12 @@ pub use self::polkavm::context::function::runtime::FUNCTION_RUNTIME_CODE as Polk
|
||||
pub use self::polkavm::context::function::yul_data::YulData as PolkaVMFunctionYulData;
|
||||
pub use self::polkavm::context::function::Function as PolkaVMFunction;
|
||||
pub use self::polkavm::context::global::Global as PolkaVMGlobal;
|
||||
pub use self::polkavm::context::pointer::heap::Keccak256OneWord as PolkaVMKeccak256OneWordFunction;
|
||||
pub use self::polkavm::context::pointer::heap::Keccak256TwoWords as PolkaVMKeccak256TwoWordsFunction;
|
||||
pub use self::polkavm::context::pointer::heap::LoadWord as PolkaVMLoadHeapWordFunction;
|
||||
pub use self::polkavm::context::pointer::heap::LoadWordNative as PolkaVMLoadHeapWordNativeFunction;
|
||||
pub use self::polkavm::context::pointer::heap::StoreWord as PolkaVMStoreHeapWordFunction;
|
||||
pub use self::polkavm::context::pointer::heap::StoreWordNative as PolkaVMStoreHeapWordNativeFunction;
|
||||
pub use self::polkavm::context::pointer::storage::LoadTransientWord as PolkaVMLoadTransientStorageWordFunction;
|
||||
pub use self::polkavm::context::pointer::storage::LoadWord as PolkaVMLoadStorageWordFunction;
|
||||
pub use self::polkavm::context::pointer::storage::StoreTransientWord as PolkaVMStoreTransientStorageWordFunction;
|
||||
@@ -84,19 +96,62 @@ pub(crate) mod target_machine;
|
||||
|
||||
static DID_INITIALIZE: OnceLock<()> = OnceLock::new();
|
||||
|
||||
/// The global LLVM options applied at [`OptimizerSettingsSizeLevel::Z`] **on the newyork path**.
|
||||
///
|
||||
/// These trade execution speed for code size: the machine outliner replaces repeated
|
||||
/// instruction sequences with calls, disabling LICM avoids i256 register pressure on the
|
||||
/// PVM target (rv64e) where hoisting causes excessive stack spills, and `ext-tsp-for-size`
|
||||
/// optimizes block layout for size instead of speed. At any other optimization level the
|
||||
/// upstream LLVM defaults apply: paying execution gas for smaller code is only acceptable
|
||||
/// when the user explicitly asked for `-Oz`.
|
||||
///
|
||||
/// They are applied only for modules produced by the newyork IR generator. Like the
|
||||
/// aggressive code-size pass pipeline in [`crate::optimizer::Optimizer::run`] and the
|
||||
/// `+unaligned-scalar-mem` target feature, these knobs are tuned and validated against
|
||||
/// newyork's outlined code; the stock Yul path keeps the upstream LLVM defaults it was
|
||||
/// validated against (matching `main`).
|
||||
const SIZE_LEVEL_Z_ARGUMENTS: &[&str] = &[
|
||||
"--disable-licm-promotion",
|
||||
"--disable-machine-licm",
|
||||
"--enable-machine-outliner",
|
||||
"--machine-outliner-reruns=2",
|
||||
"--disable-early-taildup=true",
|
||||
"--apply-ext-tsp-for-size=true",
|
||||
"--attributor-allow-deep-wrappers=true",
|
||||
"--hoist-common-insts=true",
|
||||
];
|
||||
|
||||
/// Initializes the LLVM compiler backend.
|
||||
///
|
||||
/// This is a no-op if called subsequentially.
|
||||
///
|
||||
/// `llvm_arguments` are passed as-is to the LLVM CL options parser.
|
||||
pub fn initialize_llvm(target: PolkaVMTarget, name: &str, llvm_arguments: &[String]) {
|
||||
///
|
||||
/// When `use_newyork` is set and the size level is [`OptimizerSettingsSizeLevel::Z`], the
|
||||
/// `SIZE_LEVEL_Z_ARGUMENTS` are prepended. They are gated on the newyork path because they
|
||||
/// are validated against its codegen; the stock Yul path must keep the upstream defaults.
|
||||
/// LLVM parses command line options once per process, so the first caller wins; production
|
||||
/// builds are unaffected because every contract is compiled in its own recursive process
|
||||
/// with its own optimizer settings and IR generator.
|
||||
pub fn initialize_llvm(
|
||||
target: PolkaVMTarget,
|
||||
name: &str,
|
||||
size_level: OptimizerSettingsSizeLevel,
|
||||
use_newyork: bool,
|
||||
llvm_arguments: &[String],
|
||||
) {
|
||||
let Ok(_) = DID_INITIALIZE.set(()) else {
|
||||
return; // Tests don't go through a recursive process
|
||||
};
|
||||
|
||||
let argv = [name.to_string()]
|
||||
.iter()
|
||||
.chain(llvm_arguments)
|
||||
let size_arguments = if use_newyork && size_level == OptimizerSettingsSizeLevel::Z {
|
||||
SIZE_LEVEL_Z_ARGUMENTS
|
||||
} else {
|
||||
&[]
|
||||
};
|
||||
let argv = std::iter::once(name)
|
||||
.chain(size_arguments.iter().copied())
|
||||
.chain(llvm_arguments.iter().map(String::as_str))
|
||||
.map(|arg| CString::new(arg.as_bytes()).unwrap())
|
||||
.collect::<Vec<_>>();
|
||||
let argv: Vec<*const libc::c_char> = argv.iter().map(|arg| arg.as_ptr()).collect();
|
||||
|
||||
@@ -14,24 +14,73 @@ pub mod settings;
|
||||
pub struct Optimizer {
|
||||
/// The optimizer settings.
|
||||
settings: Settings,
|
||||
/// Whether the module being optimized was produced by the newyork IR
|
||||
/// generator. The aggressive code-size pipeline is specific to newyork's
|
||||
/// outlined code; the stock Yul path must not run it (see [`Optimizer::run`]).
|
||||
#[serde(skip)]
|
||||
newyork: bool,
|
||||
}
|
||||
|
||||
impl Optimizer {
|
||||
/// A shortcut constructor.
|
||||
pub fn new(settings: Settings) -> Self {
|
||||
Self { settings }
|
||||
Self {
|
||||
settings,
|
||||
newyork: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Marks this optimizer as operating on a module produced by the newyork IR
|
||||
/// generator, enabling the aggressive code-size pipeline in [`Optimizer::run`].
|
||||
pub fn enable_newyork_pipeline(&mut self) {
|
||||
self.newyork = true;
|
||||
}
|
||||
|
||||
/// Whether the module being optimized was produced by the newyork IR generator.
|
||||
/// Gates the two pieces of newyork-specific codegen that the stock Yul path must not use:
|
||||
/// the aggressive code-size pipeline in [`Optimizer::run`] and the `+unaligned-scalar-mem`
|
||||
/// target feature in [`crate::target_machine::TargetMachine`].
|
||||
pub fn is_newyork(&self) -> bool {
|
||||
self.newyork
|
||||
}
|
||||
|
||||
/// Runs the new pass manager.
|
||||
///
|
||||
/// For modules produced by the **newyork** IR generator a code-size pipeline runs:
|
||||
///
|
||||
/// 1. `default<O{level}>`: initial optimization at the configured size level.
|
||||
/// 2. `ipsccp,deadargelim`: propagate inter-function constants through outlined helpers and
|
||||
/// remove now-constant arguments.
|
||||
/// 3. `attributor`: infer function attributes the next stage can exploit.
|
||||
/// 4. `default<O1>`: re-optimize with newly discovered constants and attributes.
|
||||
///
|
||||
/// `mergefunc` runs before stage 1, after stage 1, and after stage 4 to deduplicate functions
|
||||
/// that become identical as the surrounding passes canonicalize them.
|
||||
/// The second `default` pass is hard-coded to `O1` rather than the configured size level: an
|
||||
/// `Oz` second pass reproducibly changes the post-link PVM blob for some contracts in a way
|
||||
/// that fails pallet-revive deployment ("Code upload failed: Timeout"). The exact pass has not
|
||||
/// been isolated, so `O1` is kept as the empirically safe choice.
|
||||
///
|
||||
/// This pipeline is specific to newyork's outlined code. The stock Yul path uses a plain
|
||||
/// `default<O{level}>`, matching upstream: its IR does not benefit from the extra inter-
|
||||
/// procedural passes, and `attributor`'s inference is unsound on it (the `attributor`+`O1`
|
||||
/// combination miscompiles, e.g. `complex/array_one_element` returns the callee selector
|
||||
/// instead of the external-call result). Gating on the IR generator keeps each path on the
|
||||
/// pipeline it was validated against.
|
||||
pub fn run(
|
||||
&self,
|
||||
target_machine: &TargetMachine,
|
||||
module: &inkwell::module::Module,
|
||||
) -> Result<(), inkwell::support::LLVMString> {
|
||||
target_machine.run_optimization_passes(
|
||||
module,
|
||||
format!("default<O{}>", self.settings.middle_end_as_string()).as_str(),
|
||||
)
|
||||
let optimization_level = self.settings.middle_end_as_string();
|
||||
let pass_pipeline = if self.newyork {
|
||||
format!(
|
||||
"mergefunc,default<O{optimization_level}>,mergefunc,ipsccp,deadargelim,attributor,default<O1>,mergefunc"
|
||||
)
|
||||
} else {
|
||||
format!("default<O{optimization_level}>")
|
||||
};
|
||||
target_machine.run_optimization_passes(module, &pass_pipeline)
|
||||
}
|
||||
|
||||
/// Returns the optimizer settings reference.
|
||||
|
||||
@@ -53,6 +53,11 @@ impl<'ctx> Argument<'ctx> {
|
||||
self
|
||||
}
|
||||
|
||||
/// Returns true if the argument holds an LLVM register value (not a pointer).
|
||||
pub fn is_register(&self) -> bool {
|
||||
matches!(&self.value, Value::Register(_))
|
||||
}
|
||||
|
||||
/// Returns the inner LLVM value.
|
||||
///
|
||||
/// Panics if `self` is a pointer argument.
|
||||
|
||||
@@ -121,6 +121,147 @@ pub enum Attribute {
|
||||
//LastConstantRangeListAttr = 101,
|
||||
}
|
||||
|
||||
/// Rust mirror of LLVM's `llvm::ModRefInfo`.
|
||||
///
|
||||
/// Discriminants are the C++ enum values verbatim from
|
||||
/// `llvm/include/llvm/Support/ModRef.h` lines 28-38 of the LLVM 21.1.8
|
||||
/// source tree (`LLVM_SYS_211_PREFIX`):
|
||||
///
|
||||
/// ```text
|
||||
/// enum class ModRefInfo : uint8_t {
|
||||
/// NoModRef = 0,
|
||||
/// Ref = 1,
|
||||
/// Mod = 2,
|
||||
/// ModRef = Ref | Mod,
|
||||
/// };
|
||||
/// ```
|
||||
#[allow(dead_code)] // ModRef mirror — every variant kept for completeness.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[repr(u8)]
|
||||
enum ModRefInfo {
|
||||
NoModRef = 0,
|
||||
Ref = 1,
|
||||
Mod = 2,
|
||||
ModRef = 3,
|
||||
}
|
||||
|
||||
/// Rust mirror of LLVM's `llvm::IRMemLocation`.
|
||||
///
|
||||
/// Discriminants are the C++ enum values verbatim from
|
||||
/// `llvm/include/llvm/Support/ModRef.h` lines 60-73 of the LLVM 21.1.8
|
||||
/// source tree:
|
||||
///
|
||||
/// ```text
|
||||
/// enum class IRMemLocation {
|
||||
/// ArgMem = 0,
|
||||
/// InaccessibleMem = 1,
|
||||
/// ErrnoMem = 2,
|
||||
/// Other = 3,
|
||||
/// ...
|
||||
/// };
|
||||
/// ```
|
||||
///
|
||||
/// The discriminant also serves as the location's index into the packed
|
||||
/// `memory(...)` payload — see the `memory_location_pos` helper.
|
||||
#[allow(dead_code)] // LLVM location mirror — every variant kept for completeness.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[repr(u8)]
|
||||
enum MemoryLocation {
|
||||
ArgMem = 0,
|
||||
InaccessibleMem = 1,
|
||||
ErrnoMem = 2,
|
||||
Other = 3,
|
||||
}
|
||||
|
||||
/// Mirror of `llvm::MemoryEffectsBase::BitsPerLoc` — see
|
||||
/// `llvm/include/llvm/Support/ModRef.h:82` of LLVM 21.1.8:
|
||||
///
|
||||
/// ```text
|
||||
/// static constexpr uint32_t BitsPerLoc = 2;
|
||||
/// ```
|
||||
const MEMORY_EFFECT_BITS_PER_LOCATION: u64 = 2;
|
||||
|
||||
/// Mirror of `llvm::MemoryEffectsBase::getLocationPos` — see
|
||||
/// `llvm/include/llvm/Support/ModRef.h:85-87` of LLVM 21.1.8:
|
||||
///
|
||||
/// ```text
|
||||
/// static uint32_t getLocationPos(Location Loc) {
|
||||
/// return (uint32_t)Loc * BitsPerLoc;
|
||||
/// }
|
||||
/// ```
|
||||
const fn memory_location_pos(location: MemoryLocation) -> u64 {
|
||||
(location as u64) * MEMORY_EFFECT_BITS_PER_LOCATION
|
||||
}
|
||||
|
||||
/// Mirror of `llvm::MemoryEffectsBase::setModRef` for a single location on
|
||||
/// an initially-zero payload — see `llvm/include/llvm/Support/ModRef.h:91-94`
|
||||
/// of LLVM 21.1.8:
|
||||
///
|
||||
/// ```text
|
||||
/// void setModRef(Location Loc, ModRefInfo MR) {
|
||||
/// Data &= ~(LocMask << getLocationPos(Loc));
|
||||
/// Data |= static_cast<uint32_t>(MR) << getLocationPos(Loc);
|
||||
/// }
|
||||
/// ```
|
||||
const fn pack_memory_effect(location: MemoryLocation, mod_ref: ModRefInfo) -> u64 {
|
||||
(mod_ref as u64) << memory_location_pos(location)
|
||||
}
|
||||
|
||||
/// Per-location memory effect packed into the integer payload of
|
||||
/// [`Attribute::Memory`]. Every variant's encoding is derived through the
|
||||
/// crate-private `pack_memory_effect` helper from `MemoryLocation` /
|
||||
/// `ModRefInfo`, which are Rust mirrors of LLVM 21 enums with discriminants
|
||||
/// matching C++. No raw integer literals appear in [`MemoryEffect::encoding`].
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum MemoryEffect {
|
||||
/// Do not attach a `memory(...)` attribute to the function.
|
||||
Unrestricted,
|
||||
/// `memory(none)` — function has no observable memory effect.
|
||||
None,
|
||||
/// `memory(inaccessiblemem: read)` — the function only reads pallet-revive
|
||||
/// runtime state (storage, call context, etc.) reached via host syscalls.
|
||||
/// LLVM sees no pointer effects from the caller's perspective.
|
||||
ReadInaccessible,
|
||||
/// `memory(argmem: read, inaccessiblemem: read)` — the function reads from
|
||||
/// a pointer argument **and** from pallet-revive runtime state. Used by
|
||||
/// helpers that dereference a key buffer and forward into a host syscall.
|
||||
ReadArgAndInaccessible,
|
||||
/// `memory(inaccessiblemem: write)` — the only externally visible effect
|
||||
/// is a write into pallet-revive runtime state. Heap loads remain CSE-able
|
||||
/// across calls to such a helper.
|
||||
WriteInaccessible,
|
||||
/// `memory(other: read)` — function reads from regular heap memory only.
|
||||
/// Pointer arguments, runtime state and errno are all untouched, so the
|
||||
/// helper is invalidated by any heap store rather than by sstore wrappers.
|
||||
ReadOther,
|
||||
}
|
||||
|
||||
impl MemoryEffect {
|
||||
/// LLVM's packed integer encoding of this effect, or `None` when no
|
||||
/// `memory(...)` attribute should be applied. See the crate-private
|
||||
/// `pack_memory_effect` helper above for the LLVM 21 source the
|
||||
/// encoding mirrors.
|
||||
pub const fn encoding(self) -> Option<u64> {
|
||||
match self {
|
||||
Self::Unrestricted => None,
|
||||
Self::None => Some(0),
|
||||
Self::ReadInaccessible => Some(pack_memory_effect(
|
||||
MemoryLocation::InaccessibleMem,
|
||||
ModRefInfo::Ref,
|
||||
)),
|
||||
Self::ReadArgAndInaccessible => Some(
|
||||
pack_memory_effect(MemoryLocation::ArgMem, ModRefInfo::Ref)
|
||||
| pack_memory_effect(MemoryLocation::InaccessibleMem, ModRefInfo::Ref),
|
||||
),
|
||||
Self::WriteInaccessible => Some(pack_memory_effect(
|
||||
MemoryLocation::InaccessibleMem,
|
||||
ModRefInfo::Mod,
|
||||
)),
|
||||
Self::ReadOther => Some(pack_memory_effect(MemoryLocation::Other, ModRefInfo::Ref)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl TryFrom<&str> for Attribute {
|
||||
type Error = String;
|
||||
|
||||
@@ -139,3 +280,27 @@ impl TryFrom<&str> for Attribute {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod memory_effect_tests {
|
||||
use super::*;
|
||||
|
||||
/// Locks the encoding against LLVM 21.1.8's `MemoryEffectsBase` layout.
|
||||
/// Expected integers were cross-checked by compiling probe contracts and
|
||||
/// reading the `memory(...)` strings off the emitted LLVM IR — every
|
||||
/// helper using one of these variants printed the matching attribute
|
||||
/// (e.g. `memory(inaccessiblemem: read)` for `ReadInaccessible`,
|
||||
/// `memory(read, argmem: none, inaccessiblemem: none, errnomem: none)` —
|
||||
/// LLVM 21's verbose form of `memory(other: read)` — for `ReadOther`).
|
||||
/// If LLVM ever reorders `IRMemLocation` (as happened between LLVM 18 and
|
||||
/// 21 when `ErrnoMem` was added), this test fires before codegen does.
|
||||
#[test]
|
||||
fn encoding_matches_llvm_21_layout() {
|
||||
assert_eq!(MemoryEffect::Unrestricted.encoding(), None);
|
||||
assert_eq!(MemoryEffect::None.encoding(), Some(0));
|
||||
assert_eq!(MemoryEffect::ReadInaccessible.encoding(), Some(4));
|
||||
assert_eq!(MemoryEffect::ReadArgAndInaccessible.encoding(), Some(5));
|
||||
assert_eq!(MemoryEffect::WriteInaccessible.encoding(), Some(8));
|
||||
assert_eq!(MemoryEffect::ReadOther.encoding(), Some(64));
|
||||
}
|
||||
}
|
||||
@@ -97,6 +97,12 @@ impl<'ctx> Function<'ctx> {
|
||||
}
|
||||
|
||||
/// Sets the memory writer function attributes.
|
||||
///
|
||||
/// `Attribute::Memory` is **not** valid here — it carries a payload that
|
||||
/// this enum-only API cannot express. Use the `MEMORY_EFFECT` associated
|
||||
/// constant on the `RuntimeFunction` trait (or
|
||||
/// `inkwell::attributes::Context::create_enum_attribute` directly when
|
||||
/// not declaring through the trait) instead.
|
||||
pub fn set_attributes(
|
||||
llvm: &'ctx inkwell::context::Context,
|
||||
declaration: Declaration<'ctx>,
|
||||
@@ -104,8 +110,13 @@ impl<'ctx> Function<'ctx> {
|
||||
force: bool,
|
||||
) {
|
||||
for attribute_kind in attributes {
|
||||
assert_ne!(
|
||||
*attribute_kind,
|
||||
Attribute::Memory,
|
||||
"Attribute::Memory cannot be set through set_attributes; \
|
||||
use RuntimeFunction::MEMORY_EFFECT to express the encoding",
|
||||
);
|
||||
match attribute_kind {
|
||||
Attribute::Memory => unimplemented!("`memory` attributes are not implemented"),
|
||||
attribute_kind @ Attribute::AlwaysInline if force => {
|
||||
declaration.value.remove_enum_attribute(
|
||||
inkwell::attributes::AttributeLoc::Function,
|
||||
@@ -172,16 +183,19 @@ impl<'ctx> Function<'ctx> {
|
||||
);
|
||||
}
|
||||
|
||||
if !optimizer.settings().is_middle_end_enabled() {
|
||||
Self::set_attributes(
|
||||
llvm,
|
||||
declaration,
|
||||
&[Attribute::NoInline, Attribute::OptimizeNone],
|
||||
true,
|
||||
);
|
||||
}
|
||||
|
||||
Self::set_attributes(llvm, declaration, &[Attribute::NoFree], false);
|
||||
// `NoFree` and `NoUnwind` are facts about the PVM target, not optimization heuristics:
|
||||
// * PVM has no `free` (the heap is bump-allocated via `sbrk`), so no Solidity-emitted
|
||||
// function can ever release memory.
|
||||
// * PVM has no stack unwinding: Solidity errors leave a frame via `revert`
|
||||
// (`seal_return(1, ..)` + `unreachable`), never via an unwind edge. Marking every
|
||||
// function `nounwind` lets LLVM elide CFI directives, lower `invoke` to `call`, and
|
||||
// skip exception frame setup.
|
||||
Self::set_attributes(
|
||||
llvm,
|
||||
declaration,
|
||||
&[Attribute::NoFree, Attribute::NoUnwind],
|
||||
false,
|
||||
);
|
||||
}
|
||||
|
||||
/// Sets the front-end runtime attributes.
|
||||
|
||||
@@ -3,16 +3,33 @@
|
||||
use inkwell::values::BasicValue;
|
||||
use revive_common::BIT_LENGTH_WORD;
|
||||
|
||||
use crate::polkavm::context::attribute::Attribute;
|
||||
use crate::polkavm::context::attribute::MemoryEffect;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
use crate::polkavm::context::Context;
|
||||
use crate::polkavm::WriteLLVM;
|
||||
|
||||
/// Standard attribute set for the pure arithmetic helpers. Each helper pairs
|
||||
/// this list with [`PURE_MEMORY_EFFECT`] so that LLVM GVN/CSE can merge
|
||||
/// repeated calls and hoist invariant divisions out of loops.
|
||||
const PURE_ATTRIBUTES: &[Attribute] = &[
|
||||
Attribute::NoFree,
|
||||
Attribute::NoRecurse,
|
||||
Attribute::WillReturn,
|
||||
];
|
||||
|
||||
/// The arithmetic helpers are pure (only depend on their integer arguments).
|
||||
const PURE_MEMORY_EFFECT: MemoryEffect = MemoryEffect::None;
|
||||
|
||||
/// Implements the division operator according to the EVM specification.
|
||||
pub struct Division;
|
||||
|
||||
impl RuntimeFunction for Division {
|
||||
const NAME: &'static str = "__revive_division";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = PURE_ATTRIBUTES;
|
||||
const MEMORY_EFFECT: MemoryEffect = PURE_MEMORY_EFFECT;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.word_type().fn_type(
|
||||
&[context.word_type().into(), context.word_type().into()],
|
||||
@@ -52,6 +69,9 @@ pub struct SignedDivision;
|
||||
impl RuntimeFunction for SignedDivision {
|
||||
const NAME: &'static str = "__revive_signed_division";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = PURE_ATTRIBUTES;
|
||||
const MEMORY_EFFECT: MemoryEffect = PURE_MEMORY_EFFECT;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.word_type().fn_type(
|
||||
&[context.word_type().into(), context.word_type().into()],
|
||||
@@ -126,6 +146,9 @@ pub struct Remainder;
|
||||
impl RuntimeFunction for Remainder {
|
||||
const NAME: &'static str = "__revive_remainder";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = PURE_ATTRIBUTES;
|
||||
const MEMORY_EFFECT: MemoryEffect = PURE_MEMORY_EFFECT;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.word_type().fn_type(
|
||||
&[context.word_type().into(), context.word_type().into()],
|
||||
@@ -174,6 +197,9 @@ pub struct SignedRemainder;
|
||||
impl RuntimeFunction for SignedRemainder {
|
||||
const NAME: &'static str = "__revive_signed_remainder";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = PURE_ATTRIBUTES;
|
||||
const MEMORY_EFFECT: MemoryEffect = PURE_MEMORY_EFFECT;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.word_type().fn_type(
|
||||
&[context.word_type().into(), context.word_type().into()],
|
||||
|
||||
@@ -49,7 +49,7 @@ where
|
||||
context.set_basic_block(context.current_function().borrow().entry_block());
|
||||
|
||||
self.inner.into_llvm(context)?;
|
||||
context.set_debug_location(0, 0, None)?;
|
||||
context.set_debug_location_to_function_scope()?;
|
||||
|
||||
match context
|
||||
.basic_block()
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
use inkwell::values::BasicValue;
|
||||
|
||||
use crate::polkavm::context::attribute::MemoryEffect;
|
||||
use crate::polkavm::context::function::Attribute;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
use crate::polkavm::context::Context;
|
||||
@@ -132,3 +133,386 @@ impl WriteLLVM for Exit {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Outlined `callvalue()` function that reads the value transferred with the call.
|
||||
///
|
||||
/// The `value_transferred` runtime API writes a 256-bit value to a buffer.
|
||||
/// Each inline call site generates: alloca i256 + store 0 + ptrtoint + call + load.
|
||||
/// By outlining into a shared function, all call sites reduce to a single `call`.
|
||||
/// Contracts with many non-payable checks (OpenZeppelin ERC20: 37 sites) save
|
||||
/// significant code size through this deduplication.
|
||||
pub struct CallValue;
|
||||
|
||||
impl RuntimeFunction for CallValue {
|
||||
const NAME: &'static str = "__revive_callvalue";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = &[
|
||||
Attribute::NoFree,
|
||||
Attribute::NoRecurse,
|
||||
Attribute::WillReturn,
|
||||
];
|
||||
|
||||
/// The call value is constant for the duration of a contract invocation,
|
||||
/// so this helper is effectively pure: LLVM can DCE redundant calls and
|
||||
/// CSE duplicate calls.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::None;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.word_type().fn_type(&[], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<inkwell::values::BasicValueEnum<'ctx>>> {
|
||||
let output_pointer =
|
||||
context.build_alloca_at_entry(context.value_type(), "value_transferred");
|
||||
context.build_store(output_pointer, context.word_const(0))?;
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::VALUE_TRANSFERRED,
|
||||
&[output_pointer.to_int(context).into()],
|
||||
);
|
||||
let value = context.build_load(output_pointer, "value_transferred")?;
|
||||
Ok(Some(value))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for CallValue {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Outlined `callvalue() != 0` function that returns a boolean flag.
|
||||
///
|
||||
/// In ERC20 and similar contracts, callvalue is ONLY used for non-payable
|
||||
/// checks: `if (callvalue()) { revert(0, 0) }`. Each such check compares
|
||||
/// a 256-bit value against zero, generating 4 limb comparisons in PVM.
|
||||
/// By outlining the check into a function returning `i1`, each call site
|
||||
/// saves ~15 bytes (4 comparisons + OR vs. a single branch on i1).
|
||||
/// OpenZeppelin ERC20 has ~20 such checks, saving ~300 bytes.
|
||||
pub struct CallValueNonzero;
|
||||
|
||||
impl RuntimeFunction for CallValueNonzero {
|
||||
const NAME: &'static str = "__revive_callvalue_nonzero";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = &[
|
||||
Attribute::NoFree,
|
||||
Attribute::NoRecurse,
|
||||
Attribute::WillReturn,
|
||||
];
|
||||
|
||||
/// Same `memory(none)` rationale as [`CallValue`].
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::None;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.llvm().bool_type().fn_type(&[], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<inkwell::values::BasicValueEnum<'ctx>>> {
|
||||
let output_pointer =
|
||||
context.build_alloca_at_entry(context.value_type(), "value_transferred");
|
||||
context.build_store(output_pointer, context.word_const(0))?;
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::VALUE_TRANSFERRED,
|
||||
&[output_pointer.to_int(context).into()],
|
||||
);
|
||||
let value = context
|
||||
.build_load(output_pointer, "value_transferred")?
|
||||
.into_int_value();
|
||||
let zero = context.word_type().const_zero();
|
||||
let is_nonzero = context.builder().build_int_compare(
|
||||
inkwell::IntPredicate::NE,
|
||||
value,
|
||||
zero,
|
||||
"callvalue_nonzero",
|
||||
)?;
|
||||
Ok(Some(is_nonzero.into()))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for CallValueNonzero {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Outlined `calldataload(offset)` function that reads 32 bytes from call data.
|
||||
///
|
||||
/// The `call_data_load` runtime API writes a 256-bit value to a buffer.
|
||||
/// Each inline call site generates: alloca i256 + ptrtoint + call + load.
|
||||
/// By outlining into a shared function, all call sites reduce to `clip_to_xlen + call`.
|
||||
/// Contracts with many ABI parameters (OpenZeppelin ERC20: 51 sites) save
|
||||
/// significant code size through this deduplication.
|
||||
pub struct CallDataLoad;
|
||||
|
||||
impl RuntimeFunction for CallDataLoad {
|
||||
const NAME: &'static str = "__revive_calldataload";
|
||||
|
||||
/// Calldata is immutable for the duration of execution and only
|
||||
/// reachable through the host syscall. The helper's alloca write is
|
||||
/// invisible to callers, so the externally observable effect is a read
|
||||
/// of pallet-revive runtime state — letting LLVM GVN fold repeated
|
||||
/// `__revive_calldataload(offset)` calls even across heap mstores.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::ReadInaccessible;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context
|
||||
.word_type()
|
||||
.fn_type(&[context.xlen_type().into()], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<inkwell::values::BasicValueEnum<'ctx>>> {
|
||||
let offset = Self::paramater(context, 0).into_int_value();
|
||||
let output_pointer = context.build_alloca_at_entry(context.word_type(), "call_data_output");
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::CALL_DATA_LOAD,
|
||||
&[output_pointer.to_int(context).into(), offset.into()],
|
||||
);
|
||||
let value = context.build_load(output_pointer, "call_data_load_value")?;
|
||||
Ok(Some(value))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for CallDataLoad {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Outlined `caller()` function that reads the calling account address.
|
||||
///
|
||||
/// The `caller` runtime API writes a 20-byte address to a buffer.
|
||||
/// Each inline call site generates: get_global + ptrtoint + call + load + bswap + zext.
|
||||
/// By outlining into a shared function, all call sites reduce to a single `call`.
|
||||
/// Contracts with many msg.sender checks (OpenZeppelin ERC20: 10+ sites) save
|
||||
/// significant code size through this deduplication.
|
||||
pub struct Caller;
|
||||
|
||||
impl RuntimeFunction for Caller {
|
||||
const NAME: &'static str = "__revive_caller";
|
||||
|
||||
/// The caller is immutable within an execution and the syscall reads it
|
||||
/// from pallet-revive runtime state. The body writes the result into a
|
||||
/// function-local alloca (not the shared `@GLOBAL_ADDRESS_SPILL_BUFFER`,
|
||||
/// which `origin()` and `build_address` also touch — see the
|
||||
/// `caller_origin_aliasing` regression test), so from the caller's view
|
||||
/// only pallet-revive runtime state is read and `ReadInaccessible` is
|
||||
/// sound. This lets GVN deduplicate repeated `__revive_caller()` calls
|
||||
/// across heap mstores.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::ReadInaccessible;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.word_type().fn_type(&[], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<inkwell::values::BasicValueEnum<'ctx>>> {
|
||||
let address_type = context.integer_type(revive_common::BIT_LENGTH_ETH_ADDRESS);
|
||||
let output_pointer = context.build_alloca_at_entry(address_type, "caller_output");
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::CALLER,
|
||||
&[output_pointer.to_int(context).into()],
|
||||
);
|
||||
let value = context.build_load_address(output_pointer)?;
|
||||
Ok(Some(value))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for Caller {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Outlined `revert(0, 0)` function (empty revert / panic).
|
||||
///
|
||||
/// This is the most common revert pattern in Solidity contracts (20+ sites in ERC20).
|
||||
/// Each inlined site generates sbrk(0, 0) + ptrtoint + seal_return (~8 PVM instructions).
|
||||
/// By outlining into a zero-argument noreturn function, each call site becomes a single
|
||||
/// `call @__revive_revert_0` (~2 PVM instructions).
|
||||
pub struct RevertEmpty;
|
||||
|
||||
impl RuntimeFunction for RevertEmpty {
|
||||
const NAME: &'static str = "__revive_revert_0";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = &[Attribute::NoReturn, Attribute::NoFree];
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.void_type().fn_type(&[], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<inkwell::values::BasicValueEnum<'ctx>>> {
|
||||
let zero = context.xlen_type().const_zero();
|
||||
// Use unchecked GEP: revert data at offset 0 is always within the
|
||||
// static heap. The sbrk overhead is unnecessary since the data was
|
||||
// already written to memory before this function is called.
|
||||
let heap_pointer = context.build_heap_gep_unchecked(zero)?;
|
||||
let offset_pointer = context.builder().build_ptr_to_int(
|
||||
heap_pointer.value,
|
||||
context.xlen_type(),
|
||||
"return_data_ptr_to_int",
|
||||
)?;
|
||||
let flags = context.integer_const(crate::polkavm::XLEN, 1);
|
||||
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::RETURN,
|
||||
&[flags.into(), offset_pointer.into(), zero.into()],
|
||||
);
|
||||
context.build_unreachable();
|
||||
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for RevertEmpty {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Outlined `revert(0, length)` function for non-zero constant lengths.
|
||||
///
|
||||
/// After error data has been stored to heap memory at offset 0, this function
|
||||
/// calls seal_return(1, heap_base, length). Used for revert(0, 4), revert(0, 36),
|
||||
/// revert(0, 68), revert(0, 100) patterns. ERC20 has 34+ such sites.
|
||||
pub struct Revert;
|
||||
|
||||
impl RuntimeFunction for Revert {
|
||||
const NAME: &'static str = "__revive_revert";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = &[Attribute::NoReturn, Attribute::NoFree];
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context
|
||||
.void_type()
|
||||
.fn_type(&[context.xlen_type().into()], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<inkwell::values::BasicValueEnum<'ctx>>> {
|
||||
let length = Self::paramater(context, 0).into_int_value();
|
||||
|
||||
let zero = context.xlen_type().const_zero();
|
||||
let heap_pointer = context.build_heap_gep(zero, length)?;
|
||||
let offset_pointer = context.builder().build_ptr_to_int(
|
||||
heap_pointer.value,
|
||||
context.xlen_type(),
|
||||
"return_data_ptr_to_int",
|
||||
)?;
|
||||
let flags = context.integer_const(crate::polkavm::XLEN, 1);
|
||||
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::RETURN,
|
||||
&[flags.into(), offset_pointer.into(), length.into()],
|
||||
);
|
||||
context.build_unreachable();
|
||||
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for Revert {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Outlined `Panic(uint256)` revert helper: writes the Panic ABI encoding to
|
||||
/// scratch memory (selector at offset 0, code at offset 4) and reverts with
|
||||
/// length 36. The code is passed as an `i256` argument so a single helper
|
||||
/// covers every panic code (0x01, 0x11, 0x12, 0x21, 0x22, 0x31, 0x32, 0x41).
|
||||
/// Replaces the inline two-mstore + revert pattern at every panic site, which
|
||||
/// otherwise duplicates ~30+ instructions across every function that traps.
|
||||
pub struct RevertPanic;
|
||||
|
||||
impl RuntimeFunction for RevertPanic {
|
||||
const NAME: &'static str = "__revive_panic";
|
||||
|
||||
const ATTRIBUTES: &'static [Attribute] = &[Attribute::NoReturn, Attribute::NoFree];
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context
|
||||
.void_type()
|
||||
.fn_type(&[context.word_type().into()], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<inkwell::values::BasicValueEnum<'ctx>>> {
|
||||
let code = Self::paramater(context, 0).into_int_value();
|
||||
|
||||
let zero_xlen = context.xlen_type().const_zero();
|
||||
let four_xlen = context.xlen_type().const_int(4, false);
|
||||
let panic_selector =
|
||||
context.word_const_str_hex(revive_common::PANIC_UINT256_SELECTOR_WORD_HEX);
|
||||
|
||||
crate::polkavm::evm::memory::store_bswap_unchecked(context, zero_xlen, panic_selector)?;
|
||||
crate::polkavm::evm::memory::store_bswap_unchecked(context, four_xlen, code)?;
|
||||
|
||||
let length = context.xlen_type().const_int(0x24, false);
|
||||
let heap_pointer = context.build_heap_gep_unchecked(zero_xlen)?;
|
||||
let offset_pointer = context.builder().build_ptr_to_int(
|
||||
heap_pointer.value,
|
||||
context.xlen_type(),
|
||||
"panic_data_ptr_to_int",
|
||||
)?;
|
||||
let flags = context.integer_const(crate::polkavm::XLEN, 1);
|
||||
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::RETURN,
|
||||
&[flags.into(), offset_pointer.into(), length.into()],
|
||||
);
|
||||
context.build_unreachable();
|
||||
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for RevertPanic {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
@@ -49,7 +49,7 @@ where
|
||||
context.set_basic_block(context.current_function().borrow().entry_block());
|
||||
|
||||
self.inner.into_llvm(context)?;
|
||||
context.set_debug_location(0, 0, None)?;
|
||||
context.set_debug_location_to_function_scope()?;
|
||||
|
||||
match context
|
||||
.basic_block()
|
||||
|
||||
@@ -10,6 +10,7 @@ use inkwell::debug_info::AsDIScope;
|
||||
use inkwell::debug_info::DIScope;
|
||||
use inkwell::types::BasicType;
|
||||
use inkwell::values::BasicValue;
|
||||
use inkwell::values::InstructionOpcode;
|
||||
use revive_solc_json_interface::PolkaVMDefaultHeapMemorySize;
|
||||
use revive_solc_json_interface::PolkaVMDefaultStackMemorySize;
|
||||
use revive_solc_json_interface::SolcStandardJsonInputSettingsPolkaVMMemory;
|
||||
@@ -20,11 +21,14 @@ use crate::polkavm::DebugConfig;
|
||||
use crate::target_machine::target::Target;
|
||||
use crate::target_machine::TargetMachine;
|
||||
use crate::PolkaVMLoadHeapWordFunction;
|
||||
use crate::PolkaVMLoadHeapWordNativeFunction;
|
||||
use crate::PolkaVMSbrkFunction;
|
||||
use crate::PolkaVMStoreHeapWordFunction;
|
||||
use crate::PolkaVMStoreHeapWordNativeFunction;
|
||||
|
||||
use self::address_space::AddressSpace;
|
||||
use self::attribute::Attribute;
|
||||
use self::attribute::MemoryEffect;
|
||||
use self::build::Build;
|
||||
use self::code_type::CodeType;
|
||||
use self::debug_info::DebugInfo;
|
||||
@@ -59,6 +63,29 @@ pub mod yul_data;
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
|
||||
/// Imported host syscalls whose only externally visible effect is a read of
|
||||
/// pallet-revive runtime state. Each takes no arguments and returns a scalar —
|
||||
/// no input/output buffer and no heap traffic from the caller's view — so we
|
||||
/// can attach [`MemoryEffect::ReadInaccessible`] and let LLVM GVN dedupe
|
||||
/// repeated calls across heap mstores.
|
||||
///
|
||||
/// `code_size` is deliberately **not** in this list. Unlike the entries above,
|
||||
/// `extcodesize` passes the queried address to the syscall through the
|
||||
/// [`crate::polkavm::GLOBAL_ADDRESS_SPILL_BUFFER`] global (see
|
||||
/// [`Context::build_address_argument_store`]), so the call reads caller-accessible
|
||||
/// memory. That access lives in LLVM's "other" location, not "inaccessiblemem"
|
||||
/// (the address reaches the call as an integer, so it is not "argmem" either).
|
||||
/// Tagging it `inaccessiblemem: read` would hide the buffer read, letting GVN
|
||||
/// merge two `extcodesize` calls for different addresses — and DSE drop the
|
||||
/// first address store — so that `extcodesize(a) + extcodesize(b)` miscompiles
|
||||
/// to `2 * extcodesize(a)`. With no attribute the import keeps the inline-asm
|
||||
/// `~{memory}` barrier, matching its pre-newyork behaviour.
|
||||
const MEMORY_READ_IMPORTS: &[&str] = &[
|
||||
revive_runtime_api::polkavm_imports::CALL_DATA_SIZE,
|
||||
revive_runtime_api::polkavm_imports::GAS_LIMIT,
|
||||
revive_runtime_api::polkavm_imports::RETURNDATASIZE,
|
||||
];
|
||||
|
||||
/// The LLVM IR generator context.
|
||||
/// It is a not-so-big god-like object glueing all the compilers' complexity and act as an adapter
|
||||
/// and a superstructure over the inner `inkwell` LLVM context.
|
||||
@@ -123,8 +150,10 @@ impl<'ctx> Context<'ctx> {
|
||||
.expect("the stdlib module should be linkable");
|
||||
}
|
||||
|
||||
/// Link in the PolkaVM imports module, containing imported functions,
|
||||
/// and marking them as external (they need to be relocatable as too).
|
||||
/// Link in the PolkaVM imports module, mark host syscall imports as
|
||||
/// `Internal` linkage (they need to be relocatable too), and attach a
|
||||
/// [`MemoryEffect::ReadInaccessible`] attribute to every entry of
|
||||
/// [`MEMORY_READ_IMPORTS`].
|
||||
fn link_polkavm_imports(
|
||||
llvm: &'ctx inkwell::context::Context,
|
||||
module: &inkwell::module::Module<'ctx>,
|
||||
@@ -141,6 +170,20 @@ impl<'ctx> Context<'ctx> {
|
||||
.unwrap_or_else(|| panic!("{import} import should be declared"))
|
||||
.set_linkage(inkwell::module::Linkage::Internal);
|
||||
}
|
||||
|
||||
let read_inaccessible_encoding = MemoryEffect::ReadInaccessible
|
||||
.encoding()
|
||||
.expect("ICE: ReadInaccessible always encodes to an integer");
|
||||
let memory_read_attr =
|
||||
llvm.create_enum_attribute(Attribute::Memory as u32, read_inaccessible_encoding);
|
||||
for import in MEMORY_READ_IMPORTS {
|
||||
if let Some(function) = module.get_function(import) {
|
||||
function.add_attribute(
|
||||
inkwell::attributes::AttributeLoc::Function,
|
||||
memory_read_attr,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn link_polkavm_exports(&self, contract_path: &str) -> anyhow::Result<()> {
|
||||
@@ -287,7 +330,11 @@ impl<'ctx> Context<'ctx> {
|
||||
) -> anyhow::Result<Build> {
|
||||
self.link_polkavm_exports(contract_path)?;
|
||||
self.link_immutable_data(contract_path)?;
|
||||
let target_machine = TargetMachine::new(Target::PVM, self.optimizer.settings())?;
|
||||
let target_machine = TargetMachine::new(
|
||||
Target::PVM,
|
||||
self.optimizer.settings(),
|
||||
self.optimizer.is_newyork(),
|
||||
)?;
|
||||
self.module().set_triple(&target_machine.get_triple());
|
||||
|
||||
self.debug_config
|
||||
@@ -311,6 +358,12 @@ impl<'ctx> Context<'ctx> {
|
||||
)
|
||||
})?;
|
||||
|
||||
// Narrow large integer div/rem where operands provably fit in a smaller
|
||||
// type. LLVM's i256 div/rem backend expansion produces enormous basic
|
||||
// blocks that pallet-revive rejects as `BasicBlockTooLarge`; narrowing
|
||||
// here keeps the expansion compact.
|
||||
self.narrow_divrem_instructions();
|
||||
|
||||
self.debug_config
|
||||
.dump_llvm_ir_optimized(contract_path, self.module())?;
|
||||
|
||||
@@ -441,12 +494,35 @@ impl<'ctx> Context<'ctx> {
|
||||
&self.llvm_runtime
|
||||
}
|
||||
|
||||
/// Returns the topmost frontend function-name → mangled-name scope.
|
||||
///
|
||||
/// Panics if no scope has been pushed; callers must invoke this from inside a region
|
||||
/// that has already set up its function scope.
|
||||
pub fn get_current_scope(&mut self) -> &mut BTreeMap<String, String> {
|
||||
self.function_scope
|
||||
.last_mut()
|
||||
.expect("ICE: function scope must be pushed before declaring frontend functions")
|
||||
}
|
||||
|
||||
/// Returns the mangled symbol name for a frontend (Solidity-emitted) function.
|
||||
///
|
||||
/// Frontend names collide across compilation units: separate Yul objects can both define
|
||||
/// `fun_main` for example. The mangling appends a code-section tag (deploy / runtime) and a
|
||||
/// per-context counter so each definition gets a globally unique LLVM symbol, while the
|
||||
/// original name remains the lookup key in `function_scope`.
|
||||
fn mangle_frontend_function_name(&mut self, name: &str) -> String {
|
||||
assert!(
|
||||
!self.get_current_scope().contains_key(name),
|
||||
"ICE: function '{name}' declared subsequently in the same scope"
|
||||
);
|
||||
let counter = self.function_counter;
|
||||
self.function_counter += 1;
|
||||
let mangled = format!("{name}_{}__{counter}", self.code_type().unwrap());
|
||||
self.get_current_scope()
|
||||
.insert(name.to_string(), mangled.clone());
|
||||
mangled
|
||||
}
|
||||
|
||||
/// Appends a function to the current module.
|
||||
pub fn add_function(
|
||||
&mut self,
|
||||
@@ -458,16 +534,7 @@ impl<'ctx> Context<'ctx> {
|
||||
is_frontend: bool,
|
||||
) -> anyhow::Result<Rc<RefCell<Function<'ctx>>>> {
|
||||
let name = if is_frontend {
|
||||
assert!(
|
||||
!self.get_current_scope().contains_key(name),
|
||||
"ICE: function '{name}' declared subsequently in the same scope"
|
||||
);
|
||||
let counter = self.function_counter;
|
||||
self.function_counter += 1;
|
||||
let mangled = format!("{name}_{}__{counter}", self.code_type().unwrap());
|
||||
self.get_current_scope()
|
||||
.insert(name.to_string(), mangled.clone());
|
||||
mangled
|
||||
self.mangle_frontend_function_name(name)
|
||||
} else {
|
||||
name.to_string()
|
||||
};
|
||||
@@ -492,21 +559,28 @@ impl<'ctx> Context<'ctx> {
|
||||
let entry_block = self.llvm.append_basic_block(value, "entry");
|
||||
let return_block = self.llvm.append_basic_block(value, "return");
|
||||
|
||||
// Allocate return slots using the actual return type from the function signature.
|
||||
// Narrowed return types (e.g. i64 instead of i256) flow through unchanged, reducing
|
||||
// register pressure and spills.
|
||||
let r#return = match return_values_length {
|
||||
0 => FunctionReturn::none(),
|
||||
1 => {
|
||||
self.set_basic_block(entry_block);
|
||||
let pointer = self.build_alloca(self.word_type(), "return_pointer");
|
||||
let alloca_type = r#type
|
||||
.get_return_type()
|
||||
.unwrap_or_else(|| self.word_type().as_basic_type_enum());
|
||||
let pointer = self.build_alloca(alloca_type, "return_pointer");
|
||||
FunctionReturn::primitive(pointer)
|
||||
}
|
||||
size => {
|
||||
self.set_basic_block(entry_block);
|
||||
let pointer = self.build_alloca(
|
||||
let alloca_type = r#type.get_return_type().unwrap_or_else(|| {
|
||||
self.structure_type(
|
||||
vec![self.word_type().as_basic_type_enum(); size].as_slice(),
|
||||
),
|
||||
"return_pointer",
|
||||
);
|
||||
)
|
||||
.as_basic_type_enum()
|
||||
});
|
||||
let pointer = self.build_alloca(alloca_type, "return_pointer");
|
||||
FunctionReturn::compound(pointer, size)
|
||||
}
|
||||
};
|
||||
@@ -633,6 +707,20 @@ impl<'ctx> Context<'ctx> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Anchors the current debug location to the active function's own subprogram.
|
||||
///
|
||||
/// Synthetic trailing instructions (the implicit branch to the return block
|
||||
/// and the terminator) are emitted after the function body has been lowered,
|
||||
/// at which point the debug-info scope stack may still point at a nested or
|
||||
/// helper function generated along the way. Resolving `None` to the top scope
|
||||
/// there would attach a `!dbg` from an unrelated subprogram, which the LLVM
|
||||
/// verifier rejects. Using the function's own scope keeps the attachment valid.
|
||||
/// No-op when debug info is disabled.
|
||||
pub fn set_debug_location_to_function_scope(&self) -> anyhow::Result<()> {
|
||||
let scope = self.current_function().borrow().get_debug_scope();
|
||||
self.set_debug_location(0, 0, scope)
|
||||
}
|
||||
|
||||
/// Pushes a debug-info scope to the stack.
|
||||
pub fn push_debug_scope(&self, scope: DIScope<'ctx>) {
|
||||
if let Some(debug_info) = self.debug_info() {
|
||||
@@ -709,28 +797,35 @@ impl<'ctx> Context<'ctx> {
|
||||
}
|
||||
|
||||
/// Builds an aligned stack allocation at the function entry.
|
||||
///
|
||||
/// Allocas placed at the entry block coalesce into a single stack-frame allocation,
|
||||
/// eliminating per-alloca dynamic stack adjustment. LLVM's stack coloring pass handles
|
||||
/// lifetime-based slot reuse regardless of alloca placement.
|
||||
pub fn build_alloca_at_entry<T: BasicType<'ctx> + Clone + Copy>(
|
||||
&self,
|
||||
r#type: T,
|
||||
name: &str,
|
||||
) -> Pointer<'ctx> {
|
||||
// TODO: Revisit. While at entry should be preferred in theory:
|
||||
// - It has negligible code size impact on real word contracts.
|
||||
// - Sometimes has negative impact on code size.
|
||||
// - Messes up debug information used to analyze code size issues.
|
||||
self.build_alloca(r#type, name)
|
||||
let current_block = self.builder.get_insert_block().unwrap();
|
||||
let function = current_block.get_parent().unwrap();
|
||||
let entry_block = function.get_first_basic_block().unwrap();
|
||||
|
||||
// let current_block = self.basic_block();
|
||||
// let entry_block = self.current_function().borrow().entry_block();
|
||||
if let Some(terminator) = entry_block.get_terminator() {
|
||||
self.builder.position_before(&terminator);
|
||||
} else {
|
||||
self.builder.position_at_end(entry_block);
|
||||
}
|
||||
|
||||
// match entry_block.get_first_instruction() {
|
||||
// Some(instruction) => self.builder().position_before(&instruction),
|
||||
// None => self.builder().position_at_end(entry_block),
|
||||
// }
|
||||
let pointer = self.builder.build_alloca(r#type, name).unwrap();
|
||||
pointer
|
||||
.as_instruction()
|
||||
.unwrap()
|
||||
.set_alignment(revive_common::BYTE_LENGTH_STACK_ALIGN as u32)
|
||||
.expect("ICE: alignment is valid");
|
||||
|
||||
// let pointer = self.build_alloca(r#type, name);
|
||||
// self.set_basic_block(current_block);
|
||||
// pointer
|
||||
self.builder.position_at_end(current_block);
|
||||
|
||||
Pointer::new(r#type, AddressSpace::Stack, pointer)
|
||||
}
|
||||
|
||||
/// Builds an aligned stack allocation at the current position.
|
||||
@@ -848,6 +943,34 @@ impl<'ctx> Context<'ctx> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Builds a heap load instruction without byte-swapping.
|
||||
/// Used for internal memory operations that don't escape to external code.
|
||||
pub fn build_load_native(
|
||||
&self,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
let name = <PolkaVMLoadHeapWordNativeFunction as RuntimeFunction>::NAME;
|
||||
let declaration = <PolkaVMLoadHeapWordNativeFunction as RuntimeFunction>::declaration(self);
|
||||
let arguments = [offset.as_basic_value_enum()];
|
||||
Ok(self
|
||||
.build_call(declaration, &arguments, "heap_load_native")
|
||||
.unwrap_or_else(|| panic!("revive runtime function {name} should return a value")))
|
||||
}
|
||||
|
||||
/// Builds a heap store instruction without byte-swapping.
|
||||
/// Used for internal memory operations that don't escape to external code.
|
||||
pub fn build_store_native(
|
||||
&self,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
value: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<()> {
|
||||
let declaration =
|
||||
<PolkaVMStoreHeapWordNativeFunction as RuntimeFunction>::declaration(self);
|
||||
let arguments = [offset.as_basic_value_enum(), value.as_basic_value_enum()];
|
||||
self.build_call(declaration, &arguments, "heap_store_native");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Swap the endianness of an intvalue
|
||||
pub fn build_byte_swap(
|
||||
&self,
|
||||
@@ -1053,45 +1176,111 @@ impl<'ctx> Context<'ctx> {
|
||||
}
|
||||
|
||||
/// Truncate a memory offset to register size, trapping if it doesn't fit.
|
||||
///
|
||||
/// Handles all integer widths: at xlen the value is returned unchanged, narrower types are
|
||||
/// zero-extended (no overflow possible), word-typed values delegate to the
|
||||
/// `WordToPointer` runtime function (which returns the truncated value or jumps to invalid),
|
||||
/// and intermediate widths inline a truncate-extend-compare at the source width because that
|
||||
/// is much cheaper on 32-bit PVM than extending to i256 first (i64 compare is 2 register
|
||||
/// pairs vs i256's 8 words — roughly 10 fewer instructions per check).
|
||||
pub fn safe_truncate_int_to_xlen(
|
||||
&self,
|
||||
value: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::IntValue<'ctx>> {
|
||||
if value.get_type() == self.xlen_type() {
|
||||
let value_width = value.get_type().get_bit_width();
|
||||
let xlen_width = self.xlen_type().get_bit_width();
|
||||
let word_width = self.word_type().get_bit_width();
|
||||
|
||||
if value_width == xlen_width {
|
||||
return Ok(value);
|
||||
}
|
||||
assert_eq!(
|
||||
value.get_type(),
|
||||
self.word_type(),
|
||||
"expected XLEN or WORD sized int type for memory offset",
|
||||
);
|
||||
|
||||
Ok(self
|
||||
.build_call(
|
||||
<WordToPointer as RuntimeFunction>::declaration(self),
|
||||
&[value.into()],
|
||||
"word_to_pointer",
|
||||
)
|
||||
.unwrap_or_else(|| {
|
||||
panic!(
|
||||
"revive runtime function {} should return a value",
|
||||
<WordToPointer as RuntimeFunction>::NAME,
|
||||
if value_width < xlen_width {
|
||||
return Ok(self.builder().build_int_z_extend(
|
||||
value,
|
||||
self.xlen_type(),
|
||||
"narrow_to_xlen",
|
||||
)?);
|
||||
}
|
||||
|
||||
if value_width == word_width {
|
||||
return Ok(self
|
||||
.build_call(
|
||||
<WordToPointer as RuntimeFunction>::declaration(self),
|
||||
&[value.into()],
|
||||
"word_to_pointer",
|
||||
)
|
||||
})
|
||||
.into_int_value())
|
||||
.unwrap_or_else(|| {
|
||||
panic!(
|
||||
"ICE: revive runtime function {} should return a value",
|
||||
<WordToPointer as RuntimeFunction>::NAME,
|
||||
)
|
||||
})
|
||||
.into_int_value());
|
||||
}
|
||||
|
||||
let truncated =
|
||||
self.builder()
|
||||
.build_int_truncate(value, self.xlen_type(), "offset_truncated")?;
|
||||
let extended =
|
||||
self.builder()
|
||||
.build_int_z_extend(truncated, value.get_type(), "offset_extended")?;
|
||||
let is_overflow = self.builder().build_int_compare(
|
||||
inkwell::IntPredicate::NE,
|
||||
value,
|
||||
extended,
|
||||
"compare_truncated_extended",
|
||||
)?;
|
||||
|
||||
let block_continue = self.append_basic_block("offset_pointer_ok");
|
||||
let block_invalid = self.append_basic_block("offset_pointer_overflow");
|
||||
self.build_conditional_branch(is_overflow, block_invalid, block_continue)?;
|
||||
|
||||
self.set_basic_block(block_invalid);
|
||||
self.build_runtime_call(revive_runtime_api::polkavm_imports::INVALID, &[]);
|
||||
self.build_unreachable();
|
||||
|
||||
self.set_basic_block(block_continue);
|
||||
Ok(truncated)
|
||||
}
|
||||
|
||||
/// Clip a memory offset to the maximum value that fits into a register.
|
||||
///
|
||||
/// Behaves like [`Self::safe_truncate_int_to_xlen`] up to xlen-and-narrower widths, but for
|
||||
/// wider types it saturates to `xlen::MAX` rather than trapping. The wider-type branch
|
||||
/// keeps the original-width comparison so the codegen stays compact (word-type values
|
||||
/// remain at word_type; intermediate widths use the value's own type).
|
||||
pub fn clip_to_xlen(
|
||||
&self,
|
||||
value: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::IntValue<'ctx>> {
|
||||
let value_width = value.get_type().get_bit_width();
|
||||
let xlen_width = self.xlen_type().get_bit_width();
|
||||
let word_width = self.word_type().get_bit_width();
|
||||
|
||||
if value_width == xlen_width {
|
||||
return Ok(value);
|
||||
}
|
||||
|
||||
if value_width < xlen_width {
|
||||
return Ok(self.builder().build_int_z_extend(
|
||||
value,
|
||||
self.xlen_type(),
|
||||
"narrow_to_xlen",
|
||||
)?);
|
||||
}
|
||||
|
||||
let clipped = self.xlen_type().const_all_ones();
|
||||
let comparison_type = if value_width == word_width {
|
||||
self.word_type()
|
||||
} else {
|
||||
value.get_type()
|
||||
};
|
||||
let is_overflow = self.builder().build_int_compare(
|
||||
inkwell::IntPredicate::UGT,
|
||||
value,
|
||||
self.builder()
|
||||
.build_int_z_extend(clipped, self.word_type(), "value_clipped")?,
|
||||
.build_int_z_extend(clipped, comparison_type, "value_clipped")?,
|
||||
"is_value_overflow",
|
||||
)?;
|
||||
let truncated =
|
||||
@@ -1158,6 +1347,64 @@ impl<'ctx> Context<'ctx> {
|
||||
Ok(Pointer::new(self.byte_type(), AddressSpace::Stack, pointer))
|
||||
}
|
||||
|
||||
/// Returns a pointer to `offset` into the heap WITHOUT calling sbrk.
|
||||
///
|
||||
/// The heap global is a fixed-size byte array (the static heap) that is always live for the
|
||||
/// full duration of contract execution. sbrk only updates the *high-water mark* tracked in
|
||||
/// `GLOBAL_HEAP_SIZE`; the underlying memory backing offsets up to `HEAP_SIZE` is always
|
||||
/// addressable. Skipping sbrk is therefore safe whenever the *byte-order semantics* of the
|
||||
/// access do not depend on a previously published high-water mark — i.e., for offsets known
|
||||
/// to live inside the statically reserved scratch area (0x00-0x7f, including the free
|
||||
/// memory pointer slot at 0x40), and for compiler-internal native-mode I/O whose offset
|
||||
/// bound is proven by the type system.
|
||||
///
|
||||
/// # Panics
|
||||
/// Assumes `offset` to be a register sized value.
|
||||
pub fn build_heap_gep_unchecked(
|
||||
&self,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<Pointer<'ctx>> {
|
||||
assert_eq!(offset.get_type(), self.xlen_type());
|
||||
let heap_global: Pointer<'ctx> =
|
||||
self.get_global(crate::polkavm::GLOBAL_HEAP_MEMORY)?.into();
|
||||
let pointer = self.build_gep(
|
||||
heap_global,
|
||||
&[self.xlen_type().const_zero(), offset],
|
||||
self.byte_type(),
|
||||
"heap_unchecked_ptr",
|
||||
);
|
||||
Ok(Pointer::new(
|
||||
self.byte_type(),
|
||||
AddressSpace::Stack,
|
||||
pointer.value,
|
||||
))
|
||||
}
|
||||
|
||||
/// Ensures the heap size (msize) is at least `min_size`.
|
||||
/// This emits a branchless max(current_msize, min_size) update.
|
||||
/// Used after native stores that bypass sbrk to keep msize consistent.
|
||||
pub fn ensure_heap_size(
|
||||
&self,
|
||||
min_size: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<()> {
|
||||
let current = self
|
||||
.get_global_value(crate::polkavm::GLOBAL_HEAP_SIZE)?
|
||||
.into_int_value();
|
||||
let needs_update = self.builder().build_int_compare(
|
||||
inkwell::IntPredicate::UGT,
|
||||
min_size,
|
||||
current,
|
||||
"msize_needs_update",
|
||||
)?;
|
||||
let new_size = self
|
||||
.builder()
|
||||
.build_select(needs_update, min_size, current, "msize_new")?
|
||||
.into_int_value();
|
||||
let heap_size_global = self.get_global(crate::polkavm::GLOBAL_HEAP_SIZE)?;
|
||||
self.build_store(heap_size_global.into(), new_size)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Returns a boolean type constant.
|
||||
pub fn bool_const(&self, value: bool) -> inkwell::values::IntValue<'ctx> {
|
||||
self.bool_type().const_int(u64::from(value), false)
|
||||
@@ -1292,6 +1539,8 @@ impl<'ctx> Context<'ctx> {
|
||||
}
|
||||
|
||||
/// Returns a Yul function type with the specified arguments and number of return values.
|
||||
/// All return values use word_type (i256). For narrowed return types, use
|
||||
/// `function_type_with_returns`.
|
||||
pub fn function_type<T>(
|
||||
&self,
|
||||
argument_types: Vec<T>,
|
||||
@@ -1318,6 +1567,39 @@ impl<'ctx> Context<'ctx> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns a function type with explicit return types instead of word_type.
|
||||
/// Used by the newyork codegen for functions with narrowed return types.
|
||||
pub fn function_type_with_returns<T>(
|
||||
&self,
|
||||
argument_types: Vec<T>,
|
||||
return_types: &[inkwell::types::IntType<'ctx>],
|
||||
) -> inkwell::types::FunctionType<'ctx>
|
||||
where
|
||||
T: BasicType<'ctx>,
|
||||
{
|
||||
let argument_types: Vec<inkwell::types::BasicMetadataTypeEnum> = argument_types
|
||||
.as_slice()
|
||||
.iter()
|
||||
.map(T::as_basic_type_enum)
|
||||
.map(inkwell::types::BasicMetadataTypeEnum::from)
|
||||
.collect();
|
||||
match return_types.len() {
|
||||
0 => self
|
||||
.llvm
|
||||
.void_type()
|
||||
.fn_type(argument_types.as_slice(), false),
|
||||
1 => return_types[0].fn_type(argument_types.as_slice(), false),
|
||||
_ => {
|
||||
let field_types: Vec<inkwell::types::BasicTypeEnum> = return_types
|
||||
.iter()
|
||||
.map(|t| t.as_basic_type_enum())
|
||||
.collect();
|
||||
self.structure_type(&field_types)
|
||||
.fn_type(argument_types.as_slice(), false)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Modifies the call site value, setting the default attributes.
|
||||
/// The attributes only affect the LLVM optimizations.
|
||||
pub fn modify_call_site_value(
|
||||
@@ -1468,6 +1750,12 @@ impl<'ctx> Context<'ctx> {
|
||||
self.optimizer.settings()
|
||||
}
|
||||
|
||||
/// Whether this module is being compiled through the newyork IR generator.
|
||||
/// Gates newyork-specific codegen that is unsound on the stock Yul path.
|
||||
pub fn is_newyork(&self) -> bool {
|
||||
self.optimizer.is_newyork()
|
||||
}
|
||||
|
||||
pub fn heap_size(&self) -> inkwell::values::IntValue<'ctx> {
|
||||
self.xlen_type().const_int(
|
||||
self.memory_config
|
||||
@@ -1476,4 +1764,214 @@ impl<'ctx> Context<'ctx> {
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
/// Narrows large integer div/rem instructions whose operands provably fit
|
||||
/// in a smaller type.
|
||||
///
|
||||
/// LLVM's i256 div/rem backend expansion produces enormous basic blocks
|
||||
/// that pallet-revive rejects as `BasicBlockTooLarge`. LLVM's own
|
||||
/// `DivRemNarrowing` pass also sometimes fails to narrow div/rem in large
|
||||
/// functions after inlining. This runs as a safety net after the full
|
||||
/// LLVM optimization pipeline so we operate on the final form where the
|
||||
/// proof sources we accept (LLVM constants, `and` masks, `zext`) are
|
||||
/// observable across function boundaries after IPSCCP and inlining.
|
||||
fn narrow_divrem_instructions(&self) {
|
||||
let builder = self.llvm.create_builder();
|
||||
|
||||
for function in self.module().get_functions() {
|
||||
let mut to_narrow = Vec::new();
|
||||
|
||||
for basic_block in function.get_basic_blocks() {
|
||||
for instruction in basic_block.get_instructions() {
|
||||
let is_divrem = matches!(
|
||||
instruction.get_opcode(),
|
||||
InstructionOpcode::UDiv
|
||||
| InstructionOpcode::SDiv
|
||||
| InstructionOpcode::URem
|
||||
| InstructionOpcode::SRem
|
||||
);
|
||||
if !is_divrem {
|
||||
continue;
|
||||
}
|
||||
if instruction.get_type().into_int_type().get_bit_width() < 256 {
|
||||
continue;
|
||||
}
|
||||
|
||||
let lhs = instruction.get_operand(0).and_then(|op| op.value());
|
||||
let rhs = instruction.get_operand(1).and_then(|op| op.value());
|
||||
|
||||
if let (Some(lhs), Some(rhs)) = (lhs, rhs) {
|
||||
let lhs_width = Self::provable_bit_width(lhs);
|
||||
let rhs_width = Self::provable_bit_width(rhs);
|
||||
|
||||
if let (Some(lw), Some(rw)) = (lhs_width, rhs_width) {
|
||||
let max_operand_width = lw.max(rw);
|
||||
let narrow_width = Self::round_up_bit_width(max_operand_width);
|
||||
// Signed div/rem narrowing is sound only when bit (narrow_width - 1) of every operand is provably zero — otherwise the
|
||||
// narrow signed division treats a non-negative i256 operand as negative and the sign-extended result diverges from the
|
||||
// i256 sdiv. The proof sources accepted here (constants, AND-masks, ZExt) all pin the upper bits to zero strictly above
|
||||
// the operand width, so this holds iff the operand fits with at least one bit of headroom in the narrow type.
|
||||
let is_signed = matches!(
|
||||
instruction.get_opcode(),
|
||||
InstructionOpcode::SDiv | InstructionOpcode::SRem
|
||||
);
|
||||
let signed_sign_bit_safe =
|
||||
!is_signed || narrow_width > max_operand_width;
|
||||
if narrow_width < 256 && signed_sign_bit_safe {
|
||||
to_narrow.push((instruction, narrow_width));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (instruction, narrow_width) in to_narrow {
|
||||
let lhs = instruction
|
||||
.get_operand(0)
|
||||
.unwrap()
|
||||
.value()
|
||||
.unwrap()
|
||||
.into_int_value();
|
||||
let rhs = instruction
|
||||
.get_operand(1)
|
||||
.unwrap()
|
||||
.value()
|
||||
.unwrap()
|
||||
.into_int_value();
|
||||
let wide_type = instruction.get_type().into_int_type();
|
||||
let narrow_type = self
|
||||
.llvm
|
||||
.custom_width_int_type(
|
||||
std::num::NonZeroU32::new(narrow_width).expect("narrow width is non-zero"),
|
||||
)
|
||||
.expect("valid integer width");
|
||||
|
||||
builder.position_before(&instruction);
|
||||
|
||||
let lhs_truncated = builder.build_int_truncate(lhs, narrow_type, "").unwrap();
|
||||
let rhs_truncated = builder.build_int_truncate(rhs, narrow_type, "").unwrap();
|
||||
|
||||
let narrow_result = match instruction.get_opcode() {
|
||||
InstructionOpcode::UDiv => builder
|
||||
.build_int_unsigned_div(lhs_truncated, rhs_truncated, "")
|
||||
.unwrap(),
|
||||
InstructionOpcode::SDiv => builder
|
||||
.build_int_signed_div(lhs_truncated, rhs_truncated, "")
|
||||
.unwrap(),
|
||||
InstructionOpcode::URem => builder
|
||||
.build_int_unsigned_rem(lhs_truncated, rhs_truncated, "")
|
||||
.unwrap(),
|
||||
InstructionOpcode::SRem => builder
|
||||
.build_int_signed_rem(lhs_truncated, rhs_truncated, "")
|
||||
.unwrap(),
|
||||
_ => unreachable!(),
|
||||
};
|
||||
|
||||
let wide_result = if matches!(
|
||||
instruction.get_opcode(),
|
||||
InstructionOpcode::SDiv | InstructionOpcode::SRem
|
||||
) {
|
||||
builder
|
||||
.build_int_s_extend(narrow_result, wide_type, "")
|
||||
.unwrap()
|
||||
} else {
|
||||
builder
|
||||
.build_int_z_extend(narrow_result, wide_type, "")
|
||||
.unwrap()
|
||||
};
|
||||
|
||||
let wide_instruction = wide_result.as_instruction().unwrap();
|
||||
instruction.replace_all_uses_with(&wide_instruction);
|
||||
instruction.erase_from_basic_block();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns the provable bit width of a value, if it can be determined.
|
||||
///
|
||||
/// Checks for:
|
||||
/// - Constants: bit width needed to represent the value.
|
||||
/// - `and %x, mask`: bit width of the mask.
|
||||
/// - `zext from smaller_type`: bit width of the source type.
|
||||
/// - `trunc to smaller_type`: bit width of the result type.
|
||||
fn provable_bit_width(value: inkwell::values::BasicValueEnum) -> Option<u32> {
|
||||
let integer_value = value.into_int_value();
|
||||
if integer_value.is_const() {
|
||||
return Self::constant_bit_width(integer_value);
|
||||
}
|
||||
|
||||
let instruction = integer_value.as_instruction()?;
|
||||
match instruction.get_opcode() {
|
||||
InstructionOpcode::And => {
|
||||
let operand_0 = instruction.get_operand(0)?.value()?.into_int_value();
|
||||
let operand_1 = instruction.get_operand(1)?.value()?.into_int_value();
|
||||
if operand_1.is_const() {
|
||||
Self::constant_bit_width(operand_1)
|
||||
} else if operand_0.is_const() {
|
||||
Self::constant_bit_width(operand_0)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
InstructionOpcode::ZExt => {
|
||||
let source = instruction.get_operand(0)?.value()?.into_int_value();
|
||||
Some(source.get_type().get_bit_width())
|
||||
}
|
||||
InstructionOpcode::Trunc => {
|
||||
Some(instruction.get_type().into_int_type().get_bit_width())
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns the minimum number of bits needed to represent a constant integer.
|
||||
///
|
||||
/// For types up to 64 bits the direct LLVM API suffices. For wider types (e.g. i256) we
|
||||
/// truncate to i64 and verify the round-trip: if the reconstructed wide constant equals the
|
||||
/// original (LLVM constants are interned, so pointer equality is correct), the value fits
|
||||
/// in u64 and we report `64 - leading_zeros`.
|
||||
fn constant_bit_width(integer_value: inkwell::values::IntValue) -> Option<u32> {
|
||||
if let Some(value) = integer_value.get_zero_extended_constant() {
|
||||
return Some(if value == 0 {
|
||||
1
|
||||
} else {
|
||||
64 - value.leading_zeros()
|
||||
});
|
||||
}
|
||||
|
||||
let wide_type = integer_value.get_type();
|
||||
if wide_type.get_bit_width() > 64 {
|
||||
let i64_type = wide_type.get_context().i64_type();
|
||||
let truncated = integer_value.const_truncate(i64_type);
|
||||
if let Some(value) = truncated.get_zero_extended_constant() {
|
||||
let reconstructed = wide_type.const_int(value, false);
|
||||
if reconstructed == integer_value {
|
||||
return Some(if value == 0 {
|
||||
1
|
||||
} else {
|
||||
64 - value.leading_zeros()
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
/// Rounds a bit width up to the next standard integer type width.
|
||||
fn round_up_bit_width(bits: u32) -> u32 {
|
||||
if bits <= 8 {
|
||||
8
|
||||
} else if bits <= 16 {
|
||||
16
|
||||
} else if bits <= 32 {
|
||||
32
|
||||
} else if bits <= 64 {
|
||||
64
|
||||
} else if bits <= 128 {
|
||||
128
|
||||
} else {
|
||||
256
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,15 +1,20 @@
|
||||
//! The revive simulated EVM linear memory pointer functions.
|
||||
|
||||
use std::num::NonZeroU32;
|
||||
|
||||
use inkwell::values::BasicValueEnum;
|
||||
|
||||
use revive_common::BYTE_LENGTH_BYTE;
|
||||
use revive_common::BYTE_LENGTH_WORD;
|
||||
use revive_common::BYTE_LENGTH_X64;
|
||||
|
||||
use crate::polkavm::context::attribute::MemoryEffect;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
use crate::polkavm::context::Context;
|
||||
use crate::polkavm::WriteLLVM;
|
||||
|
||||
/// Load a word size value from a heap pointer.
|
||||
/// Uses 4x 64-bit loads with bswap for efficient byte-order conversion.
|
||||
pub struct LoadWord;
|
||||
|
||||
impl RuntimeFunction for LoadWord {
|
||||
@@ -21,6 +26,282 @@ impl RuntimeFunction for LoadWord {
|
||||
.fn_type(&[context.xlen_type().into()], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<BasicValueEnum<'ctx>>> {
|
||||
let offset = Self::paramater(context, 0).into_int_value();
|
||||
let length = context
|
||||
.xlen_type()
|
||||
.const_int(BYTE_LENGTH_WORD as u64, false);
|
||||
let pointer = context.build_heap_gep(offset, length)?;
|
||||
let result = build_efficient_load_swap(context, pointer.value)?;
|
||||
Ok(Some(result))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for LoadWord {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Store a word size value through a heap pointer.
|
||||
/// Uses 4x 64-bit stores with bswap for efficient byte-order conversion.
|
||||
pub struct StoreWord;
|
||||
|
||||
impl RuntimeFunction for StoreWord {
|
||||
const NAME: &'static str = "__revive_store_heap_word";
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.void_type().fn_type(
|
||||
&[context.xlen_type().into(), context.word_type().into()],
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<BasicValueEnum<'ctx>>> {
|
||||
let offset = Self::paramater(context, 0).into_int_value();
|
||||
let length = context
|
||||
.xlen_type()
|
||||
.const_int(BYTE_LENGTH_WORD as u64, false);
|
||||
let pointer = context.build_heap_gep(offset, length)?;
|
||||
let value = Self::paramater(context, 1).into_int_value();
|
||||
build_efficient_store_swap(context, pointer.value, value)?;
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds an efficient 256-bit load with byte-swap using 4x 64-bit operations.
|
||||
///
|
||||
/// This is much more efficient than LLVM's default lowering of llvm.bswap.i256
|
||||
/// which generates byte-by-byte loads. We instead:
|
||||
/// 1. Load 4x 64-bit values
|
||||
/// 2. Byte-swap each 64-bit value using llvm.bswap.i64
|
||||
/// 3. Combine them in reversed word order to form the 256-bit result
|
||||
///
|
||||
/// The unrolled form only pays off with the `+unaligned-scalar-mem` target feature,
|
||||
/// which is enabled for newyork modules exclusively. Without it the backend splits
|
||||
/// each 64-bit access into byte operations, producing larger and slower code than
|
||||
/// the single word load it replaces, so the stock Yul path keeps the legacy
|
||||
/// `llvm.bswap.i256` lowering it was validated against.
|
||||
pub(crate) fn build_efficient_load_swap<'ctx>(
|
||||
context: &Context<'ctx>,
|
||||
pointer: inkwell::values::PointerValue<'ctx>,
|
||||
) -> anyhow::Result<BasicValueEnum<'ctx>> {
|
||||
if !context.is_newyork() {
|
||||
let value = context
|
||||
.builder()
|
||||
.build_load(context.word_type(), pointer, "value")?;
|
||||
context
|
||||
.basic_block()
|
||||
.get_last_instruction()
|
||||
.expect("ICE: load instruction always exists")
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("ICE: alignment is valid");
|
||||
return context.build_byte_swap(value);
|
||||
}
|
||||
let i64_type = context
|
||||
.llvm()
|
||||
.custom_width_int_type(NonZeroU32::new(64).expect("const is non-zero"))
|
||||
.expect("valid integer width");
|
||||
let i8_type = context
|
||||
.llvm()
|
||||
.custom_width_int_type(NonZeroU32::new(8).expect("const is non-zero"))
|
||||
.expect("valid integer width");
|
||||
let word_type = context.word_type();
|
||||
|
||||
let bswap64 = inkwell::intrinsics::Intrinsic::find("llvm.bswap.i64")
|
||||
.expect("ICE: llvm.bswap.i64 intrinsic exists");
|
||||
let bswap64_function = bswap64
|
||||
.get_declaration(context.module(), &[i64_type.into()])
|
||||
.expect("ICE: bswap.i64 declaration");
|
||||
|
||||
let mut swapped_x64_values = Vec::with_capacity(4);
|
||||
for index in 0..4 {
|
||||
let gep_offset = context
|
||||
.xlen_type()
|
||||
.const_int(index * BYTE_LENGTH_X64 as u64, false);
|
||||
let byte_pointer = unsafe {
|
||||
context.builder().build_gep(
|
||||
i8_type,
|
||||
pointer,
|
||||
&[gep_offset],
|
||||
&format!("byte_pointer_{index}"),
|
||||
)?
|
||||
};
|
||||
let x64_value = context
|
||||
.builder()
|
||||
.build_load(i64_type, byte_pointer, &format!("x64_value_{index}"))?
|
||||
.into_int_value();
|
||||
context
|
||||
.basic_block()
|
||||
.get_last_instruction()
|
||||
.expect("ICE: load instruction always exists")
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("ICE: alignment is valid");
|
||||
|
||||
let swapped_x64_value = context
|
||||
.builder()
|
||||
.build_call(
|
||||
bswap64_function,
|
||||
&[x64_value.into()],
|
||||
&format!("swapped_x64_value_{index}"),
|
||||
)?
|
||||
.try_as_basic_value()
|
||||
.unwrap_basic()
|
||||
.into_int_value();
|
||||
swapped_x64_values.push(swapped_x64_value);
|
||||
}
|
||||
|
||||
// Combine in reversed register order: chunks 3, 2, 1, 0 from most to
|
||||
// least significant. This achieves the full big-endian to little-endian
|
||||
// conversion of the 256-bit word.
|
||||
let mut result =
|
||||
context
|
||||
.builder()
|
||||
.build_int_z_extend(swapped_x64_values[3], word_type, "ext_0")?;
|
||||
for (index, &x64_value) in swapped_x64_values.iter().rev().skip(1).enumerate() {
|
||||
let extended = context.builder().build_int_z_extend(
|
||||
x64_value,
|
||||
word_type,
|
||||
&format!("ext_{}", index + 1),
|
||||
)?;
|
||||
let shift_amount = word_type.const_int(64 * (index + 1) as u64, false);
|
||||
let shifted = context.builder().build_left_shift(
|
||||
extended,
|
||||
shift_amount,
|
||||
&format!("shift_{}", index + 1),
|
||||
)?;
|
||||
result = context
|
||||
.builder()
|
||||
.build_or(result, shifted, &format!("or_{}", index + 1))?;
|
||||
}
|
||||
|
||||
Ok(result.into())
|
||||
}
|
||||
|
||||
/// Builds an efficient 256-bit store with byte-swap using 4x 64-bit operations.
|
||||
///
|
||||
/// This is much more efficient than LLVM's default lowering of llvm.bswap.i256
|
||||
/// which generates byte-by-byte stores. We instead:
|
||||
/// 1. Extract 4x 64-bit values from the 256-bit word
|
||||
/// 2. Byte-swap each 64-bit value using llvm.bswap.i64
|
||||
/// 3. Store them in reversed word order
|
||||
///
|
||||
/// As with [`build_efficient_load_swap`], the unrolled form requires the
|
||||
/// `+unaligned-scalar-mem` target feature and is therefore reserved for newyork
|
||||
/// modules; the stock Yul path keeps the legacy `llvm.bswap.i256` lowering.
|
||||
pub(crate) fn build_efficient_store_swap<'ctx>(
|
||||
context: &Context<'ctx>,
|
||||
pointer: inkwell::values::PointerValue<'ctx>,
|
||||
value: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<()> {
|
||||
if !context.is_newyork() {
|
||||
let swapped_value = context.build_byte_swap(value.into())?;
|
||||
context
|
||||
.builder()
|
||||
.build_store(pointer, swapped_value)?
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("ICE: alignment is valid");
|
||||
return Ok(());
|
||||
}
|
||||
let i64_type = context
|
||||
.llvm()
|
||||
.custom_width_int_type(NonZeroU32::new(64).expect("const is non-zero"))
|
||||
.expect("valid integer width");
|
||||
let i8_type = context
|
||||
.llvm()
|
||||
.custom_width_int_type(NonZeroU32::new(8).expect("const is non-zero"))
|
||||
.expect("valid integer width");
|
||||
let word_type = context.word_type();
|
||||
|
||||
let bswap64 = inkwell::intrinsics::Intrinsic::find("llvm.bswap.i64")
|
||||
.expect("ICE: llvm.bswap.i64 intrinsic exists");
|
||||
let bswap64_function = bswap64
|
||||
.get_declaration(context.module(), &[i64_type.into()])
|
||||
.expect("ICE: bswap.i64 declaration");
|
||||
|
||||
for index in 0..4 {
|
||||
let shift_amount = word_type.const_int(64 * index as u64, false);
|
||||
let shifted = context.builder().build_right_shift(
|
||||
value,
|
||||
shift_amount,
|
||||
false,
|
||||
&format!("shift_{index}"),
|
||||
)?;
|
||||
let x64_value =
|
||||
context
|
||||
.builder()
|
||||
.build_int_truncate(shifted, i64_type, &format!("trunc_{index}"))?;
|
||||
|
||||
let swapped_x64_value = context
|
||||
.builder()
|
||||
.build_call(
|
||||
bswap64_function,
|
||||
&[x64_value.into()],
|
||||
&format!("swap_x64_value_{index}"),
|
||||
)?
|
||||
.try_as_basic_value()
|
||||
.unwrap_basic()
|
||||
.into_int_value();
|
||||
|
||||
// Reverse the position so the least-significant chunk lands at the
|
||||
// highest byte offset; this performs the big-endian byte order on
|
||||
// the 256-bit word as a whole.
|
||||
let store_byte_offset = (3 - index) * BYTE_LENGTH_X64;
|
||||
let gep_offset = context
|
||||
.xlen_type()
|
||||
.const_int(store_byte_offset as u64, false);
|
||||
let byte_pointer = unsafe {
|
||||
context.builder().build_gep(
|
||||
i8_type,
|
||||
pointer,
|
||||
&[gep_offset],
|
||||
&format!("store_pointer_{index}"),
|
||||
)?
|
||||
};
|
||||
let store_instruction = context
|
||||
.builder()
|
||||
.build_store(byte_pointer, swapped_x64_value)?;
|
||||
store_instruction
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("ICE: alignment is valid");
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl WriteLLVM for StoreWord {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Load a word size value from a heap pointer without byte-swapping.
|
||||
/// Used for internal memory operations that don't escape to external code.
|
||||
pub struct LoadWordNative;
|
||||
|
||||
impl RuntimeFunction for LoadWordNative {
|
||||
const NAME: &'static str = "__revive_load_heap_word_native";
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context
|
||||
.word_type()
|
||||
.fn_type(&[context.xlen_type().into()], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
@@ -36,16 +317,14 @@ impl RuntimeFunction for LoadWord {
|
||||
context
|
||||
.basic_block()
|
||||
.get_last_instruction()
|
||||
.expect("Always exists")
|
||||
.expect("ICE: load instruction always exists")
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("Alignment is valid");
|
||||
|
||||
let swapped_value = context.build_byte_swap(value)?;
|
||||
Ok(Some(swapped_value))
|
||||
.expect("ICE: alignment is valid");
|
||||
Ok(Some(value))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for LoadWord {
|
||||
impl WriteLLVM for LoadWordNative {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
@@ -55,11 +334,12 @@ impl WriteLLVM for LoadWord {
|
||||
}
|
||||
}
|
||||
|
||||
/// Store a word size value through a heap pointer.
|
||||
pub struct StoreWord;
|
||||
/// Store a word size value through a heap pointer without byte-swapping.
|
||||
/// Used for internal memory operations that don't escape to external code.
|
||||
pub struct StoreWordNative;
|
||||
|
||||
impl RuntimeFunction for StoreWord {
|
||||
const NAME: &'static str = "__revive_store_heap_word";
|
||||
impl RuntimeFunction for StoreWordNative {
|
||||
const NAME: &'static str = "__revive_store_heap_word_native";
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.void_type().fn_type(
|
||||
@@ -77,19 +357,178 @@ impl RuntimeFunction for StoreWord {
|
||||
.xlen_type()
|
||||
.const_int(BYTE_LENGTH_WORD as u64, false);
|
||||
let pointer = context.build_heap_gep(offset, length)?;
|
||||
|
||||
let value = context.build_byte_swap(Self::paramater(context, 1))?;
|
||||
let value = Self::paramater(context, 1);
|
||||
|
||||
context
|
||||
.builder()
|
||||
.build_store(pointer.value, value)?
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("Alignment is valid");
|
||||
.expect("ICE: alignment is valid");
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for StoreWord {
|
||||
impl WriteLLVM for StoreWordNative {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Keccak256 hash of one 256-bit word from scratch memory.
|
||||
/// Equivalent to: mstore(0, word0); sha3(0, 32)
|
||||
/// Deduplicated into a single function to reduce code size for storage slot lookups.
|
||||
pub struct Keccak256OneWord;
|
||||
|
||||
impl Keccak256OneWord {
|
||||
/// The function name.
|
||||
pub const NAME: &'static str = "__revive_keccak256_one_word";
|
||||
}
|
||||
|
||||
impl RuntimeFunction for Keccak256OneWord {
|
||||
const NAME: &'static str = "__revive_keccak256_one_word";
|
||||
|
||||
const ATTRIBUTES: &'static [crate::polkavm::context::Attribute] = &[
|
||||
crate::polkavm::context::Attribute::NoFree,
|
||||
crate::polkavm::context::Attribute::NoRecurse,
|
||||
crate::polkavm::context::Attribute::WillReturn,
|
||||
crate::polkavm::context::Attribute::NoInline,
|
||||
];
|
||||
|
||||
/// Writes the input back to heap[0..32] (matching EVM's
|
||||
/// `mstore(0, word0); keccak256(0, 32)`), then hashes from there.
|
||||
/// This lets `mem_opt`'s keccak fusion safely dead-eliminate the
|
||||
/// preceding `mstore(0, k)` — a later `mload(0)` whose mem_opt
|
||||
/// forwarding has been invalidated (e.g., by an intervening
|
||||
/// external call) reads the helper's heap write and gets the
|
||||
/// same value EVM would. The helper is no longer
|
||||
/// `MemoryEffect::None`; `Unrestricted` gives LLVM accurate
|
||||
/// information about its memory effects.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::Unrestricted;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context
|
||||
.word_type()
|
||||
.fn_type(&[context.word_type().into()], false)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<BasicValueEnum<'ctx>>> {
|
||||
let word0 = Self::paramater(context, 0).into_int_value();
|
||||
|
||||
// Mirror EVM's `mstore(0, word0)`: byte-swap and store to heap[0..32].
|
||||
let zero_xlen = context.xlen_type().const_zero();
|
||||
crate::polkavm::evm::memory::store_bswap_unchecked(context, zero_xlen, word0)?;
|
||||
|
||||
let length = context
|
||||
.xlen_type()
|
||||
.const_int(BYTE_LENGTH_WORD as u64, false);
|
||||
|
||||
let output_pointer = context.build_alloca_at_entry(context.word_type(), "output_pointer");
|
||||
let input_pointer = context.build_heap_gep_unchecked(zero_xlen)?;
|
||||
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::HASH_KECCAK_256,
|
||||
&[
|
||||
input_pointer.to_int(context).into(),
|
||||
length.into(),
|
||||
output_pointer.to_int(context).into(),
|
||||
],
|
||||
);
|
||||
|
||||
let result = context.build_byte_swap(context.build_load(output_pointer, "sha3_output")?)?;
|
||||
Ok(Some(result))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for Keccak256OneWord {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
fn into_llvm(self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::emit(&self, context)
|
||||
}
|
||||
}
|
||||
|
||||
/// Keccak256 hash of two 256-bit words from scratch memory.
|
||||
/// Equivalent to: mstore(0, word0); mstore(32, word1); sha3(0, 64)
|
||||
/// Deduplicated into a single function to reduce code size for mapping lookups.
|
||||
pub struct Keccak256TwoWords;
|
||||
|
||||
impl Keccak256TwoWords {
|
||||
/// The function name.
|
||||
pub const NAME: &'static str = "__revive_keccak256_two_words";
|
||||
}
|
||||
|
||||
impl RuntimeFunction for Keccak256TwoWords {
|
||||
const NAME: &'static str = "__revive_keccak256_two_words";
|
||||
|
||||
const ATTRIBUTES: &'static [crate::polkavm::context::Attribute] = &[
|
||||
crate::polkavm::context::Attribute::NoFree,
|
||||
crate::polkavm::context::Attribute::NoRecurse,
|
||||
crate::polkavm::context::Attribute::WillReturn,
|
||||
crate::polkavm::context::Attribute::NoInline,
|
||||
];
|
||||
|
||||
/// Writes the inputs back to heap[0..64] (matching EVM's
|
||||
/// `mstore(0, word0); mstore(32, word1); keccak256(0, 64)`), then
|
||||
/// hashes from there. This lets `mem_opt`'s keccak fusion safely
|
||||
/// dead-eliminate the preceding mstores — a later `mload` at offset
|
||||
/// 0 or 32 whose mem_opt forwarding has been invalidated reads the
|
||||
/// helper's heap writes and gets the same value EVM would.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::Unrestricted;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context.word_type().fn_type(
|
||||
&[context.word_type().into(), context.word_type().into()],
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
fn emit_body<'ctx>(
|
||||
&self,
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<Option<BasicValueEnum<'ctx>>> {
|
||||
let word0 = Self::paramater(context, 0).into_int_value();
|
||||
let word1 = Self::paramater(context, 1).into_int_value();
|
||||
|
||||
// Mirror EVM's `mstore(0, word0); mstore(32, word1)`: byte-swap
|
||||
// and store both words to heap[0..64].
|
||||
let zero_xlen = context.xlen_type().const_zero();
|
||||
let word_xlen = context
|
||||
.xlen_type()
|
||||
.const_int(BYTE_LENGTH_WORD as u64, false);
|
||||
crate::polkavm::evm::memory::store_bswap_unchecked(context, zero_xlen, word0)?;
|
||||
crate::polkavm::evm::memory::store_bswap_unchecked(context, word_xlen, word1)?;
|
||||
|
||||
let length = context
|
||||
.xlen_type()
|
||||
.const_int(2 * BYTE_LENGTH_WORD as u64, false);
|
||||
|
||||
let output_pointer = context.build_alloca_at_entry(context.word_type(), "output_pointer");
|
||||
let input_pointer = context.build_heap_gep_unchecked(zero_xlen)?;
|
||||
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::HASH_KECCAK_256,
|
||||
&[
|
||||
input_pointer.to_int(context).into(),
|
||||
length.into(),
|
||||
output_pointer.to_int(context).into(),
|
||||
],
|
||||
);
|
||||
|
||||
let result = context.build_byte_swap(context.build_load(output_pointer, "sha3_output")?)?;
|
||||
Ok(Some(result))
|
||||
}
|
||||
}
|
||||
|
||||
impl WriteLLVM for Keccak256TwoWords {
|
||||
fn declare(&mut self, context: &mut Context) -> anyhow::Result<()> {
|
||||
<Self as RuntimeFunction>::declare(self, context)
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
use inkwell::values::BasicValueEnum;
|
||||
|
||||
use crate::polkavm::context::attribute::MemoryEffect;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
use crate::polkavm::context::Context;
|
||||
use crate::polkavm::WriteLLVM;
|
||||
@@ -12,6 +13,13 @@ pub struct LoadWord;
|
||||
impl RuntimeFunction for LoadWord {
|
||||
const NAME: &'static str = "__revive_load_storage_word";
|
||||
|
||||
/// The helper dereferences its key pointer (a read through `argmem`) and
|
||||
/// reads storage (pallet-revive runtime state, invisible to LLVM). Heap
|
||||
/// is not touched. Intervening `__revive_store_storage_word` carries the
|
||||
/// default attributes and still acts as a barrier, so this lets GVN
|
||||
/// dedupe `__revive_load_storage_word` calls across heap mstores.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::ReadArgAndInaccessible;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context
|
||||
.word_type()
|
||||
@@ -46,6 +54,11 @@ pub struct LoadTransientWord;
|
||||
impl RuntimeFunction for LoadTransientWord {
|
||||
const NAME: &'static str = "__revive_load_transient_storage_word";
|
||||
|
||||
/// Same shape as [`LoadWord`]: argmem read (key pointer) plus a read of
|
||||
/// pallet-revive runtime state. Transient-store wrappers retain unknown
|
||||
/// memory effects and act as a barrier preventing incorrect CSE.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::ReadArgAndInaccessible;
|
||||
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx> {
|
||||
context
|
||||
.word_type()
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
//! This benefits contract code size.
|
||||
|
||||
use crate::optimizer::settings::size_level::SizeLevel;
|
||||
use crate::polkavm::context::attribute::MemoryEffect;
|
||||
use crate::polkavm::context::function::declaration::Declaration;
|
||||
use crate::polkavm::context::function::Function;
|
||||
use crate::polkavm::context::Attribute;
|
||||
@@ -21,10 +22,19 @@ pub trait RuntimeFunction {
|
||||
Attribute::WillReturn,
|
||||
];
|
||||
|
||||
/// The `memory(...)` effect to attach to the function declaration.
|
||||
///
|
||||
/// Helpers that interact only with their own allocas or with pallet-revive
|
||||
/// runtime state (via host syscalls) can advertise a tighter effect than
|
||||
/// the conservative LLVM default. The trait wires the encoding through
|
||||
/// [`Self::declare`] so concrete impls don't have to call the raw inkwell
|
||||
/// API. Default: do not attach the attribute.
|
||||
const MEMORY_EFFECT: MemoryEffect = MemoryEffect::Unrestricted;
|
||||
|
||||
/// The function type.
|
||||
fn r#type<'ctx>(context: &Context<'ctx>) -> inkwell::types::FunctionType<'ctx>;
|
||||
|
||||
/// Declare the function.
|
||||
/// Declare the function with standard attributes.
|
||||
fn declare(&self, context: &mut Context) -> anyhow::Result<()> {
|
||||
let function = context.add_function(
|
||||
Self::NAME,
|
||||
@@ -46,6 +56,14 @@ pub trait RuntimeFunction {
|
||||
&attributes,
|
||||
true,
|
||||
);
|
||||
if let Some(encoding) = Self::MEMORY_EFFECT.encoding() {
|
||||
function.borrow().declaration().value.add_attribute(
|
||||
inkwell::attributes::AttributeLoc::Function,
|
||||
context
|
||||
.llvm()
|
||||
.create_enum_attribute(Attribute::Memory as u32, encoding),
|
||||
);
|
||||
}
|
||||
let function = function.borrow().declaration().function_value();
|
||||
let comdat = context
|
||||
.module()
|
||||
|
||||
@@ -7,7 +7,13 @@ use crate::PolkaVMTarget;
|
||||
|
||||
/// Initializes the LLVM compiler backend.
|
||||
fn initialize_llvm() {
|
||||
crate::initialize_llvm(PolkaVMTarget::PVM, "resolc", Default::default());
|
||||
crate::initialize_llvm(
|
||||
PolkaVMTarget::PVM,
|
||||
"resolc",
|
||||
crate::OptimizerSettingsSizeLevel::Zero,
|
||||
false,
|
||||
Default::default(),
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -18,6 +18,30 @@ pub fn load<'ctx>(
|
||||
context.build_load(output_pointer, "call_data_load_value")
|
||||
}
|
||||
|
||||
/// Calls the outlined `__revive_calldataload(xlen) -> i256` runtime function.
|
||||
///
|
||||
/// Functionally identical to [`load`] but calls a shared outlined function
|
||||
/// instead of inlining the alloca+call+load sequence at every call site.
|
||||
pub fn load_outlined<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
use crate::polkavm::context::function::runtime::revive::CallDataLoad;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
let offset = context.clip_to_xlen(offset)?;
|
||||
let function = context
|
||||
.get_function(CallDataLoad::NAME, false)
|
||||
.expect("ICE: __revive_calldataload should be declared");
|
||||
let result = context
|
||||
.build_call(
|
||||
function.borrow().declaration(),
|
||||
&[offset.into()],
|
||||
"calldataload_result",
|
||||
)
|
||||
.expect("ICE: __revive_calldataload should return a value");
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
/// Translates the calldata size.
|
||||
pub fn size<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
|
||||
@@ -148,3 +148,23 @@ pub fn caller<'ctx>(
|
||||
);
|
||||
context.build_load_address(pointer)
|
||||
}
|
||||
|
||||
/// Calls the outlined `__revive_caller() -> i256` runtime function.
|
||||
///
|
||||
/// This is functionally identical to [`caller`] but calls a shared outlined function
|
||||
/// instead of inlining the get_global+call+load+bswap+zext sequence at every call site.
|
||||
/// For contracts with many msg.sender checks (e.g. OpenZeppelin ERC20: 10+ sites),
|
||||
/// this significantly reduces code size.
|
||||
pub fn caller_outlined<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
use crate::polkavm::context::function::runtime::revive::Caller;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
let function = context
|
||||
.get_function(Caller::NAME, false)
|
||||
.expect("ICE: __revive_caller should be declared");
|
||||
let result = context
|
||||
.build_call(function.borrow().declaration(), &[], "caller_result")
|
||||
.expect("ICE: __revive_caller should return a value");
|
||||
Ok(result)
|
||||
}
|
||||
@@ -11,7 +11,7 @@ pub fn sha3<'ctx>(
|
||||
let offset_casted = context.safe_truncate_int_to_xlen(offset)?;
|
||||
let length_casted = context.safe_truncate_int_to_xlen(length)?;
|
||||
let input_pointer = context.build_heap_gep(offset_casted, length_casted)?;
|
||||
let output_pointer = context.build_alloca(context.word_type(), "output_pointer");
|
||||
let output_pointer = context.build_alloca_at_entry(context.word_type(), "output_pointer");
|
||||
|
||||
context.build_runtime_call(
|
||||
revive_runtime_api::polkavm_imports::HASH_KECCAK_256,
|
||||
@@ -24,3 +24,52 @@ pub fn sha3<'ctx>(
|
||||
|
||||
context.build_byte_swap(context.build_load(output_pointer, "sha3_output")?)
|
||||
}
|
||||
|
||||
/// Translates keccak256 of one 256-bit word via a deduplicated helper function.
|
||||
/// Equivalent to: mstore(0, word0); sha3(0, 32)
|
||||
/// but emitted as a single function call to reduce code size.
|
||||
///
|
||||
/// Falls back to inline code if the helper function is not declared
|
||||
/// (e.g. when there are too few call sites to justify the function body cost).
|
||||
pub fn sha3_one_word<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
word0: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
use crate::polkavm::context::pointer::heap::Keccak256OneWord;
|
||||
let Some(function) = context.get_function(Keccak256OneWord::NAME, false) else {
|
||||
let offset = context.word_const(0);
|
||||
let length = context.word_const(32);
|
||||
crate::polkavm::evm::memory::store(context, offset, word0)?;
|
||||
return sha3(context, offset, length);
|
||||
};
|
||||
let result = context
|
||||
.build_call(
|
||||
function.borrow().declaration(),
|
||||
&[word0.into()],
|
||||
"keccak256_single_result",
|
||||
)
|
||||
.expect("ICE: keccak256_one_word should return a value");
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
/// Translates keccak256 of two 256-bit words via a deduplicated helper function.
|
||||
/// Equivalent to: mstore(0, word0); mstore(32, word1); sha3(0, 64)
|
||||
/// but emitted as a single function call to reduce code size.
|
||||
pub fn sha3_two_words<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
word0: inkwell::values::IntValue<'ctx>,
|
||||
word1: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
use crate::polkavm::context::pointer::heap::Keccak256TwoWords;
|
||||
let function = context
|
||||
.get_function(Keccak256TwoWords::NAME, false)
|
||||
.expect("ICE: __revive_keccak256_two_words should be declared");
|
||||
let result = context
|
||||
.build_call(
|
||||
function.borrow().declaration(),
|
||||
&[word0.into(), word1.into()],
|
||||
"keccak256_pair_result",
|
||||
)
|
||||
.expect("ICE: keccak256_two_words should return a value");
|
||||
Ok(result)
|
||||
}
|
||||
@@ -30,6 +30,46 @@ pub fn value<'ctx>(
|
||||
context.build_load(output_pointer, "value_transferred")
|
||||
}
|
||||
|
||||
/// Calls the outlined `__revive_callvalue() -> i256` runtime function.
|
||||
///
|
||||
/// This is functionally identical to [`value`] but calls a shared outlined function
|
||||
/// instead of inlining the alloca+store+call+load sequence at every call site.
|
||||
/// For contracts with many non-payable checks (e.g. OpenZeppelin ERC20: 37 sites),
|
||||
/// this significantly reduces code size.
|
||||
pub fn value_outlined<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
use crate::polkavm::context::function::runtime::revive::CallValue;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
let function = context
|
||||
.get_function(CallValue::NAME, false)
|
||||
.expect("ICE: __revive_callvalue should be declared");
|
||||
let result = context
|
||||
.build_call(function.borrow().declaration(), &[], "callvalue_result")
|
||||
.expect("ICE: __revive_callvalue should return a value");
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
/// Calls the outlined `__revive_callvalue_nonzero() -> i1` runtime function.
|
||||
///
|
||||
/// Returns true (i1) if callvalue is nonzero. This is more efficient than
|
||||
/// `value_outlined()` followed by `icmp ne i256 %cv, 0` because the
|
||||
/// 256-bit comparison is done once inside the outlined function body,
|
||||
/// and each call site only receives a single boolean flag.
|
||||
pub fn value_nonzero_outlined<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
use crate::polkavm::context::function::runtime::revive::CallValueNonzero;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
let function = context
|
||||
.get_function(CallValueNonzero::NAME, false)
|
||||
.expect("ICE: __revive_callvalue_nonzero should be declared");
|
||||
let result = context
|
||||
.build_call(function.borrow().declaration(), &[], "callvalue_nonzero")
|
||||
.expect("ICE: __revive_callvalue_nonzero should return a value");
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
/// Translates the `balance` instructions.
|
||||
pub fn balance<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
//! Translates the heap memory operations.
|
||||
|
||||
use inkwell::values::BasicValue;
|
||||
use inkwell::values::BasicValueEnum;
|
||||
use revive_common::BYTE_LENGTH_BYTE;
|
||||
|
||||
use crate::polkavm::context::address_space::AddressSpace;
|
||||
use crate::polkavm::context::pointer::heap::{
|
||||
build_efficient_load_swap, build_efficient_store_swap,
|
||||
};
|
||||
use crate::polkavm::context::pointer::Pointer;
|
||||
use crate::polkavm::context::Context;
|
||||
|
||||
@@ -81,6 +85,74 @@ pub fn store_byte<'ctx>(
|
||||
.builder()
|
||||
.build_store(pointer, value)?
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("Alignment is valid");
|
||||
.expect("ICE: alignment is valid");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Builds an efficient 256-bit store with byte-swap at a pointer obtained via unchecked GEP
|
||||
/// (no sbrk bounds check). For use by the newyork InlineByteSwap mode on constant offsets
|
||||
/// within the static heap.
|
||||
pub fn store_bswap_unchecked<'ctx>(
|
||||
context: &Context<'ctx>,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
value: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<()> {
|
||||
let pointer = context.build_heap_gep_unchecked(offset)?;
|
||||
build_efficient_store_swap(context, pointer.value, value)
|
||||
}
|
||||
|
||||
/// Builds an efficient 256-bit load with byte-swap at a pointer obtained via unchecked GEP
|
||||
/// (no sbrk bounds check). For use by the newyork InlineByteSwap mode on constant offsets
|
||||
/// within the static heap.
|
||||
pub fn load_bswap_unchecked<'ctx>(
|
||||
context: &Context<'ctx>,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<BasicValueEnum<'ctx>> {
|
||||
let pointer = context.build_heap_gep_unchecked(offset)?;
|
||||
build_efficient_load_swap(context, pointer.value)
|
||||
}
|
||||
|
||||
/// Translates the `mload` instruction without byte-swapping, inlined.
|
||||
/// This version inlines the load directly without a function call,
|
||||
/// saving both call overhead and function body size when not all
|
||||
/// memory accesses need native mode.
|
||||
pub fn load_native_inline<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<inkwell::values::BasicValueEnum<'ctx>> {
|
||||
let length = context
|
||||
.xlen_type()
|
||||
.const_int(revive_common::BYTE_LENGTH_WORD as u64, false);
|
||||
let pointer = context.build_heap_gep(offset, length)?;
|
||||
let value = context
|
||||
.builder()
|
||||
.build_load(context.word_type(), pointer.value, "native_load")?;
|
||||
context
|
||||
.basic_block()
|
||||
.get_last_instruction()
|
||||
.expect("ICE: load instruction always exists")
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("ICE: alignment is valid");
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
/// Translates the `mstore` instruction without byte-swapping, inlined.
|
||||
/// This version inlines the store directly without a function call,
|
||||
/// saving both call overhead and function body size when not all
|
||||
/// memory accesses need native mode.
|
||||
pub fn store_native_inline<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
offset: inkwell::values::IntValue<'ctx>,
|
||||
value: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<()> {
|
||||
let length = context
|
||||
.xlen_type()
|
||||
.const_int(revive_common::BYTE_LENGTH_WORD as u64, false);
|
||||
let pointer = context.build_heap_gep(offset, length)?;
|
||||
context
|
||||
.builder()
|
||||
.build_store(pointer.value, value)?
|
||||
.set_alignment(BYTE_LENGTH_BYTE as u32)
|
||||
.expect("ICE: alignment is valid");
|
||||
Ok(())
|
||||
}
|
||||
@@ -43,6 +43,37 @@ pub fn revert<'ctx>(
|
||||
)
|
||||
}
|
||||
|
||||
/// Calls the outlined `__revive_revert_0()` function for empty reverts.
|
||||
pub fn revert_empty_outlined(context: &mut Context) -> anyhow::Result<()> {
|
||||
use crate::polkavm::context::function::runtime::revive::RevertEmpty;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
let function = context
|
||||
.get_function(RevertEmpty::NAME, false)
|
||||
.expect("ICE: __revive_revert_0 should be declared");
|
||||
context.build_call(function.borrow().declaration(), &[], "revert_empty");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Calls the outlined `__revive_revert(length)` function for constant-length reverts.
|
||||
///
|
||||
/// The length parameter is xlen-sized (already truncated from i256).
|
||||
pub fn revert_outlined<'ctx>(
|
||||
context: &mut Context<'ctx>,
|
||||
length: inkwell::values::IntValue<'ctx>,
|
||||
) -> anyhow::Result<()> {
|
||||
use crate::polkavm::context::function::runtime::revive::Revert;
|
||||
use crate::polkavm::context::runtime::RuntimeFunction;
|
||||
let function = context
|
||||
.get_function(Revert::NAME, false)
|
||||
.expect("ICE: __revive_revert should be declared");
|
||||
context.build_call(
|
||||
function.borrow().declaration(),
|
||||
&[length.into()],
|
||||
"revert_outlined",
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Translates the `stop` instruction.
|
||||
/// Is the same as `return(0, 0)`.
|
||||
pub fn stop(context: &mut Context) -> anyhow::Result<()> {
|
||||
|
||||
@@ -144,7 +144,8 @@ pub fn build_symbols(
|
||||
);
|
||||
}
|
||||
|
||||
let target_machine = PolkaVMTargetMachine::new(PolkaVMTarget::PVM, &OptimizerSettings::none())?;
|
||||
let target_machine =
|
||||
PolkaVMTargetMachine::new(PolkaVMTarget::PVM, &OptimizerSettings::none(), false)?;
|
||||
let buffer = target_machine
|
||||
.write_to_memory_buffer(&module)
|
||||
.expect("ICE: the symbols module should be valid");
|
||||
|
||||
@@ -31,15 +31,35 @@ impl TargetMachine {
|
||||
pub const VM_FEATURES: &'static str =
|
||||
"+e,+m,+a,+c,+zbb,+auipc-addi-fusion,+ld-add-fusion,+lui-addi-fusion,+xtheadcondmov,+relax";
|
||||
|
||||
/// RISC-V backend feature that lets the code generator emit single (potentially unaligned)
|
||||
/// scalar memory accesses instead of splitting them into byte-wise sequences. Enabled for
|
||||
/// newyork modules only; the stock Yul path omits it, matching the upstream target
|
||||
/// configuration. Enabling it on the Yul path was observed to introduce an end-to-end
|
||||
/// regression (`complex/internal_function_pointers/mixed_features_2`); the exact interaction
|
||||
/// was not isolated, so the feature is restricted to the path it was validated against.
|
||||
pub const VM_FEATURE_UNALIGNED: &'static str = ",+unaligned-scalar-mem";
|
||||
|
||||
/// A shortcut constructor.
|
||||
/// A separate instance for every optimization level is created.
|
||||
pub fn new(target: Target, optimizer_settings: &OptimizerSettings) -> anyhow::Result<Self> {
|
||||
///
|
||||
/// `unaligned_scalar_mem` appends [`Self::VM_FEATURE_UNALIGNED`]; pass `true` only for newyork
|
||||
/// modules (see the constant's documentation).
|
||||
pub fn new(
|
||||
target: Target,
|
||||
optimizer_settings: &OptimizerSettings,
|
||||
unaligned_scalar_mem: bool,
|
||||
) -> anyhow::Result<Self> {
|
||||
let features = if unaligned_scalar_mem {
|
||||
format!("{}{}", Self::VM_FEATURES, Self::VM_FEATURE_UNALIGNED)
|
||||
} else {
|
||||
Self::VM_FEATURES.to_string()
|
||||
};
|
||||
let target_machine = inkwell::targets::Target::from_name(target.name())
|
||||
.ok_or_else(|| anyhow::anyhow!("LLVM target machine `{}` not found", target.name()))?
|
||||
.create_target_machine(
|
||||
&inkwell::targets::TargetTriple::create(target.triple()),
|
||||
Self::VM_TARGET_CPU,
|
||||
Self::VM_FEATURES,
|
||||
&features,
|
||||
optimizer_settings.level_back_end,
|
||||
inkwell::targets::RelocMode::PIC,
|
||||
inkwell::targets::CodeModel::Default,
|
||||
@@ -71,6 +91,21 @@ impl TargetMachine {
|
||||
}
|
||||
|
||||
/// Runs the optimization passes on `module`.
|
||||
///
|
||||
/// The pass builder's `MergeFunctions` insertion is disabled for both the stock and newyork
|
||||
/// pipelines. It can collapse `@deploy` and `@call` into a single function body when their
|
||||
/// lowered control flow becomes equivalent (e.g. a bare `invalid()` in both code sections);
|
||||
/// the resulting ELF then has two exports pointing at the same block, which triggers a
|
||||
/// non-determinism bug in polkavm-linker's reachability consistency check (the exports are
|
||||
/// stored as `Vec<usize>` and compared by order, so equal-but-differently-ordered vectors
|
||||
/// are rejected as inconsistent). Disabling the pass keeps the export targets distinct.
|
||||
///
|
||||
/// Disabling it on the stock path as well (not only for newyork) is intentional and not a
|
||||
/// size regression: the polkavm-linker performs its own function deduplication/outlining, so
|
||||
/// on the OpenZeppelin corpus dropping LLVM's `MergeFunctions` leaves real contracts slightly
|
||||
/// *smaller* rather than larger (only trivial interface blobs differ, by a few bytes). The
|
||||
/// newyork pipeline still merges where it helps, via the explicit `mergefunc` passes in its
|
||||
/// pass string.
|
||||
pub fn run_optimization_passes(
|
||||
&self,
|
||||
module: &inkwell::module::Module,
|
||||
@@ -79,11 +114,10 @@ impl TargetMachine {
|
||||
let pass_builder_options = inkwell::passes::PassBuilderOptions::create();
|
||||
pass_builder_options.set_verify_each(self.optimizer_settings.is_verify_each_enabled);
|
||||
pass_builder_options.set_debug_logging(self.optimizer_settings.is_debug_logging_enabled);
|
||||
|
||||
pass_builder_options.set_loop_unrolling(
|
||||
self.optimizer_settings.level_middle_end_size == OptimizerSettingsSizeLevel::Zero,
|
||||
);
|
||||
pass_builder_options.set_merge_functions(true);
|
||||
pass_builder_options.set_merge_functions(false);
|
||||
|
||||
module.run_passes(passes, &self.target_machine, pass_builder_options)
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user