Commit Graph

6 Commits

Author SHA1 Message Date
Omar d93824d973 Updated Reporting Infrastructure (#151)
* Remove the old reporting infra

* Use the Test struct more in the code

* Implement the initial set of reporter events

* Add more runner events to the reporter and refine the structure

* Add reporting infra for reporting ignored tests

* Update report to use better map data structures

* Add case status information to the report

* Integrate the reporting infrastructure with the
CLI reporter used by the program.

* Include contract compilation information in report

* Cleanup report model

* Add information on the deployed contracts
2025-08-25 11:16:09 +00:00
Omar 76d6a154c1 Fix concurrency issues (#142)
* Fix the OS FD error

* Cache the compiler versions

* Allow for auto display impl in declare wrapper type macro

* Better logging and fix concurrency issues

* Fix tests

* Format

* Make the code even more concurrent
2025-08-19 06:47:36 +00:00
James Wilson 09d56f5177 Redo how we parse and use modes (#125)
* WIP redo how we parse and use modes

* test expanding, too

* WIP integrate new Mode/ParsedMode into rest of code

* First pass integrated new mode bits

* fmt

* clippy

* Remove mode we no longer support from test metadata

* Address nits

* Add ability for compiler to opt out if it can't work with some Mode/version

* Elide viaIR input if compiler does not support it

* Improve test output a little; string modes and list ignored tests

* Move Mode to common crate

* constants.mod, and Display for CaseIdx to use it

* fmt

* Rename ModePipeline::E/Y

* Re-arrange Mode things; ParsedMode in format and Mode etc in common

* Move compile check to prepare_tests

* Remove now-unused deps

* clippy nits

* Update fallback tx weights to avoid out of gas errors

* Update kitchensink weights too and fmt

* Bump default geth timeout to 10s

* 30s timeout

* Improve geth stdout logging on failure

* fix line logging

* remove --networkid and arg, back to 5s timeout for geth
2025-08-16 11:38:17 +00:00
Omar f51693cb9f Support multiple compiler versions (#92)
* Allow for downloader to use version requirements.

We will soon add support for the compiler version requirement from the
metadata files to be honored. The compiler version is specified in the
solc modes section of the file and its specified as a `VersionReq` and
not as a version.

Therefore, we need to have the ability to honor this version requirement
and find the best version that satisfies the requirement.

* Request `VersionOrRequirement` in compiler interface

* Honor the compiler version requirement in metadata

This commit honors the compiler version requirement listed in the solc
modes of the metadata file. If this version requirement is provided then
it overrides what was passed in the CLI. Otherwise, the CLI version will
be used.

* Make compiler IO completely generic.

Before this commit, the types that were used for the compiler input and
output were the resolc compiler types which was a leaky abstraction as
we have traits to abstract the compilers away but we expose their
internal types out to other crates.

This commit did the following:
1. Made the compiler IO types fully generic so that all of the logic for
   constructing the map of compiled contracts is all done by the
   compiler implementation and not by the consuming code.
2. Changed the input types used for Solc to be the forge standard JSON
   types for Solc instead of resolc.

* Fix machete

* Add resolc to CI

* Add resolc to CI

* Add resolc to CI

* Add resolc to CI
2025-07-30 04:56:23 +00:00
Omar 0513a4befb Use tracing for logging. (#29)
This commit updates how logging is done in the differential testing
harness to use `tracing` instead of using the `log` crate. This allows
us to be able to better associate logs with the cases being executed
which makes it easier to debug and understand what the harness is doing.
2025-07-10 07:28:16 +00:00
xermicus 10bfaed461 Implement basic reporting facility (#18)
* wip

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

* save to file after all tasks done

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

* error out early if the workdir does not exist

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

* the compiler statistics

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

* allow compiler statistics per implementation

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

* save compiler problems

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

* add flag whether to extract compiler errors

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

* whitespace

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

---------

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
2025-05-23 17:15:04 +00:00