17 Commits

Author SHA1 Message Date
Omar b71445b632 Wire up reporting to benchmarks (#195)
* Modify the structure of the `MinedBlockInformation`

* Report the step path to the watcher

* Make report format more benchmark friendly

* make report more benchmarks friendly

* Add more models to the report

* Remove corpus from the report

* Add step information to the benchmark report

* Include the contract information in the report

* Add the block information to the report

* compute metrics in each report

* Cleanup watcher from temp code
2025-10-24 02:15:29 +00:00
Omar 491c23efb3 Remove the revive network (#188)
* Remove the revive network

* Add a provider method to the `EthereumNode`

* Report the ref time and proof size for substrate chains in block information

* Remove un-needed dependency
2025-10-14 13:50:36 +00:00
Omar 74fdeb4a2e Core Benchmarking Infra (#175)
* Implement a solution for the pre-fund account limit

* Update the account pre-funding handling

* Fix the lighthouse node tracing issue

* refactor existing dt infra

* Implement the platform driver

* Wire up the cleaned up driver implementation

* Implement the core benchmarking components

* Remove some debug logging

* Fix issues in the benchmarks driver

* Implement a global concurrency limit on provider requests

* Update the concurrency limit

* Update the concurrency limit

* Cleanups

* Update the lighthouse ports

* Ignore certain tests

* Update the new geth test
2025-10-05 15:09:01 +00:00
Omar 98b62d705f Make our traits object safe and implement the ReviveDevNodePolkaVMResolc target. (#159)
* Generate schema for the metadata file

* Groundwork for dyn traits

* Make the ethereum node trait object compatible

* Allow for compilers to be created in the dyn trait

* Add more identifiers to the platform

* Implement the dyn compiler trait for compilers

* Support the dyn compiler in the builder pattern

* Introduce a geth platform

* Provide a common node implementation for substrate chains

* Add all of the platforms that we support

* Add a way to convert platform identifier into a platform

* Replace infra with the dyn infra

* Remoe all references to leader and follower

* Remove the old traits

* Remove an un-needed dependency

* Update the default values for the platforms

* Final set of renames

* Update the default values of the cli

* Update tests
2025-09-19 21:59:28 +00:00
Omar 76d6a154c1 Fix concurrency issues (#142)
* Fix the OS FD error

* Cache the compiler versions

* Allow for auto display impl in declare wrapper type macro

* Better logging and fix concurrency issues

* Fix tests

* Format

* Make the code even more concurrent
2025-08-19 06:47:36 +00:00
Omar 67d767ffde Implement storage empty assertion (#134) 2025-08-11 13:17:19 +00:00
Omar f7fbe094ec Balance assertions (#133)
* Make metadata serializable

* Refactor tests to use steps

* Add a balance assertion test step

* Test balance deserialization

* Box the test steps

* Permit size difference in step output
2025-08-11 12:11:16 +00:00
Omar 56c2fe8c0c Parallelize Cases (#109)
* Parallelize over cases

* Rename the state and driver

* Parallelize execution

* Update the default config of the tool

* Make codebase async

* Fix machete

* Fix tests & clear node directories before startup

* Cleanup the cleanup logic

* Rename geth node
2025-08-01 11:00:08 +00:00
Omar 90fb89adc0 Add a common crate (#75)
* Add a barebones common crate

* Refactor some code into the common crate

* Add a `ResolverApi` interface.

This commit adds a `ResolverApi` trait to the `format` crate that can be
implemented by any type that can act as a resolver. A resolver is able
to provide information on the chain state. This chain state could be
fresh or it could be cached (which is something that we will do in a
future PR).

This cleans up our crate graph so that `format` is not depending on the
node interactions crate for the `EthereumNode` trait.

* Cleanup the blocking executor
2025-07-24 12:42:45 +00:00
Omar 6f4aa731ab Handle exceptions (#54)
* Add support for wrapper types

* Move `FilesWithExtensionIterator` to `core::common`

* Remove unneeded use of two `HashMap`s

* Make metadata structs more typed

* Impl new_from for wrapper types

* Implement the new input handling logic

* Fix edge-case in input handling

* Ignore macro doc comment tests

* Correct comment

* Fix edge-case in deployment order

* Handle calldata better

* Allow for the use of function signatures

* Add support for exceptions

* Cached nonce allocator

* Fix tests

* Add support for address replacement

* Cleanup implementation

* Cleanup mutability

* Wire up address replacement with rest of code

* Implement caller replacement

* Switch to callframe trace for exceptions

* Add a way to skip tests if they don't match the target

* Handle values from the metadata files

* Remove address replacement

* Correct the arguments

* Remove empty impl

* Remove address replacement

* Correct the arguments

* Remove empty impl

* Fix size_requirement underflow

* Add support for wildcards in exceptions

* Fix calldata construction of single calldata

* Better handling for length in equivalency checks

* Make initial balance a constant

* Fix size_requirement underflow

* Add support for wildcards in exceptions

* Fix calldata construction of single calldata

* Better handling for length in equivalency checks

* Fix tests
2025-07-24 03:45:53 +00:00
Omar 0b97d7dc29 Support other matterlabs variables (#43)
* Introduce a custom kitchensink network

* fix formatting

* Added `--dev` to `substrate-node` arguments.

This commit adds the `--dev` argument to the `substrate-node` to allow
the chain to keep advancing as time goes own. We have found that if this
option is not added then the chain won't advance forward.

* fix clippy warning

* fix clippy warning

* Fix function selector and argument encoding

* Avoid extra buffer allocation

* Remove reliance on the web3 crate

* Update the async runtime with syntactic sugar.

* Fix tests

* Fix doc test

* Give nodes a standard way to get their alloy provider

* Add ability to get the chain_id from node

* Get kitchensink provider to use kitchensink network

* Use provider method in tests

* Add support for getting the gas limit from the node

* Add a way to get the coinbase address

* Add a way to get the block difficulty from the node

* Add a way to get block info from the node

* Expose APIs for getting the info of a specific block

* Add resolution logic for other matterlabs variables

* Fix tests

* Add comment on alternative solutions

* Change kitchensink gas limit assertion

* Remove un-needed profile config
2025-07-18 12:06:40 +00:00
Omar 14888f9767 Update the async runtime (#42)
* Update the async runtime with syntactic sugar.

* Fix doc test

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Update crates/node-interaction/src/blocking_executor.rs

Co-authored-by: xermicus <cyrill@parity.io>

* Improve the comments

* Update the release profile

---------

Co-authored-by: xermicus <cyrill@parity.io>
2025-07-15 11:19:17 +00:00
Omar 0513a4befb Use tracing for logging. (#29)
This commit updates how logging is done in the differential testing
harness to use `tracing` instead of using the `log` crate. This allows
us to be able to better associate logs with the cases being executed
which makes it easier to debug and understand what the harness is doing.
2025-07-10 07:28:16 +00:00
activecoder10 ee97b62e70 Added fetch_add_nonce method for NodeInteraction trait. Added extra logging. (#25)
* added logging

* added fetch_add_nonce method

* Added nonce for legacy transaction also

* Addressed PR comments
2025-06-18 19:43:16 +00:00
xermicus c8eb8cf7b0 the state diff method belongs to node interactions (#20)
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
2025-06-05 07:50:54 +00:00
activecoder10 ae1174febe Added basic CI workflow (#13) 2025-05-12 13:00:13 +03:00
xermicus c590fa7bfd Scaffold utility and library (#3)
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: xermicus <bigcyrill@hotmail.com>
2025-03-31 11:40:05 +02:00