* sp-api: Make the generated code act based on `std` in `sp-api`
Instead of letting the macro generate code that checks if the `std` feature is enabled, it will now
generate code that checks if the `std` feature is enabled for the `sp-api` crate. The old
implementation basically required that the crate in which the macro was used, had a `std` feature.
Now we don't have this requirement anymore and act accordingly the feature in `sp-api` directly.
* Missing feature!
---------
Co-authored-by: parity-processbot <>
* globally upgrade quote to latest 1.0.x (1.0.26)
* globally upgrade syn to final 1.0.x version (1.0.109)
* globally upgrade proc-macro2 to 1.0.56
* upgrade to syn v2.0.13 and fix everything except NestedMeta
* fix parse nested metadata code in decl_runtime_apis.rs
* Port more stuff to syn 2.0
* Make the rest compile
* Ignore error
* update to syn 2.0.14
---------
Co-authored-by: Bastian Köcher <info@kchr.de>
* impl_runtime_apis: Generate getters for `metadata_at` functions
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* runtime: Implement new `Metadata` runtime trait
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* runtime: Move `metadata_at` functions to construct_runtime macro
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* contruct_runtime: Use `OpaqueMetadata` from hidden imports
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Adjust testing
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/tests: Add tests for the new API
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Helper to extract documentation literals
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Helper to filter all `cfg` attributes
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Generate documentation getters for metadata
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Avoid trait collision with snake case methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* proc-macro/tests: Check doc getters
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Generate metadata for runtime methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/api: Export scale-info and frame-metadata
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Generate metadata for runtime traits
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/runtime: Expose metadata v15 internally
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* test: Use metadata v15 from `lexnv/md_v15_test` branch
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Generate crate access one module up
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame: Implement `runtime_metadata` for mocks and tests
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Fix warnings
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Add no-docs flag
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame: Adjust more tests
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/tests: Check runtime metadata correctness
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/benchmarking: Adjust benchmarks
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/benchmarks: Adjust more benchmarks
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/api: Fix clippy
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/proc-macro: Generate runtime metadata on the `decl_runtime_apis`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame: Abuse Deref to resolve `runtime_metadata`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Revert "frame: Implement `runtime_metadata` for mocks and tests"
This reverts commit b7f41aa189218589392a6e713ea9488e93c4db45.
Revert "frame: Adjust more tests"
This reverts commit 3cba5982c7f45552e76335e96c430aecbc42d8c6.
Revert "frame/benchmarking: Adjust benchmarks"
This reverts commit 60b382ada486c791ffceeb65da587e949b90ec5d.
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Revert "frame/benchmarks: Adjust more benchmarks"
This reverts commit eb75c477179b1a27347a5554c5732ef26a00d7e8.
* primitives/proc-macro: Remove unused imports and function
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/support: Adjust runtime metadata test
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/tests: Remove doc getter test
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/support: Enable `no-metadata-docs` feature from `sp-api`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/tests: Add `TypeInfo` for test::extrinsic
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/api: Expose scale-info and frame-metadata
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update frame-metadata to include v15
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Fix merge conflicts
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/metadata_ir: Add IR for runtime API metadata
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/metadata_ir: Convert IR to V15
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/api: Collect IR metadata for runtime API
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/api: Move `metadata_ir` from frame/support
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/tests: Adjust testing
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame/tests: Adjust `metadata_versions` test
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/runtime_metadata: Exclude default type parameters from methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/src/metadata_ir/types.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/src/metadata_ir/mod.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/utils.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* primitives: Fix build
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives/metadata-ir: Move IR to dedicated crate
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives: Reexport metadata-ir and frame-metadata
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* frame: Use apis field instead of runtime
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Better documentation for the `Deref` abstraction
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* ui-tests: Check empty `impl_runtime_apis`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives: Remove unneeded bounds on generic params
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives: Rename `collect_where_bounds` to `get_argument_type_param`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives: Generate crate access per fn call
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Revert "primitives: Remove unneeded bounds on generic params"
This reverts commit 5178e38cf21cfb481156eefd628d62989201d59a.
* metadata-ir: Add no-std
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* primitives: Adjust where bounds
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Change `frame-metadata` branch to "origin/main"
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update to `main` from origin
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update frame-metadata to crates.io v15.1
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Revert "ui-tests: Check empty `impl_runtime_apis`"
This reverts commit cf78a7190ad9cba3c3bb2e78dc3d0dc382b2fea9.
* Move ui test to primitives/ui
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update frame/support/test/tests/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Update primitives/api/proc-macro/src/runtime_metadata.rs
Co-authored-by: Bastian Köcher <git@kchr.de>
* Test already covered by `empty_impl_runtime_apis_call.stderr`
This reverts commit 3bafb294cbe9745569bf5e5a1a2e6b4a4c1aadc5.
* Retriger CI
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Import `TokenStream` as `TokenStream2`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
* sp-api: Support expanding the macro code
This pr introduces the `expander` crate to expand the generated source code into a file. This gives
better error reporting when trying to fix issues in the macro itself as Rustc will point to the line
in this file. The feature can be enabled by setting `SP_API_EXPAND=1` at compile time.
Besides that the generated code is changed to fix warnings in the exanped version.
* Fixes
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* BlockId removal: refactor of runtime API
It changes the arguments of:
- `ApiExt` methods: `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`
It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* BlockId removal: refactor of runtime API - tests
- tests adjusted to new runtime API,
- some tests migrated from block number to block hash
* benchmarking-cli: BlockId(0) migrated to info().genesis_hash
`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.
* trivial: api.call(BlockId) -> api.call(Hash)
- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,
* CallApiAtParams::at changed to Block::Hash
* missed doc updated
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* ".git/.scripts/commands/fmt/fmt.sh"
* BlockId removal: Benchmark::consumed_weight
Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.
* at_hash renamed
* wrong merge fixed
* beefy worker: merged with master
* beefy: tests: missing block problem fixed
* Apply review suggestion
* fix
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
The runtime api implementation contained invalid unsafe trait bounds. `Sync` was never correct there
and `Send` should have not been "force implemented".
* Runtime API versioning
Related to issue #11577
Add support for multiple versions of a Runtime API. The purpose is to
have one main version of the API, which is considered stable and
multiple unstable (aka staging) ones.
How it works
===========
Some methods of the API trait can be tagged with `#[api_version(N)]`
attribute where N is version number bigger than the main one. Let's call
them **staging methods** for brevity.
The implementor of the API decides which version to implement.
Example (from https://github.com/paritytech/substrate/issues/11577#issuecomment-1145347025):
```
decl_runtime_apis! {
#{api_version(10)]
trait Test {
fn something() -> Vec<u8>;
#[api_version(11)]
fn new_cool_function() -> u32;
}
}
```
```
impl_runtime_apis! {
#[api_version(11)]
impl Test for Runtime {
fn something() -> Vec<u8> { vec![1, 2, 3] }
fn new_cool_function() -> u32 {
10
}
}
}
```
Version safety checks (currently not implemented)
=================================================
By default in the API trait all staging methods has got default
implementation calling `unimplemented!()`. This is a problem because if
the developer wants to implement version 11 in the example above and
forgets to add `fn new_cool_function()` in `impl_runtime_apis!` the
runtime will crash when the function is executed.
Ideally a compilation error should be generated in such cases.
TODOs
=====
Things not working well at the moment:
[ ] Version safety check
[ ] Integration tests of `primitives/api` are messed up a bit. More
specifically `primitives/api/test/tests/decl_and_impl.rs`
[ ] Integration test covering the new functionality.
[ ] Some duplicated code
* Update primitives/api/proc-macro/src/impl_runtime_apis.rs
Code review feedback and formatting
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
* Code review feedback
Applying suggestions from @bkchr
* fmt
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Code review feedback
* dummy trait -> versioned trait
* Implement only versioned traits (not compiling)
* Remove native API calls (still not compiling)
* fmt
* Fix compilation
* Comments
* Remove unused code
* Remove native runtime tests
* Remove unused code
* Fix UI tests
* Code review feedback
* Code review feedback
* attribute_names -> common
* Rework `append_api_version`
* Code review feedback
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Code review feedback
* Code review feedback
* Code review feedback
* Use type alias for the default trait - doesn't compile
* Fixes
* Better error for `method_api_ver < trait_api_version`
* fmt
* Rework how we call runtime functions
* Update UI tests
* Fix warnings
* Fix doctests
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix formatting and small compilation errors
* Update primitives/api/proc-macro/src/impl_runtime_apis.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Ensure that the sp api macros don't use functions without providing the full path to the function.
This hygiene ensures that we don't actually try to call a method of an imported trait for example.
* Upgraded dependencies
* Adapting code to scale v3
* Empty commit to trigger CI
* Triggering CI
* Fixing UI test
* Remove superfluous dev-dep added by #9228
* Cryout for CI
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Do not call `initialize_block` before any runtime api
Before this change we always called `initialize_block` before calling
into the runtime. There was already support with `skip_initialize` to skip
the initialization. Almost no runtime_api requires that
`initialize_block` is called before. Actually this only leads to higher
execution times most of the time, because all runtime modules are
initialized and this is especially expensive when the block contained a
runtime upgrade.
TLDR: Do not call `initialize_block` before calling a runtime api.
* Change `validate_transaction` interface
* Fix rpc test
* Fixes and comments
* Some docs
* Make offchain indexing work
This fixes some bugs with offchain indexing to make it actually working ;)
* Fix tests
* Fix browser build
* Update client/db/src/offchain.rs
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* Remove seperation between prefix and key
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
* chore/error: remove from str conversion and add deprecation notifications
* fixup changes
* fix test looking for gone ::Msg variant
* another test fix
* one is duplicate, the other is not, so duplicates reported are n-1
* darn spaces
Co-authored-by: Andronik Ordian <write@reusable.software>
* remove pointless doc comments of error variants without any value
* low hanging fruits (for a tall person)
* moar error type variants
* avoid the storage modules for now
They are in need of a refactor, and the pain is rather large
removing all String error and DefaultError occurences.
* chore remove pointless error generic
* fix test for mocks, add a bunch of non_exhaustive
* max line width
* test fixes due to error changes
* fin
* error outputs... again
* undo stderr adjustments
* Update client/consensus/slots/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove closure clutter
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* more error types
* introduce ApiError
* extract Mock error
* ApiError refactor
* even more error types
* the last for now
* chore unused deps
* another extraction
* reduce should panic, due to extended error messages
* error test happiness
* shift error lines by one
* doc tests
* white space
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Into -> From
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove pointless codec
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* avoid pointless self import
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Make `decl_runtime_apis!` fail on methods with default implementation
Runtime api functions are not allowed to have default implementations.
This fixes this by throwing an error when we detect such a function.
* Update primitives/api/proc-macro/src/decl_runtime_apis.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update primitives/api/test/tests/ui/no_default_implementation.stderr
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* `decl_runtime_apis!` - check that a method without `changed_in` exists
This adds another check to the macro that ensures that not only methods
with `changed_in` exists, but there are also the default methods exist.
* Update primitives/api/proc-macro/src/decl_runtime_apis.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Fix test
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Implements mocking of runtime apis
This pr adds support for easily mock runtime api implementations for
tests by using the `mock_impl_runtime_apis!` macro. The syntax is
similar to `impl_runtime_apis!`. The mocked implementation automatically
implements `ApiExt`, `ApiErrorExt` and `Core` as these are required by
the runtime api traits, but not required in tests or only a subset of them.
* Fix warnings
* Update primitives/api/proc-macro/src/utils.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Review feedback
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Bastian Köcher