* sp-api: Support expanding the macro code
This pr introduces the `expander` crate to expand the generated source code into a file. This gives
better error reporting when trying to fix issues in the macro itself as Rustc will point to the line
in this file. The feature can be enabled by setting `SP_API_EXPAND=1` at compile time.
Besides that the generated code is changed to fix warnings in the exanped version.
* Fixes
* BlockId removal: refactor of runtime API
It changes the arguments of:
- `ApiExt` methods: `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`
It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* BlockId removal: refactor of runtime API - tests
- tests adjusted to new runtime API,
- some tests migrated from block number to block hash
* benchmarking-cli: BlockId(0) migrated to info().genesis_hash
`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.
* trivial: api.call(BlockId) -> api.call(Hash)
- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,
* CallApiAtParams::at changed to Block::Hash
* missed doc updated
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* ".git/.scripts/commands/fmt/fmt.sh"
* BlockId removal: Benchmark::consumed_weight
Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.
* at_hash renamed
* wrong merge fixed
* beefy worker: merged with master
* beefy: tests: missing block problem fixed
* Apply review suggestion
* fix
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
* Runtime API versioning
Related to issue #11577
Add support for multiple versions of a Runtime API. The purpose is to
have one main version of the API, which is considered stable and
multiple unstable (aka staging) ones.
How it works
===========
Some methods of the API trait can be tagged with `#[api_version(N)]`
attribute where N is version number bigger than the main one. Let's call
them **staging methods** for brevity.
The implementor of the API decides which version to implement.
Example (from https://github.com/paritytech/substrate/issues/11577#issuecomment-1145347025):
```
decl_runtime_apis! {
#{api_version(10)]
trait Test {
fn something() -> Vec<u8>;
#[api_version(11)]
fn new_cool_function() -> u32;
}
}
```
```
impl_runtime_apis! {
#[api_version(11)]
impl Test for Runtime {
fn something() -> Vec<u8> { vec![1, 2, 3] }
fn new_cool_function() -> u32 {
10
}
}
}
```
Version safety checks (currently not implemented)
=================================================
By default in the API trait all staging methods has got default
implementation calling `unimplemented!()`. This is a problem because if
the developer wants to implement version 11 in the example above and
forgets to add `fn new_cool_function()` in `impl_runtime_apis!` the
runtime will crash when the function is executed.
Ideally a compilation error should be generated in such cases.
TODOs
=====
Things not working well at the moment:
[ ] Version safety check
[ ] Integration tests of `primitives/api` are messed up a bit. More
specifically `primitives/api/test/tests/decl_and_impl.rs`
[ ] Integration test covering the new functionality.
[ ] Some duplicated code
* Update primitives/api/proc-macro/src/impl_runtime_apis.rs
Code review feedback and formatting
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
* Code review feedback
Applying suggestions from @bkchr
* fmt
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Code review feedback
* dummy trait -> versioned trait
* Implement only versioned traits (not compiling)
* Remove native API calls (still not compiling)
* fmt
* Fix compilation
* Comments
* Remove unused code
* Remove native runtime tests
* Remove unused code
* Fix UI tests
* Code review feedback
* Code review feedback
* attribute_names -> common
* Rework `append_api_version`
* Code review feedback
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Code review feedback
* Code review feedback
* Code review feedback
* Use type alias for the default trait - doesn't compile
* Fixes
* Better error for `method_api_ver < trait_api_version`
* fmt
* Rework how we call runtime functions
* Update UI tests
* Fix warnings
* Fix doctests
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix formatting and small compilation errors
* Update primitives/api/proc-macro/src/impl_runtime_apis.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
* Prepare for rust stable 1.59
Besides preparing the UI tests this also adds a new script update-rust-stable.sh script for
simplifying the update of a rust stable version. This script will run all UI tests for the new
rust stable version and updating the expected output.
* Ensure we run the UI tests in CI
* use staging ci image
* More test updates
* Unignore test (#11097)
* empty commit for pipeline rerun
* empty commit for pipeline rerun
* Try to make clippy happy
* More clippy fixes
* FMT
* ci image production
Co-authored-by: alvicsam <alvicsam@gmail.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
* Update CI image to the latest rustc
* Update rustc error messages
* whitespace
* update test output
* updating compiler errors
* kid's finally learning to read and spots the TRYBUILD=overwrite message.
* undoing a little blessing
* imperfect error msg
* revert ci image to production
Co-authored-by: gilescope <gilescope@gmail.com>
* Prepare UI tests for rust 1.54
* Delete wrong_page.stderr
* CI: run with a staging CI image
* Revert "CI: run with a staging CI image"
This reverts commit 66f5b00d14b50fd9d8fbf773f7e884f380697591.
* CI: debug, again
* LOG_TARGET is only used in std
* Remove unnecessary unsafe
* Fixes
* Use correct rustc locally
* FMT
* Compile with benchmarking
* Review feedback
* Some ui tests
* I know...
* Fix wasm tests
Co-authored-by: Denis P <denis.pisarev@parity.io>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
* CI: revert me
* fix stderr
* CI: revert me
* typo
* more stderr fixes
* Revert "CI: revert me"
This reverts commit 5f47effc4965fa5c0c2a6ed92e434a6adb6b1dce.
* Revert "CI: revert me"
This reverts commit 7f785660c797b703dd36272cbe313056dd7a1858.
* tests: fix UI test so we can update CI image
* CI: remove diener installation from the script as it's installed in CI image
* tests: another fix
* tests: fix another fix
* tests: NLoEOF
* tests: another broken stderr
* chore/error: remove from str conversion and add deprecation notifications
* fixup changes
* fix test looking for gone ::Msg variant
* another test fix
* one is duplicate, the other is not, so duplicates reported are n-1
* darn spaces
Co-authored-by: Andronik Ordian <write@reusable.software>
* remove pointless doc comments of error variants without any value
* low hanging fruits (for a tall person)
* moar error type variants
* avoid the storage modules for now
They are in need of a refactor, and the pain is rather large
removing all String error and DefaultError occurences.
* chore remove pointless error generic
* fix test for mocks, add a bunch of non_exhaustive
* max line width
* test fixes due to error changes
* fin
* error outputs... again
* undo stderr adjustments
* Update client/consensus/slots/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove closure clutter
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* more error types
* introduce ApiError
* extract Mock error
* ApiError refactor
* even more error types
* the last for now
* chore unused deps
* another extraction
* reduce should panic, due to extended error messages
* error test happiness
* shift error lines by one
* doc tests
* white space
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Into -> From
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove pointless codec
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* avoid pointless self import
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Improve `mock_impl_runtime_apis!`
This adds a new attribute for functions being implemented in the
`mock_impl_runtime_apis!` macro, the `advanced` attribute. When this
attribute is given the user gets access to the `at` parameter and is
able to return a `Result`, instead of letting the macro generate this
stuff.
* Use the `at_param_name` directly
* Prevent clashing of `params`
* Make `decl_runtime_apis!` fail on methods with default implementation
Runtime api functions are not allowed to have default implementations.
This fixes this by throwing an error when we detect such a function.
* Update primitives/api/proc-macro/src/decl_runtime_apis.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update primitives/api/test/tests/ui/no_default_implementation.stderr
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* draft
* revert
* WIP
* all that remains is tests
* update Cargo.lock
* tests WIP
* WIP refactor node-template-runtime and node-runtime
* implments sc_cli::RuntimeAdapter for node_template_runtime::Runtime
* final draft
* fix update_config for subcommands
* proper AccountId decoding
* test-runtime tests
* revert
* move RuntimeAdapter to cli-utils
* use &'static str for TryFrom::<&'a str>::Error for Ss58AddressFormat
* tests
* add frame-system to sc-cli dev-dependencies
* add frame-system to sc-cli dev-dependencies
* fix ui test
* wip
* fixed inspect test
* bump impl version
* bump impl version, fixx spaces remove todos
* pallet-balances-cli, rustc for some reason cannot resolve pallet_balances_cli in node-cli 😩
* wip
* Subcommand::run takes &self
* can't believe i missed that 🤦🏾♂️
* bump wasm-bindgen for some reason
* adds key subcommand, rename generate-node-key to generate-node-id
* cargo update and crossed fingers 🤞🏽
* update ui test
* update more ui tests
* should be all good now
* revert subkey change
* revert subkey change
* adds frame-utilities-cli
* Apply suggestions from code review
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
* removes frame from sc-cli, fix license
* my editor and ci disagrees on line width
* bump spec version
* turn off default features for parity-scale-codec
* enable full_crypto feature for sp-core in cli-utils
* merge frame-utilities-cli with pallet-balances-cli
* remove full_crypto feature from sp_core in cli-utils
* bump Cargo.lock
* cli-utils -> frame-utils
* rename BlockNumber to GenericNumber, fix spaces
* fix spaces
* construct additional_signed manually
* sign test
* remove unused vars
* implement subkey with frame-utilities-cli and sc_cli
* fix moduleid test
* CI and clion disagree on line widths
* adds associated Params type to SignedExtensionProvider
* Apply suggestions from code review
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
* move some code around
* removes unneccesary generic params
* moves module_id back to frame_utilities_cli
* Apply suggestions from code review
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove print_ext
* remove MaybeDisplay from pallet_balances::Trait::Balance
* a lot of stuff tbh
* adds ExtrasParamsBuilder
* remove tests for ModuleIdCmd
* address comments from PR
* bump Cargo.lock
* manually insert key into keystore
* remove unnecessary SharedParams
* add validation to vanity pattern, remove unused arg
* remove SharedParams from Sign, Vanity, Verify
* remove SharedParams from ModuleIdCmd, remove expect from Verify, new line to Cargo.toml
* remove SharedParams from InsertCmd
* 🤦🏾♂️
* deleted prometheus.yml
* move a few things around
* fix vanity test
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api
* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa
* drop sc-service from sc-rpc
* drop sc-service from sc-consensus-aura
* drop sc-client from manual-seal and babe
* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli
* drop sc-client from bin/node and bin/node-template
* drop sc-client
* fix tests
* remove check -p sc-client from gitlab.yml
* fix warnings
* fixes ui test
* fix light client tests
* adds associated Client type to AbstractService
* adds UsageProvider to Client
* fixed ui test, again
* tried and failed to get node-cli to compile for wasm
* thanks to tomaka for helping me get node-cli to compile for wasmm
* ui test pls pas 🙏🏾
* all tests passing 🪄
* no_run documentation code
* rm -f documentation code
* ClientProvider
* fix mega trait
* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client
* adds license to sc-consensus
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* `decl_runtime_apis!` - check that a method without `changed_in` exists
This adds another check to the macro that ensures that not only methods
with `changed_in` exists, but there are also the default methods exist.
* Update primitives/api/proc-macro/src/decl_runtime_apis.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Fix test
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Implements mocking of runtime apis
This pr adds support for easily mock runtime api implementations for
tests by using the `mock_impl_runtime_apis!` macro. The syntax is
similar to `impl_runtime_apis!`. The mocked implementation automatically
implements `ApiExt`, `ApiErrorExt` and `Core` as these are required by
the runtime api traits, but not required in tests or only a subset of them.
* Fix warnings
* Update primitives/api/proc-macro/src/utils.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Review feedback
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Bastian Köcher