* Runtime API versioning
Related to issue #11577
Add support for multiple versions of a Runtime API. The purpose is to
have one main version of the API, which is considered stable and
multiple unstable (aka staging) ones.
How it works
===========
Some methods of the API trait can be tagged with `#[api_version(N)]`
attribute where N is version number bigger than the main one. Let's call
them **staging methods** for brevity.
The implementor of the API decides which version to implement.
Example (from https://github.com/paritytech/substrate/issues/11577#issuecomment-1145347025):
```
decl_runtime_apis! {
#{api_version(10)]
trait Test {
fn something() -> Vec<u8>;
#[api_version(11)]
fn new_cool_function() -> u32;
}
}
```
```
impl_runtime_apis! {
#[api_version(11)]
impl Test for Runtime {
fn something() -> Vec<u8> { vec![1, 2, 3] }
fn new_cool_function() -> u32 {
10
}
}
}
```
Version safety checks (currently not implemented)
=================================================
By default in the API trait all staging methods has got default
implementation calling `unimplemented!()`. This is a problem because if
the developer wants to implement version 11 in the example above and
forgets to add `fn new_cool_function()` in `impl_runtime_apis!` the
runtime will crash when the function is executed.
Ideally a compilation error should be generated in such cases.
TODOs
=====
Things not working well at the moment:
[ ] Version safety check
[ ] Integration tests of `primitives/api` are messed up a bit. More
specifically `primitives/api/test/tests/decl_and_impl.rs`
[ ] Integration test covering the new functionality.
[ ] Some duplicated code
* Update primitives/api/proc-macro/src/impl_runtime_apis.rs
Code review feedback and formatting
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
* Code review feedback
Applying suggestions from @bkchr
* fmt
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Code review feedback
* dummy trait -> versioned trait
* Implement only versioned traits (not compiling)
* Remove native API calls (still not compiling)
* fmt
* Fix compilation
* Comments
* Remove unused code
* Remove native runtime tests
* Remove unused code
* Fix UI tests
* Code review feedback
* Code review feedback
* attribute_names -> common
* Rework `append_api_version`
* Code review feedback
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Code review feedback
* Code review feedback
* Code review feedback
* Use type alias for the default trait - doesn't compile
* Fixes
* Better error for `method_api_ver < trait_api_version`
* fmt
* Rework how we call runtime functions
* Update UI tests
* Fix warnings
* Fix doctests
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix formatting and small compilation errors
* Update primitives/api/proc-macro/src/impl_runtime_apis.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
* Prepare for rust stable 1.59
Besides preparing the UI tests this also adds a new script update-rust-stable.sh script for
simplifying the update of a rust stable version. This script will run all UI tests for the new
rust stable version and updating the expected output.
* Ensure we run the UI tests in CI
* use staging ci image
* More test updates
* Unignore test (#11097)
* empty commit for pipeline rerun
* empty commit for pipeline rerun
* Try to make clippy happy
* More clippy fixes
* FMT
* ci image production
Co-authored-by: alvicsam <alvicsam@gmail.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
* Update CI image to the latest rustc
* Update rustc error messages
* whitespace
* update test output
* updating compiler errors
* kid's finally learning to read and spots the TRYBUILD=overwrite message.
* undoing a little blessing
* imperfect error msg
* revert ci image to production
Co-authored-by: gilescope <gilescope@gmail.com>
* Prepare UI tests for rust 1.54
* Delete wrong_page.stderr
* CI: run with a staging CI image
* Revert "CI: run with a staging CI image"
This reverts commit 66f5b00d14b50fd9d8fbf773f7e884f380697591.
* CI: debug, again
* LOG_TARGET is only used in std
* Remove unnecessary unsafe
* Fixes
* Use correct rustc locally
* FMT
* Compile with benchmarking
* Review feedback
* Some ui tests
* I know...
* Fix wasm tests
Co-authored-by: Denis P <denis.pisarev@parity.io>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
* CI: revert me
* fix stderr
* CI: revert me
* typo
* more stderr fixes
* Revert "CI: revert me"
This reverts commit 5f47effc4965fa5c0c2a6ed92e434a6adb6b1dce.
* Revert "CI: revert me"
This reverts commit 7f785660c797b703dd36272cbe313056dd7a1858.
* tests: fix UI test so we can update CI image
* CI: remove diener installation from the script as it's installed in CI image
* tests: another fix
* tests: fix another fix
* tests: NLoEOF
* tests: another broken stderr
* chore/error: remove from str conversion and add deprecation notifications
* fixup changes
* fix test looking for gone ::Msg variant
* another test fix
* one is duplicate, the other is not, so duplicates reported are n-1
* darn spaces
Co-authored-by: Andronik Ordian <write@reusable.software>
* remove pointless doc comments of error variants without any value
* low hanging fruits (for a tall person)
* moar error type variants
* avoid the storage modules for now
They are in need of a refactor, and the pain is rather large
removing all String error and DefaultError occurences.
* chore remove pointless error generic
* fix test for mocks, add a bunch of non_exhaustive
* max line width
* test fixes due to error changes
* fin
* error outputs... again
* undo stderr adjustments
* Update client/consensus/slots/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove closure clutter
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* more error types
* introduce ApiError
* extract Mock error
* ApiError refactor
* even more error types
* the last for now
* chore unused deps
* another extraction
* reduce should panic, due to extended error messages
* error test happiness
* shift error lines by one
* doc tests
* white space
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Into -> From
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove pointless codec
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* avoid pointless self import
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Improve `mock_impl_runtime_apis!`
This adds a new attribute for functions being implemented in the
`mock_impl_runtime_apis!` macro, the `advanced` attribute. When this
attribute is given the user gets access to the `at` parameter and is
able to return a `Result`, instead of letting the macro generate this
stuff.
* Use the `at_param_name` directly
* Prevent clashing of `params`
* Make `decl_runtime_apis!` fail on methods with default implementation
Runtime api functions are not allowed to have default implementations.
This fixes this by throwing an error when we detect such a function.
* Update primitives/api/proc-macro/src/decl_runtime_apis.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update primitives/api/test/tests/ui/no_default_implementation.stderr
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* draft
* revert
* WIP
* all that remains is tests
* update Cargo.lock
* tests WIP
* WIP refactor node-template-runtime and node-runtime
* implments sc_cli::RuntimeAdapter for node_template_runtime::Runtime
* final draft
* fix update_config for subcommands
* proper AccountId decoding
* test-runtime tests
* revert
* move RuntimeAdapter to cli-utils
* use &'static str for TryFrom::<&'a str>::Error for Ss58AddressFormat
* tests
* add frame-system to sc-cli dev-dependencies
* add frame-system to sc-cli dev-dependencies
* fix ui test
* wip
* fixed inspect test
* bump impl version
* bump impl version, fixx spaces remove todos
* pallet-balances-cli, rustc for some reason cannot resolve pallet_balances_cli in node-cli 😩
* wip
* Subcommand::run takes &self
* can't believe i missed that 🤦🏾♂️
* bump wasm-bindgen for some reason
* adds key subcommand, rename generate-node-key to generate-node-id
* cargo update and crossed fingers 🤞🏽
* update ui test
* update more ui tests
* should be all good now
* revert subkey change
* revert subkey change
* adds frame-utilities-cli
* Apply suggestions from code review
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
* removes frame from sc-cli, fix license
* my editor and ci disagrees on line width
* bump spec version
* turn off default features for parity-scale-codec
* enable full_crypto feature for sp-core in cli-utils
* merge frame-utilities-cli with pallet-balances-cli
* remove full_crypto feature from sp_core in cli-utils
* bump Cargo.lock
* cli-utils -> frame-utils
* rename BlockNumber to GenericNumber, fix spaces
* fix spaces
* construct additional_signed manually
* sign test
* remove unused vars
* implement subkey with frame-utilities-cli and sc_cli
* fix moduleid test
* CI and clion disagree on line widths
* adds associated Params type to SignedExtensionProvider
* Apply suggestions from code review
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
* move some code around
* removes unneccesary generic params
* moves module_id back to frame_utilities_cli
* Apply suggestions from code review
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove print_ext
* remove MaybeDisplay from pallet_balances::Trait::Balance
* a lot of stuff tbh
* adds ExtrasParamsBuilder
* remove tests for ModuleIdCmd
* address comments from PR
* bump Cargo.lock
* manually insert key into keystore
* remove unnecessary SharedParams
* add validation to vanity pattern, remove unused arg
* remove SharedParams from Sign, Vanity, Verify
* remove SharedParams from ModuleIdCmd, remove expect from Verify, new line to Cargo.toml
* remove SharedParams from InsertCmd
* 🤦🏾♂️
* deleted prometheus.yml
* move a few things around
* fix vanity test
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api
* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa
* drop sc-service from sc-rpc
* drop sc-service from sc-consensus-aura
* drop sc-client from manual-seal and babe
* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli
* drop sc-client from bin/node and bin/node-template
* drop sc-client
* fix tests
* remove check -p sc-client from gitlab.yml
* fix warnings
* fixes ui test
* fix light client tests
* adds associated Client type to AbstractService
* adds UsageProvider to Client
* fixed ui test, again
* tried and failed to get node-cli to compile for wasm
* thanks to tomaka for helping me get node-cli to compile for wasmm
* ui test pls pas 🙏🏾
* all tests passing 🪄
* no_run documentation code
* rm -f documentation code
* ClientProvider
* fix mega trait
* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client
* adds license to sc-consensus
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* `decl_runtime_apis!` - check that a method without `changed_in` exists
This adds another check to the macro that ensures that not only methods
with `changed_in` exists, but there are also the default methods exist.
* Update primitives/api/proc-macro/src/decl_runtime_apis.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Fix test
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Implements mocking of runtime apis
This pr adds support for easily mock runtime api implementations for
tests by using the `mock_impl_runtime_apis!` macro. The syntax is
similar to `impl_runtime_apis!`. The mocked implementation automatically
implements `ApiExt`, `ApiErrorExt` and `Core` as these are required by
the runtime api traits, but not required in tests or only a subset of them.
* Fix warnings
* Update primitives/api/proc-macro/src/utils.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Review feedback
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Alexander Samusev