Runtime API versioning (#11779)

* Runtime API versioning

Related to issue #11577

Add support for multiple versions of a Runtime API. The purpose is to
have one main version of the API, which is considered stable and
multiple unstable (aka staging) ones.

How it works
===========
Some methods of the API trait can be tagged with `#[api_version(N)]`
attribute where N is version number bigger than the main one. Let's call
them **staging methods** for brevity.

The implementor of the API decides which version to implement.

Example (from https://github.com/paritytech/substrate/issues/11577#issuecomment-1145347025):

```
decl_runtime_apis! {
    #{api_version(10)]
    trait Test {
         fn something() -> Vec<u8>;
         #[api_version(11)]
         fn new_cool_function() -> u32;
    }
}
```

```
impl_runtime_apis! {
    #[api_version(11)]
    impl Test for Runtime {
         fn something() -> Vec<u8> { vec![1, 2, 3] }

         fn new_cool_function() -> u32 {
             10
         }
    }
}
```

Version safety checks (currently not implemented)
=================================================
By default in the API trait all staging methods has got default
implementation calling `unimplemented!()`. This is a problem because if
the developer wants to implement version 11 in the example above and
forgets to add `fn new_cool_function()` in `impl_runtime_apis!` the
runtime will crash when the function is executed.

Ideally a compilation error should be generated in such cases.

TODOs
=====

Things not working well at the moment:
[ ] Version safety check
[ ] Integration tests of `primitives/api` are messed up a bit. More
specifically `primitives/api/test/tests/decl_and_impl.rs`
[ ] Integration test covering the new functionality.
[ ] Some duplicated code

* Update primitives/api/proc-macro/src/impl_runtime_apis.rs

Code review feedback and formatting

Co-authored-by: asynchronous rob <rphmeier@gmail.com>

* Code review feedback

Applying suggestions from @bkchr

* fmt

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Code review feedback

* dummy trait -> versioned trait

* Implement only versioned traits (not compiling)

* Remove native API calls (still not compiling)

* fmt

* Fix compilation

* Comments

* Remove unused code

* Remove native runtime tests

* Remove unused code

* Fix UI tests

* Code review feedback

* Code review feedback

* attribute_names -> common

* Rework `append_api_version`

* Code review feedback

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Code review feedback

* Code review feedback

* Code review feedback

* Use type alias for the default trait - doesn't compile

* Fixes

* Better error for `method_api_ver < trait_api_version`

* fmt

* Rework how we call runtime functions

* Update UI tests

* Fix warnings

* Fix doctests

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix formatting and small compilation errors

* Update primitives/api/proc-macro/src/impl_runtime_apis.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
This commit is contained in:
Tsvetomir Dimitrov
2022-08-13 14:56:40 +03:00
committed by GitHub
parent 8e927daa77
commit 2bff2f84e3
32 changed files with 944 additions and 862 deletions
@@ -15,49 +15,6 @@ note: type in trait
= note: expected fn pointer `fn(u64)`
found fn pointer `fn(std::string::String)`
error[E0053]: method `Api_test_runtime_api_impl` has an incompatible type for trait
--> tests/ui/impl_incorrect_method_signature.rs:17:1
|
17 | sp_api::impl_runtime_apis! {
| -^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| _expected `u64`, found struct `std::string::String`
| |
18 | | impl self::Api<Block> for Runtime {
19 | | fn test(data: String) {}
20 | | }
... |
32 | | }
33 | | }
| |_- help: change the parameter type to match the trait: `std::option::Option<u64>`
|
note: type in trait
--> tests/ui/impl_incorrect_method_signature.rs:11:1
|
11 | / sp_api::decl_runtime_apis! {
12 | | pub trait Api {
13 | | fn test(data: u64);
14 | | }
15 | | }
| |_^
= note: expected fn pointer `fn(&RuntimeApiImpl<__SR_API_BLOCK__, RuntimeApiImplCall>, &BlockId<__SR_API_BLOCK__>, ExecutionContext, std::option::Option<u64>, Vec<_>) -> Result<_, _>`
found fn pointer `fn(&RuntimeApiImpl<__SR_API_BLOCK__, RuntimeApiImplCall>, &BlockId<__SR_API_BLOCK__>, ExecutionContext, std::option::Option<std::string::String>, Vec<_>) -> Result<_, _>`
= note: this error originates in the macro `sp_api::impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0308]: mismatched types
--> tests/ui/impl_incorrect_method_signature.rs:17:1
|
17 | / sp_api::impl_runtime_apis! {
18 | | impl self::Api<Block> for Runtime {
19 | | fn test(data: String) {}
20 | | }
... |
32 | | }
33 | | }
| |_^ expected `u64`, found struct `std::string::String`
|
= note: this error originates in the macro `sp_api::impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0308]: mismatched types
--> tests/ui/impl_incorrect_method_signature.rs:19:11
|
@@ -0,0 +1,40 @@
use sp_runtime::traits::{Block as BlockT, GetNodeBlockType};
use substrate_test_runtime_client::runtime::Block;
struct Runtime {}
impl GetNodeBlockType for Runtime {
type NodeBlock = Block;
}
sp_api::decl_runtime_apis! {
#[api_version(2)]
pub trait Api {
fn test1();
fn test2();
#[api_version(3)]
fn test3();
}
}
sp_api::impl_runtime_apis! {
#[api_version(4)]
impl self::Api<Block> for Runtime {
fn test1() {}
fn test2() {}
fn test3() {}
}
impl sp_api::Core<Block> for Runtime {
fn version() -> sp_version::RuntimeVersion {
unimplemented!()
}
fn execute_block(_: Block) {
unimplemented!()
}
fn initialize_block(_: &<Block as BlockT>::Header) {
unimplemented!()
}
}
}
fn main() {}
@@ -0,0 +1,14 @@
error[E0433]: failed to resolve: could not find `ApiV4` in `runtime_decl_for_Api`
--> tests/ui/impl_missing_version.rs:21:13
|
21 | impl self::Api<Block> for Runtime {
| ^^^ could not find `ApiV4` in `runtime_decl_for_Api`
error[E0405]: cannot find trait `ApiV4` in module `self::runtime_decl_for_Api`
--> tests/ui/impl_missing_version.rs:21:13
|
11 | pub trait Api {
| ------------- similarly named trait `ApiV2` defined here
...
21 | impl self::Api<Block> for Runtime {
| ^^^ help: a trait with a similar name exists: `ApiV2`
@@ -1,5 +1,5 @@
error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
--> $DIR/invalid_api_version.rs:2:4
--> tests/ui/invalid_api_version_1.rs:2:2
|
2 | #[api_version]
| ^^^^^^^^^^^
| ^
@@ -1,5 +1,5 @@
error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
--> $DIR/invalid_api_version_2.rs:2:4
--> tests/ui/invalid_api_version_2.rs:2:2
|
2 | #[api_version("1")]
| ^^^^^^^^^^^
| ^
@@ -1,5 +1,5 @@
error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
--> $DIR/invalid_api_version_3.rs:2:4
--> tests/ui/invalid_api_version_3.rs:2:2
|
2 | #[api_version()]
| ^^^^^^^^^^^
| ^
@@ -0,0 +1,8 @@
sp_api::decl_runtime_apis! {
pub trait Api {
#[api_version("1")]
fn test(data: u64);
}
}
fn main() {}
@@ -0,0 +1,5 @@
error: Unexpected `api_version` attribute. The supported format is `api_version(1)`
--> tests/ui/invalid_api_version_4.rs:3:3
|
3 | #[api_version("1")]
| ^
@@ -0,0 +1,9 @@
sp_api::decl_runtime_apis! {
#[api_version(2)]
pub trait Api {
#[api_version(1)]
fn test(data: u64);
}
}
fn main() {}
@@ -0,0 +1,11 @@
error: Method version `1` is older than (or equal to) trait version `2`.Methods can't define versions older than the trait version.
--> tests/ui/method_ver_lower_than_trait_ver.rs:4:3
|
4 | #[api_version(1)]
| ^
error: Trait version is set here.
--> tests/ui/method_ver_lower_than_trait_ver.rs:2:2
|
2 | #[api_version(2)]
| ^
@@ -0,0 +1,39 @@
use sp_runtime::traits::{Block as BlockT, GetNodeBlockType};
use substrate_test_runtime_client::runtime::Block;
struct Runtime {}
impl GetNodeBlockType for Runtime {
type NodeBlock = Block;
}
sp_api::decl_runtime_apis! {
#[api_version(2)]
pub trait Api {
fn test1();
fn test2();
#[api_version(3)]
fn test3();
}
}
sp_api::impl_runtime_apis! {
#[api_version(3)]
impl self::Api<Block> for Runtime {
fn test1() {}
fn test2() {}
}
impl sp_api::Core<Block> for Runtime {
fn version() -> sp_version::RuntimeVersion {
unimplemented!()
}
fn execute_block(_: Block) {
unimplemented!()
}
fn initialize_block(_: &<Block as BlockT>::Header) {
unimplemented!()
}
}
}
fn main() {}
@@ -0,0 +1,8 @@
error[E0046]: not all trait items implemented, missing: `test3`
--> tests/ui/missing_versioned_method.rs:21:2
|
15 | fn test3();
| ----------- `test3` from trait
...
21 | impl self::Api<Block> for Runtime {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing `test3` in implementation
@@ -0,0 +1,42 @@
use sp_runtime::traits::{Block as BlockT, GetNodeBlockType};
use substrate_test_runtime_client::runtime::Block;
struct Runtime {}
impl GetNodeBlockType for Runtime {
type NodeBlock = Block;
}
sp_api::decl_runtime_apis! {
#[api_version(2)]
pub trait Api {
fn test1();
fn test2();
#[api_version(3)]
fn test3();
#[api_version(4)]
fn test4();
}
}
sp_api::impl_runtime_apis! {
#[api_version(4)]
impl self::Api<Block> for Runtime {
fn test1() {}
fn test2() {}
fn test4() {}
}
impl sp_api::Core<Block> for Runtime {
fn version() -> sp_version::RuntimeVersion {
unimplemented!()
}
fn execute_block(_: Block) {
unimplemented!()
}
fn initialize_block(_: &<Block as BlockT>::Header) {
unimplemented!()
}
}
}
fn main() {}
@@ -0,0 +1,8 @@
error[E0046]: not all trait items implemented, missing: `test3`
--> tests/ui/missing_versioned_method_multiple_vers.rs:23:2
|
15 | fn test3();
| ----------- `test3` from trait
...
23 | impl self::Api<Block> for Runtime {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing `test3` in implementation
@@ -12,7 +12,7 @@ struct MockApi;
sp_api::mock_impl_runtime_apis! {
impl Api<Block> for MockApi {
#[advanced]
fn test(&self, _: BlockId<Block>) -> Result<sp_core::NativeOrEncoded<()>, ApiError> {
fn test(&self, _: BlockId<Block>) -> Result<(), ApiError> {
Ok(().into())
}
}
@@ -1,10 +1,10 @@
error: `BlockId` needs to be taken by reference and not by value!
--> $DIR/mock_advanced_block_id_by_value.rs:12:1
--> tests/ui/mock_advanced_block_id_by_value.rs:12:1
|
12 | / sp_api::mock_impl_runtime_apis! {
13 | | impl Api<Block> for MockApi {
14 | | #[advanced]
15 | | fn test(&self, _: BlockId<Block>) -> Result<sp_core::NativeOrEncoded<()>, ApiError> {
15 | | fn test(&self, _: BlockId<Block>) -> Result<(), ApiError> {
... |
18 | | }
19 | | }
@@ -12,7 +12,7 @@ struct MockApi;
sp_api::mock_impl_runtime_apis! {
impl Api<Block> for MockApi {
#[advanced]
fn test(&self) -> Result<sp_core::NativeOrEncoded<()>, ApiError> {
fn test(&self) -> Result<(), ApiError> {
Ok(().into())
}
}
@@ -1,5 +1,5 @@
error: If using the `advanced` attribute, it is required that the function takes at least one argument, the `BlockId`.
--> $DIR/mock_advanced_missing_blockid.rs:15:3
--> tests/ui/mock_advanced_missing_blockid.rs:15:3
|
15 | fn test(&self) -> Result<sp_core::NativeOrEncoded<()>, ApiError> {
15 | fn test(&self) -> Result<(), ApiError> {
| ^^
@@ -10,62 +10,80 @@ error: Only `&self` is supported!
16 | fn test2(&mut self, data: u64) {}
| ^
error[E0053]: method `Api_test_runtime_api_impl` has an incompatible type for trait
error[E0050]: method `test` has 2 parameters but the declaration in trait `Api::test` has 3
--> tests/ui/mock_only_self_reference.rs:12:1
|
12 | sp_api::mock_impl_runtime_apis! {
| -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| _expected `u64`, found `()`
| |
3 | / sp_api::decl_runtime_apis! {
4 | | pub trait Api {
5 | | fn test(data: u64);
| |_________________________- trait requires 3 parameters
...
12 | / sp_api::mock_impl_runtime_apis! {
13 | | impl Api<Block> for MockApi {
14 | | fn test(self, data: u64) {}
15 | |
16 | | fn test2(&mut self, data: u64) {}
17 | | }
18 | | }
| |_- help: change the parameter type to match the trait: `Option<u64>`
| |_^ expected 3 parameters, found 2
|
note: type in trait
--> tests/ui/mock_only_self_reference.rs:3:1
|
3 | / sp_api::decl_runtime_apis! {
4 | | pub trait Api {
5 | | fn test(data: u64);
6 | | fn test2(data: u64);
7 | | }
8 | | }
| |_^
= note: expected fn pointer `fn(&MockApi, &BlockId<sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<u64, sp_runtime::traits::BlakeTwo256>, Extrinsic>>, ExecutionContext, Option<u64>, Vec<_>) -> Result<_, _>`
found fn pointer `fn(&MockApi, &BlockId<sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<u64, sp_runtime::traits::BlakeTwo256>, Extrinsic>>, ExecutionContext, Option<()>, Vec<_>) -> Result<_, _>`
= note: this error originates in the macro `sp_api::mock_impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0053]: method `Api_test2_runtime_api_impl` has an incompatible type for trait
error[E0050]: method `test2` has 2 parameters but the declaration in trait `Api::test2` has 3
--> tests/ui/mock_only_self_reference.rs:12:1
|
12 | sp_api::mock_impl_runtime_apis! {
| -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| _expected `u64`, found `()`
| |
3 | / sp_api::decl_runtime_apis! {
4 | | pub trait Api {
5 | | fn test(data: u64);
6 | | fn test2(data: u64);
| |__________________________- trait requires 3 parameters
...
12 | / sp_api::mock_impl_runtime_apis! {
13 | | impl Api<Block> for MockApi {
14 | | fn test(self, data: u64) {}
15 | |
16 | | fn test2(&mut self, data: u64) {}
17 | | }
18 | | }
| |_- help: change the parameter type to match the trait: `Option<u64>`
| |_^ expected 3 parameters, found 2
|
note: type in trait
--> tests/ui/mock_only_self_reference.rs:3:1
= note: this error originates in the macro `sp_api::mock_impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0050]: method `test_with_context` has 3 parameters but the declaration in trait `Api::test_with_context` has 4
--> tests/ui/mock_only_self_reference.rs:12:1
|
3 | / sp_api::decl_runtime_apis! {
4 | | pub trait Api {
5 | | fn test(data: u64);
| |_________________________- trait requires 4 parameters
...
12 | / sp_api::mock_impl_runtime_apis! {
13 | | impl Api<Block> for MockApi {
14 | | fn test(self, data: u64) {}
15 | |
16 | | fn test2(&mut self, data: u64) {}
17 | | }
18 | | }
| |_^ expected 4 parameters, found 3
|
= note: this error originates in the macro `sp_api::mock_impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0050]: method `test2_with_context` has 3 parameters but the declaration in trait `Api::test2_with_context` has 4
--> tests/ui/mock_only_self_reference.rs:12:1
|
3 | / sp_api::decl_runtime_apis! {
4 | | pub trait Api {
5 | | fn test(data: u64);
6 | | fn test2(data: u64);
7 | | }
8 | | }
| |_^
= note: expected fn pointer `fn(&MockApi, &BlockId<sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<u64, sp_runtime::traits::BlakeTwo256>, Extrinsic>>, ExecutionContext, Option<u64>, Vec<_>) -> Result<_, _>`
found fn pointer `fn(&MockApi, &BlockId<sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<u64, sp_runtime::traits::BlakeTwo256>, Extrinsic>>, ExecutionContext, Option<()>, Vec<_>) -> Result<_, _>`
| |__________________________- trait requires 4 parameters
...
12 | / sp_api::mock_impl_runtime_apis! {
13 | | impl Api<Block> for MockApi {
14 | | fn test(self, data: u64) {}
15 | |
16 | | fn test2(&mut self, data: u64) {}
17 | | }
18 | | }
| |_^ expected 4 parameters, found 3
|
= note: this error originates in the macro `sp_api::mock_impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
@@ -0,0 +1,41 @@
use sp_runtime::traits::{Block as BlockT, GetNodeBlockType};
use substrate_test_runtime_client::runtime::Block;
struct Runtime {}
impl GetNodeBlockType for Runtime {
type NodeBlock = Block;
}
sp_api::decl_runtime_apis! {
#[api_version(2)]
pub trait Api {
fn test1();
fn test2();
#[api_version(3)]
fn test3();
#[api_version(4)]
fn test4();
}
}
sp_api::impl_runtime_apis! {
#[api_version(2)]
impl self::Api<Block> for Runtime {
fn test1() {}
fn test2() {}
}
impl sp_api::Core<Block> for Runtime {
fn version() -> sp_version::RuntimeVersion {
unimplemented!()
}
fn execute_block(_: Block) {
unimplemented!()
}
fn initialize_block(_: &<Block as BlockT>::Header) {
unimplemented!()
}
}
}
fn main() {}
@@ -15,49 +15,6 @@ note: type in trait
= note: expected fn pointer `fn(u64)`
found fn pointer `fn(&u64)`
error[E0053]: method `Api_test_runtime_api_impl` has an incompatible type for trait
--> tests/ui/type_reference_in_impl_runtime_apis_call.rs:17:1
|
17 | sp_api::impl_runtime_apis! {
| -^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| _expected `u64`, found `&u64`
| |
18 | | impl self::Api<Block> for Runtime {
19 | | fn test(data: &u64) {
20 | | unimplemented!()
... |
34 | | }
35 | | }
| |_- help: change the parameter type to match the trait: `std::option::Option<u64>`
|
note: type in trait
--> tests/ui/type_reference_in_impl_runtime_apis_call.rs:11:1
|
11 | / sp_api::decl_runtime_apis! {
12 | | pub trait Api {
13 | | fn test(data: u64);
14 | | }
15 | | }
| |_^
= note: expected fn pointer `fn(&RuntimeApiImpl<__SR_API_BLOCK__, RuntimeApiImplCall>, &BlockId<__SR_API_BLOCK__>, ExecutionContext, std::option::Option<u64>, Vec<_>) -> Result<_, _>`
found fn pointer `fn(&RuntimeApiImpl<__SR_API_BLOCK__, RuntimeApiImplCall>, &BlockId<__SR_API_BLOCK__>, ExecutionContext, std::option::Option<&u64>, Vec<_>) -> Result<_, _>`
= note: this error originates in the macro `sp_api::impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0308]: mismatched types
--> tests/ui/type_reference_in_impl_runtime_apis_call.rs:17:1
|
17 | / sp_api::impl_runtime_apis! {
18 | | impl self::Api<Block> for Runtime {
19 | | fn test(data: &u64) {
20 | | unimplemented!()
... |
34 | | }
35 | | }
| |_^ expected `u64`, found `&u64`
|
= note: this error originates in the macro `sp_api::impl_runtime_apis` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0308]: mismatched types
--> tests/ui/type_reference_in_impl_runtime_apis_call.rs:19:11
|