Followup test for checking propagated documentation (#514)

* codegen: Composite fields docs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Propagate docs for Event type

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Propagate docs for calls module

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Update polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Propagate documentation for `TypeDefGenKind::Enum` kind

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Update polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen/tests: Recursively obtain raw metadata documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen/tests: Generate runtime interface from node metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen/tests: Obtain documentation from generated runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen/tests: Match raw documentation with the generated one

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert not longer needed "codegen: Composite fields docs"

This reverts commit 5460bbafc6262ac4f53e11d4ef11e44e1ce8296f.

* codegen/tests: Improve test regex documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Add integration-tests feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Guard integration tests under feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* test-runtime: Place build.rs under feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Pass `integration-tests` feature to `test-runtime`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* CI: Use `integration-tests` feature to run all tests

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Rely on `#[cfg(feature = "integration-tests")]` for integration

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt/metadata: Manually construct test metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Move scale binary blob to dedicated folder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Update path to metadata blob

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Rely on artifact metadata blob for benches

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Remove `test-runtime` dependency

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Modify runtime path for `custom_type_derives`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Remove tests folder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* test-runtime: Remove `integration-tests` feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* integration-tests: Add an integration test crate for subxt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Remove `test-runtime` dependency

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Add comment for feature flags

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* integration-tests: Trim dependencies

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* integration-tests: Move dependencies under dev

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "CI: Use `integration-tests` feature to run all tests"

This reverts commit 8e5f38ba8c633ac40420fadf58700ac402f762d4.

* codegen: Move documentation test to integration crate

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen_documentation: Add license + fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update `polkadot.rs`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Move test under `testing` folder

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update testing/integration-tests/src/codegen/codegen_documentation.rs

Co-authored-by: Tarik Gul <47201679+TarikGul@users.noreply.github.com>

Co-authored-by: Tarik Gul <47201679+TarikGul@users.noreply.github.com>
This commit is contained in:
Alexandru Vasile
2022-06-17 20:25:13 +03:00
committed by GitHub
parent cb18ad75f3
commit 0d67b80cc5
10 changed files with 565 additions and 688 deletions
+4
View File
@@ -154,7 +154,11 @@ pub fn generate_calls(
})
.unzip();
let call_ty = type_gen.resolve_type(call.ty.id());
let docs = call_ty.docs();
quote! {
#( #[doc = #docs ] )*
pub mod calls {
use super::root_mod;
use super::#types_mod_ident;
+3
View File
@@ -81,8 +81,11 @@ pub fn generate_events(
}
});
let event_type = type_gen.resolve_type_path(event.ty.id(), &[]);
let event_ty = type_gen.resolve_type(event.ty.id());
let docs = event_ty.docs();
quote! {
#( #[doc = #docs ] )*
pub type Event = #event_type;
pub mod events {
use super::#types_mod_ident;
+8
View File
@@ -46,6 +46,8 @@ pub struct TypeDefGen {
derives: Derives,
/// The kind of type to be generated.
ty_kind: TypeDefGenKind,
/// Type documentation.
ty_docs: TokenStream,
}
impl TypeDefGen {
@@ -119,10 +121,14 @@ impl TypeDefGen {
_ => TypeDefGenKind::BuiltIn,
};
let docs = ty.docs();
let ty_docs = quote! { #( #[doc = #docs ] )* };
Self {
type_params,
derives,
ty_kind,
ty_docs,
}
}
}
@@ -152,8 +158,10 @@ impl quote::ToTokens for TypeDefGen {
let enum_ident = format_ident!("{}", type_name);
let type_params = &self.type_params;
let derives = &self.derives;
let docs = &self.ty_docs;
let ty_toks = quote! {
#derives
#docs
pub enum #enum_ident #type_params {
#( #variants, )*
}
-33
View File
@@ -1,33 +0,0 @@
[package]
name = "subxt-integration"
version = "0.21.0"
authors = ["Parity Technologies <admin@parity.io>"]
edition = "2021"
license = "GPL-3.0"
readme = "../README.md"
repository = "https://github.com/paritytech/subxt"
documentation = "https://docs.rs/subxt"
homepage = "https://www.parity.io/"
description = "Subxt integration tests that rely on the Substrate binary"
[features]
default = ["subxt/integration-tests"]
[dev-dependencies]
assert_matches = "1.5.0"
codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false, features = ["derive", "full", "bit-vec"] }
frame-metadata = "15.0.0"
futures = "0.3.13"
hex = "0.4.3"
scale-info = { version = "2.0.0", features = ["bit-vec"] }
sp-core = { version = "6.0.0", default-features = false }
sp-keyring = "6.0.0"
sp-runtime = "6.0.0"
subxt = { version = "0.21.0", path = "../subxt" }
test-runtime = { path = "../test-runtime" }
tokio = { version = "1.8", features = ["macros", "time"] }
tracing = "0.1.34"
tracing-subscriber = "0.3.11"
wabt = "0.10.0"
which = "4.0.2"
-38
View File
@@ -1,38 +0,0 @@
// Copyright 2019-2022 Parity Technologies (UK) Ltd.
// This file is part of subxt.
//
// subxt is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// subxt is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with subxt. If not, see <http://www.gnu.org/licenses/>.
#![deny(unused_crate_dependencies)]
#[cfg(test)]
mod codegen;
#[cfg(test)]
mod utils;
#[cfg(test)]
mod client;
#[cfg(test)]
mod events;
#[cfg(test)]
mod frame;
#[cfg(test)]
mod metadata;
#[cfg(test)]
mod storage;
#[cfg(test)]
use test_runtime::node_runtime;
#[cfg(test)]
use utils::*;
@@ -1,340 +0,0 @@
// Copyright 2019-2022 Parity Technologies (UK) Ltd.
// This file is part of subxt.
//
// subxt is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// subxt is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with subxt. If not, see <http://www.gnu.org/licenses/>.
use crate::{
test_context,
TestContext,
};
use frame_metadata::{
ExtrinsicMetadata,
PalletCallMetadata,
PalletMetadata,
PalletStorageMetadata,
RuntimeMetadataPrefixed,
RuntimeMetadataV14,
StorageEntryMetadata,
StorageEntryModifier,
StorageEntryType,
};
use scale_info::{
build::{
Fields,
Variants,
},
meta_type,
Path,
Type,
TypeInfo,
};
use subxt::{
ClientBuilder,
DefaultConfig,
Metadata,
SubstrateExtrinsicParams,
};
use crate::utils::node_runtime;
type RuntimeApi =
node_runtime::RuntimeApi<DefaultConfig, SubstrateExtrinsicParams<DefaultConfig>>;
async fn metadata_to_api(metadata: RuntimeMetadataV14, cxt: &TestContext) -> RuntimeApi {
let prefixed = RuntimeMetadataPrefixed::from(metadata);
let metadata = Metadata::try_from(prefixed).unwrap();
ClientBuilder::new()
.set_url(cxt.node_proc.ws_url().to_string())
.set_metadata(metadata)
.build()
.await
.unwrap()
.to_runtime_api::<node_runtime::RuntimeApi<
DefaultConfig,
SubstrateExtrinsicParams<DefaultConfig>,
>>()
}
#[tokio::test]
async fn full_metadata_check() {
let cxt = test_context().await;
let api = &cxt.api;
// Runtime metadata is identical to the metadata used during API generation.
assert!(api.validate_metadata().is_ok());
// Modify the metadata.
let mut metadata: RuntimeMetadataV14 = {
let locked_client_metadata = api.client.metadata();
let client_metadata = locked_client_metadata.read();
client_metadata.runtime_metadata().clone()
};
metadata.pallets[0].name = "NewPallet".to_string();
let new_api = metadata_to_api(metadata, &cxt).await;
assert_eq!(
new_api
.validate_metadata()
.err()
.expect("Validation should fail for incompatible metadata"),
::subxt::MetadataError::IncompatibleMetadata
);
}
#[tokio::test]
async fn constants_check() {
let cxt = test_context().await;
let api = &cxt.api;
// Ensure that `ExistentialDeposit` is compatible before altering the metadata.
assert!(cxt.api.constants().balances().existential_deposit().is_ok());
// Modify the metadata.
let mut metadata: RuntimeMetadataV14 = {
let locked_client_metadata = api.client.metadata();
let client_metadata = locked_client_metadata.read();
client_metadata.runtime_metadata().clone()
};
let mut existential = metadata
.pallets
.iter_mut()
.find(|pallet| pallet.name == "Balances")
.expect("Metadata must contain Balances pallet")
.constants
.iter_mut()
.find(|constant| constant.name == "ExistentialDeposit")
.expect("ExistentialDeposit constant must be present");
existential.value = vec![0u8; 32];
let new_api = metadata_to_api(metadata, &cxt).await;
assert!(new_api.validate_metadata().is_err());
assert!(new_api
.constants()
.balances()
.existential_deposit()
.is_err());
// Other constant validation should not be impacted.
assert!(new_api.constants().balances().max_locks().is_ok());
}
fn default_pallet() -> PalletMetadata {
PalletMetadata {
name: "Test",
storage: None,
calls: None,
event: None,
constants: vec![],
error: None,
index: 0,
}
}
fn pallets_to_metadata(pallets: Vec<PalletMetadata>) -> RuntimeMetadataV14 {
RuntimeMetadataV14::new(
pallets,
ExtrinsicMetadata {
ty: meta_type::<()>(),
version: 0,
signed_extensions: vec![],
},
meta_type::<()>(),
)
}
#[tokio::test]
async fn calls_check() {
let cxt = test_context().await;
// Ensure that `Unbond` and `WinthdrawUnbonded` calls are compatible before altering the metadata.
assert!(cxt.api.tx().staking().unbond(123_456_789_012_345).is_ok());
assert!(cxt.api.tx().staking().withdraw_unbonded(10).is_ok());
// Reconstruct the `Staking` call as is.
struct CallRec;
impl TypeInfo for CallRec {
type Identity = Self;
fn type_info() -> Type {
Type::builder()
.path(Path::new("Call", "pallet_staking::pallet::pallet"))
.variant(
Variants::new()
.variant("unbond", |v| {
v.index(0).fields(Fields::named().field(|f| {
f.compact::<u128>()
.name("value")
.type_name("BalanceOf<T>")
}))
})
.variant("withdraw_unbonded", |v| {
v.index(1).fields(Fields::named().field(|f| {
f.ty::<u32>().name("num_slashing_spans").type_name("u32")
}))
}),
)
}
}
let pallet = PalletMetadata {
name: "Staking",
calls: Some(PalletCallMetadata {
ty: meta_type::<CallRec>(),
}),
..default_pallet()
};
let metadata = pallets_to_metadata(vec![pallet]);
let new_api = metadata_to_api(metadata, &cxt).await;
assert!(new_api.tx().staking().unbond(123_456_789_012_345).is_ok());
assert!(new_api.tx().staking().withdraw_unbonded(10).is_ok());
// Change `Unbond` call but leave the rest as is.
struct CallRecSecond;
impl TypeInfo for CallRecSecond {
type Identity = Self;
fn type_info() -> Type {
Type::builder()
.path(Path::new("Call", "pallet_staking::pallet::pallet"))
.variant(
Variants::new()
.variant("unbond", |v| {
v.index(0).fields(Fields::named().field(|f| {
// Is of type u32 instead of u128.
f.compact::<u32>().name("value").type_name("BalanceOf<T>")
}))
})
.variant("withdraw_unbonded", |v| {
v.index(1).fields(Fields::named().field(|f| {
f.ty::<u32>().name("num_slashing_spans").type_name("u32")
}))
}),
)
}
}
let pallet = PalletMetadata {
name: "Staking",
calls: Some(PalletCallMetadata {
ty: meta_type::<CallRecSecond>(),
}),
..default_pallet()
};
let metadata = pallets_to_metadata(vec![pallet]);
let new_api = metadata_to_api(metadata, &cxt).await;
// Unbond call should fail, while withdraw_unbonded remains compatible.
assert!(new_api.tx().staking().unbond(123_456_789_012_345).is_err());
assert!(new_api.tx().staking().withdraw_unbonded(10).is_ok());
}
#[tokio::test]
async fn storage_check() {
let cxt = test_context().await;
// Ensure that `ExtrinsicCount` and `EventCount` storages are compatible before altering the metadata.
assert!(cxt
.api
.storage()
.system()
.extrinsic_count(None)
.await
.is_ok());
assert!(cxt
.api
.storage()
.system()
.all_extrinsics_len(None)
.await
.is_ok());
// Reconstruct the storage.
let storage = PalletStorageMetadata {
prefix: "System",
entries: vec![
StorageEntryMetadata {
name: "ExtrinsicCount",
modifier: StorageEntryModifier::Optional,
ty: StorageEntryType::Plain(meta_type::<u32>()),
default: vec![0],
docs: vec![],
},
StorageEntryMetadata {
name: "AllExtrinsicsLen",
modifier: StorageEntryModifier::Optional,
ty: StorageEntryType::Plain(meta_type::<u32>()),
default: vec![0],
docs: vec![],
},
],
};
let pallet = PalletMetadata {
name: "System",
storage: Some(storage),
..default_pallet()
};
let metadata = pallets_to_metadata(vec![pallet]);
let new_api = metadata_to_api(metadata, &cxt).await;
assert!(new_api
.storage()
.system()
.extrinsic_count(None)
.await
.is_ok());
assert!(new_api
.storage()
.system()
.all_extrinsics_len(None)
.await
.is_ok());
// Reconstruct the storage while modifying ExtrinsicCount.
let storage = PalletStorageMetadata {
prefix: "System",
entries: vec![
StorageEntryMetadata {
name: "ExtrinsicCount",
modifier: StorageEntryModifier::Optional,
// Previously was u32.
ty: StorageEntryType::Plain(meta_type::<u8>()),
default: vec![0],
docs: vec![],
},
StorageEntryMetadata {
name: "AllExtrinsicsLen",
modifier: StorageEntryModifier::Optional,
ty: StorageEntryType::Plain(meta_type::<u32>()),
default: vec![0],
docs: vec![],
},
],
};
let pallet = PalletMetadata {
name: "System",
storage: Some(storage),
..default_pallet()
};
let metadata = pallets_to_metadata(vec![pallet]);
let new_api = metadata_to_api(metadata, &cxt).await;
assert!(new_api
.storage()
.system()
.extrinsic_count(None)
.await
.is_err());
assert!(new_api
.storage()
.system()
.all_extrinsics_len(None)
.await
.is_ok());
}
+3
View File
@@ -20,11 +20,14 @@ codec = { package = "parity-scale-codec", version = "3.0.0", default-features =
frame-metadata = "15.0.0"
futures = "0.3.13"
hex = "0.4.3"
regex = "1.5.0"
scale-info = { version = "2.0.0", features = ["bit-vec"] }
sp-core = { version = "6.0.0", default-features = false }
sp-keyring = "6.0.0"
sp-runtime = "6.0.0"
syn = "1.0.0"
subxt = { version = "0.21.0", path = "../../subxt" }
subxt-codegen = { version = "0.21.0", path = "../../codegen" }
test-runtime = { path = "../test-runtime" }
tokio = { version = "1.8", features = ["macros", "time"] }
tracing = "0.1.34"
@@ -0,0 +1,119 @@
// Copyright 2019-2022 Parity Technologies (UK) Ltd.
// This file is part of subxt.
//
// subxt is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// subxt is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with subxt. If not, see <http://www.gnu.org/licenses/>.
use regex::Regex;
use subxt_codegen::{
DerivesRegistry,
RuntimeGenerator,
};
fn metadata_docs() -> Vec<String> {
// Load the runtime metadata downloaded from a node via `test-runtime`.
let bytes = test_runtime::METADATA;
let meta: frame_metadata::RuntimeMetadataPrefixed =
codec::Decode::decode(&mut &*bytes).expect("Cannot decode scale metadata");
let metadata = match meta.1 {
frame_metadata::RuntimeMetadata::V14(v14) => v14,
_ => panic!("Unsupported metadata version {:?}", meta.1),
};
// Inspect the metadata types and collect the documentation.
let mut docs = Vec::new();
for ty in metadata.types.types() {
docs.extend_from_slice(ty.ty().docs());
}
for pallet in metadata.pallets {
if let Some(storage) = pallet.storage {
for entry in storage.entries {
docs.extend(entry.docs);
}
}
// Note: Calls, Events and Errors are deduced directly to
// PortableTypes which are handled above.
for constant in pallet.constants {
docs.extend(constant.docs);
}
}
// Note: Extrinsics do not have associated documentation, but is implied by
// associated Type.
docs
}
fn generate_runtime_interface() -> String {
// Load the runtime metadata downloaded from a node via `test-runtime`.
let bytes = test_runtime::METADATA;
let metadata: frame_metadata::RuntimeMetadataPrefixed =
codec::Decode::decode(&mut &*bytes).expect("Cannot decode scale metadata");
// Generate a runtime interface from the provided metadata.
let generator = RuntimeGenerator::new(metadata);
let item_mod = syn::parse_quote!(
pub mod api {}
);
let derives = DerivesRegistry::default();
generator.generate_runtime(item_mod, derives).to_string()
}
fn interface_docs() -> Vec<String> {
// Generate the runtime interface from the node's metadata.
// Note: the API is generated on a single line.
let runtime_api = generate_runtime_interface();
// Documentation lines have the following format:
// # [ doc = "Upward message is invalid XCM."]
// Given the API is generated on a single line, the regex matching
// must be lazy hence the `?` in the matched group `(.*?)`.
//
// The greedy `non-?` matching would lead to one single match
// from the beginning of the first documentation tag, containing everything up to
// the last documentation tag
// `# [ doc = "msg"] # [ doc = "msg2"] ... api ... # [ doc = "msgN" ]`
//
// The `(.*?)` stands for match any character zero or more times lazily.
let re = Regex::new(r#"\# \[doc = "(.*?)"\]"#).unwrap();
re.captures_iter(&runtime_api)
.filter_map(|capture| {
// Get the matched group (ie index 1).
capture.get(1).as_ref().map(|doc| {
// Generated documentation will escape special characters.
// Replace escaped characters with unescaped variants for
// exact matching on the raw metadata documentation.
doc.as_str()
.replace("\\n", "\n")
.replace("\\t", "\t")
.replace("\\\"", "\"")
})
})
.collect()
}
#[test]
fn check_documentation() {
// Inspect metadata recursively and obtain all associated documentation.
let raw_docs = metadata_docs();
// Obtain documentation from the generated API.
let runtime_docs = interface_docs();
for raw in raw_docs.iter() {
assert!(
runtime_docs.contains(raw),
"Documentation not present in runtime API: {}",
raw
);
}
}
@@ -24,3 +24,5 @@
#[rustfmt::skip]
#[allow(clippy::all)]
mod polkadot;
mod codegen_documentation;
File diff suppressed because it is too large Load Diff