speed up ui tests. (#944)

* speed up ui tests by reducing number of pallets * add small and tiny versions of polkadot metadata * change drop implementation fix test * rust yml add nextest and merge tests * 2 * 3 * 4 * 5 * 6 * add script for generating files * Upgrade to `syn 2.0` (#875) * WIP updating to syn 2.0.0 * WIP darling compat * Update darling and syn workspace deps * NestedMeta::parse_meta_list * Rename attribute keyword type property to path * Fmt * Update more type to path * Unused darling * Cargo.lock * Add missing syn features * Metadata V15: Generate Runtime APIs (#918) * Update frame-metadata to v15.1.0 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Enable V15 unstable metadata in frame-metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Move validation hashing to dedicated file Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Use sp-metadata-ir from substrate to work with metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Revert using sp-metadata-ir in favor of conversion to v15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Convert v14 to v15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Use v15 for validation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Use v15 for codegen Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata/bench: Use v15 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust to v15 metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Adjust testing Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Improve documentation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * force CI Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * rpc: Fetch metadata at version Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * artifacts: Update polkadot.scale from commit 6dc9e84dde2 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Fetch V15 using the new API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Add runtime API interface Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Hash runtime API metadata for validation Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Extract runtime API metadata wrapper from subxt::Metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Adjust hashing cache to reflect root+item keys Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * rpc: Add raw state_call API method Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime_api: Add payload with static and dynamic variants Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Allow payloads to call into the runtime Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * examples: Add example to make a runtime API call both static and dynamic Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update polkadot.rs Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Simplify client fetching Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Address feedback and fallback to old API if needed Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime_api: Make mutability conditional on input params Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Regenerate polkadot.rs Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Retain only pallets without runtime API info Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Retry via `Metadata_metadata` without conversion Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * payload: Remove `Decode` and change validation fn Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * metadata: Retain runtime API types Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Runtime APIs documentation based on flag Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update examples/examples/custom_metadata_url.rs Co-authored-by: James Wilson <james@jsdw.me> * Update artifacts from polkadot-a6cfdb16e9 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update polkadot.rs with polkadot-a6cfdb16e9 Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Generate input structures for runtime API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * runtime_api: Remove the static paylaod and use single impl Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * examples: Fetch account nonce Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Adjust build script to fetch latest metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Check account nonce from runtime API Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update cargo.lock Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Fix doc generation for runtime types Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Rename `inputs` runtime calls module to `types` Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Generate Calls structs inside the types module Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Check Alice account nonce before submitting the tx Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Add metadata version option flag supporting v14 and unstable Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Specify version to fetch Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Fallback to fetching latest stable metadata Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * subxt: Add unstable-metadata feature to fetch the latest Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * RuntimeVersion with Latest and Version(u32) Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Update polkadot.rs Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Adjust fetch_metadata to inspect version list Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * testing: Adjust metadata to metadata_legacy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * events: Adjust docs to use metadata_legacy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * have a pass over fetch_metadata * cargo fmt * Option<String> when fetch metadata via latest API * clippy * fmt * cli: Use the MetadataVersion from codegen Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Specify latest as default for MetadataVersion Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * cli: Remove version from metadata and use the one from file_or_url Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * Fix clippy Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> * codegen: Decode metadata independently for different RPC calls Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: James Wilson <james@jsdw.me> * add small and tiny versions of polkadot metadata * add script for generating files * new metadata * Runtime APIs; don't ask for validation hash anywhere except new_static (#947) * remove validation hash where not wanted * .into() * fix example * cargo fmt * Subxt Guide (#890) * WIP Starting to write book; extrinsics first pass done * cargo fmt * Ongoing work; events, constants, wip blocks * at_latest() and wip blocks * remove need to import parity-scale-codec crate with Subxt for macro to work * More docs; expanding on setup guide and finish pass of main sections * Tidy and remove example section for now * format book lines to 100chars * Fix example code * cargo fmt * cargo fmt * fix example * Fix typos * fix broken doc links, pub mods * Update Subxt macro docs * can't link to Subxt here * move macro docs to Subxt to make linking better and fix example code * note on macro about docs * cargo fmt * document the no_default_derives macro feature * Address feedback and remove redundant text * address review comments; minor tweaks * WIP add Runtime calls to book * Improve Runtime API docs * expose thing we forgot to expose and doc link fixes * use new metadata files in book * use mtadata full in docs * regenerate polkadot.rs * use small metadata in a couple of places * Update scripts/artifacts.sh Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com> --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Andrew Jones <ascjones@gmail.com> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: James Wilson <james@jsdw.me> Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
2026-04-25 09:27:57 +00:00 · 2023-05-12 11:27:24 +02:00
parent bb394156a2
commit 6087dc0002
38 changed files with 891 additions and 565 deletions
@@ -20,7 +20,7 @@ pub struct Second<T, U>(T, U);
 pub struct DoesntImplEncodeDecodeAsType(u16);

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress<A, B>",
        // Discarding both params:
@@ -30,7 +30,7 @@ pub struct DoesntImplEncodeDecodeAsType(u16);
 pub mod node_runtime {}

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress<A, B>",
        // Discarding second param:
@@ -40,7 +40,7 @@ pub mod node_runtime {}
 pub mod node_runtime2 {}

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress<A, B>",
        // Discarding first param:
@@ -50,7 +50,7 @@ pub mod node_runtime2 {}
 pub mod node_runtime3 {}

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress<A, B>",
        // Swapping params:
@@ -60,7 +60,7 @@ pub mod node_runtime3 {}
 pub mod node_runtime4 {}

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress",
        // Ignore input params and just use concrete types on output:
@@ -70,7 +70,7 @@ pub mod node_runtime4 {}
 pub mod node_runtime5 {}

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress<A, B>",
        // We can put a static type in, too:
@@ -80,7 +80,7 @@ pub mod node_runtime5 {}
 pub mod node_runtime6 {}

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress<A, B>",
        // Check that things can be wrapped in our Static type:
@@ -90,7 +90,7 @@ pub mod node_runtime6 {}
 pub mod node_runtime7 {}

 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_small.scale",
    substitute_type(
        path = "sp_runtime::multiaddress::MultiAddress<A, B>",
        // Recursive type param substitution should work too (swapping out nested A and B):
@@ -1,4 +1,4 @@
-#[subxt::subxt(runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale")]
+#[subxt::subxt(runtime_metadata_path = "../../../../artifacts/polkadot_metadata_tiny.scale")]
 pub mod node_runtime {
    pub struct SomeStruct;
    pub enum SomeEnum {
@@ -1,5 +1,5 @@
 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_tiny.scale",
    substitute_type(
        path = "sp_arithmetic::per_things::Perbill",
        with = "sp_runtime::Perbill"
@@ -1,5 +1,5 @@
 #[subxt::subxt(
-    runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+    runtime_metadata_path = "../../../../artifacts/polkadot_metadata_tiny.scale",
    runtime_metadata_url = "wss://rpc.polkadot.io:443"
 )]
 pub mod node_runtime {}
@@ -2,7 +2,7 @@ error: Only one of 'runtime_metadata_path' or 'runtime_metadata_url' can be prov
 --> src/incorrect/url_and_path_provided.rs:1:1
  |
 1 | / #[subxt::subxt(
-2 | |     runtime_metadata_path = "../../../../artifacts/polkadot_metadata.scale",
+2 | |     runtime_metadata_path = "../../../../artifacts/polkadot_metadata_tiny.scale",
 3 | |     runtime_metadata_url = "wss://rpc.polkadot.io:443"
 4 | | )]
  | |__^
@@ -23,7 +23,8 @@ use crate::utils::{MetadataTestRunner, PalletMetadataTestRunner};
 #[test]
 fn ui_tests() {
    let mut m = MetadataTestRunner::default();
-    let mut p = PalletMetadataTestRunner::new();
+    // specify pallets we want to test the metadata for (None => all pallets, but specifying only Some(..) speeds up test)
+    let mut p = PalletMetadataTestRunner::new(Some(&["Babe", "Claims", "Grandpa", "Balances"]));
    let t = trybuild::TestCases::new();

    t.pass("src/correct/*.rs");
@@ -8,15 +8,17 @@ use std::io::Read;
 use subxt_metadata::{metadata_v14_to_latest, retain_metadata_pallets};

 static TEST_DIR_PREFIX: &str = "subxt_generated_pallets_ui_tests_";
-static METADATA_FILE: &str = "../../artifacts/polkadot_metadata.scale";
+static METADATA_FILE: &str = "../../artifacts/polkadot_metadata_full.scale";

 pub struct PalletMetadataTestRunner {
    metadata: RuntimeMetadataV15,
    index: usize,
+    pallet_names: Option<Vec<String>>,
 }

 impl PalletMetadataTestRunner {
-    pub fn new() -> PalletMetadataTestRunner {
+    /// if pallet_names is Some(..) only the provided pallets will be tested.
+    pub fn new(pallet_names: Option<&[&str]>) -> PalletMetadataTestRunner {
        let mut file =
            std::fs::File::open(METADATA_FILE).expect("Cannot open metadata.scale artifact");

@@ -32,8 +34,12 @@ impl PalletMetadataTestRunner {
            frame_metadata::RuntimeMetadata::V15(v15) => v15,
            _ => panic!("Unsupported metadata version {:?}", meta.1),
        };
-
-        PalletMetadataTestRunner { metadata, index: 0 }
+        let pallet_names = pallet_names.map(|v| v.iter().map(|e| e.to_string()).collect());
+        PalletMetadataTestRunner {
+            metadata,
+            index: 0,
+            pallet_names,
+        }
    }

    pub fn path_to_next_ui_test(&mut self) -> Option<String> {
@@ -41,10 +47,15 @@ impl PalletMetadataTestRunner {
            return None
        };
        let test_name = &pallet.name;
-
        // Increment test index to avoid overlaps.
        let index = self.index;
        self.index += 1;
+        // if a pallet filter is set (pallet_names is Some), return the next pallet if this pallet is not in the filter.
+        if let Some(name_filter) = self.pallet_names.as_ref() {
+            if !name_filter.contains(test_name) {
+                return self.path_to_next_ui_test();
+            }
+        }

        // Build custom metadata containing only this pallet.
        let mut metadata = self.metadata.clone();
@@ -92,9 +103,17 @@ impl PalletMetadataTestRunner {
 impl Drop for PalletMetadataTestRunner {
    fn drop(&mut self) {
        for i in 0..self.index {
-            let mut tmp_dir = std::env::temp_dir();
-            tmp_dir.push(format!("{TEST_DIR_PREFIX}{i}"));
-            std::fs::remove_dir_all(tmp_dir).expect("cannot cleanup temp files");
+            if let Some(pallet) = self.metadata.pallets.get(self.index) {
+                if let Some(name_filter) = self.pallet_names.as_ref() {
+                    if !name_filter.contains(&pallet.name) {
+                        continue;
+                    }
+                }
+
+                let mut tmp_dir = std::env::temp_dir();
+                tmp_dir.push(format!("{TEST_DIR_PREFIX}{i}"));
+                std::fs::remove_dir_all(tmp_dir).expect("cannot cleanup temp files");
+            };
        }
    }
 }