Metadata V15: Generate Runtime APIs (#918)

* Update frame-metadata to v15.1.0

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Enable V15 unstable metadata in frame-metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Move validation hashing to dedicated file

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use sp-metadata-ir from substrate to work with metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert using sp-metadata-ir in favor of conversion to v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Convert v14 to v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Use v15 for validation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Use v15 for codegen

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata/bench: Use v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust to v15 metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* force CI

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc: Fetch metadata at version

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Update polkadot.scale from commit 6dc9e84dde2

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Fetch V15 using the new API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Add runtime API interface

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Hash runtime API metadata for validation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Extract runtime API metadata wrapper from subxt::Metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Adjust hashing cache to reflect root+item keys

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc: Add raw state_call API method

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime_api: Add payload with static and dynamic variants

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Allow payloads to call into the runtime

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Add example to make a runtime API call both static and dynamic

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Simplify client fetching

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Address feedback and fallback to old API if needed

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime_api: Make mutability conditional on input params

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Regenerate polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Retain only pallets without runtime API info

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Retry via `Metadata_metadata` without conversion

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* payload: Remove `Decode` and change validation fn

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Retain runtime API types

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Runtime APIs documentation based on flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update examples/examples/custom_metadata_url.rs

Co-authored-by: James Wilson <james@jsdw.me>

* Update artifacts from polkadot-a6cfdb16e9

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update polkadot.rs with polkadot-a6cfdb16e9

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Generate input structures for runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime_api: Remove the static paylaod and use single impl

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Fetch account nonce

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Adjust build script to fetch latest metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Check account nonce from runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Fix doc generation for runtime types

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Rename `inputs` runtime calls module to `types`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Generate Calls structs inside the types module

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Check Alice account nonce before submitting the tx

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Add metadata version option flag supporting v14 and unstable

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Specify version to fetch

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Fallback to fetching latest stable metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Add unstable-metadata feature to fetch the latest

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* RuntimeVersion with Latest and Version(u32)

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Adjust fetch_metadata to inspect version list

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Adjust metadata to metadata_legacy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* events: Adjust docs to use metadata_legacy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* have a pass over fetch_metadata

* cargo fmt

* Option<String> when fetch metadata via latest API

* clippy

* fmt

* cli: Use the MetadataVersion from codegen

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Specify latest as default for MetadataVersion

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Remove version from metadata and use the one from file_or_url

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Decode metadata independently for different RPC calls

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>

codegen/src/api/mod.rs

@@ -8,6 +8,7 @@ mod calls;
 mod constants;
 mod errors;
 mod events;
+mod runtime_apis;
 mod storage;
 
 use frame_metadata::v15::RuntimeMetadataV15;
@@ -18,7 +19,7 @@ use crate::error::CodegenError;
 use crate::{
     ir,
     types::{CompositeDef, CompositeDefFields, TypeGenerator, TypeSubstitutes},
-    utils::{fetch_metadata_bytes_blocking, Uri},
+    utils::{fetch_metadata_bytes_blocking, MetadataVersion, Uri},
     CratePath,
 };
 use codec::Decode;
@@ -95,7 +96,11 @@ pub fn generate_runtime_api_from_url(
     should_gen_docs: bool,
     runtime_types_only: bool,
 ) -> Result<TokenStream2, CodegenError> {
-    let bytes = fetch_metadata_bytes_blocking(url)?;
+    // Fetch latest unstable version, if that fails fall back to the latest stable.
+    let bytes = match fetch_metadata_bytes_blocking(url, MetadataVersion::Unstable) {
+        Ok(bytes) => bytes,
+        Err(_) => fetch_metadata_bytes_blocking(url, MetadataVersion::Latest)?,
+    };
 
     generate_runtime_api_from_bytes(
         item_mod,
@@ -434,6 +439,14 @@ impl RuntimeGenerator {
 
         let rust_items = item_mod_ir.rust_items();
 
+        let apis_mod = runtime_apis::generate_runtime_apis(
+            &self.metadata,
+            &type_gen,
+            types_mod_ident,
+            &crate_path,
+            should_gen_docs,
+        )?;
+
         Ok(quote! {
             #( #item_mod_attrs )*
             #[allow(dead_code, unused_imports, non_camel_case_types)]
@@ -487,6 +500,12 @@ impl RuntimeGenerator {
                     TransactionApi
                 }
 
+                pub fn apis() -> runtime_apis::RuntimeApi {
+                    runtime_apis::RuntimeApi
+                }
+
+                #apis_mod
+
                 pub struct ConstantsApi;
                 impl ConstantsApi {
                     #(