Commit Graph

19 Commits

Author SHA1 Message Date
Tadeo Hepperle 992ec9ab9b Fix codegen validation when Runtime APIs are stripped (#1000)
* modify code gen

* cargo fmt

* fix return

* move the runtime_apis static

* small layout change

* tidy up metadata UI testing and test validation function too

* fix validation tests to point to metadata

* update codegen

* fix clippy and tests

* tweak a comment

* rename validate_codegen => is_codegen_valid_for

---------

Co-authored-by: James Wilson <james@jsdw.me>
2023-06-09 10:10:48 +01:00
James Wilson b9f5419095 Introduce Metadata type (#974)
* WIP new Metadata type

* Finish basic Metadata impl inc hashing and validation

* remove caching from metadata; can add that higher up

* remove caches

* update retain to use Metadata

* clippy fixes

* update codegen to use Metadata

* clippy

* WIP fixing subxt lib

* WIP fixing tests, rebuild artifacts, fix OrderedMap::retain

* get --all-targets compiling

* move DispatchError type lookup back to being optional

* cargo clippy

* fix docs

* re-use VariantIndex to get variants

* add docs and enforce docs on metadata crate

* fix docs

* add test and fix docs

* cargo fmt

* address review comments

* update lockfiles

* ExactSizeIter so we can ask for len() of things (and hopefully soon is_empty()
2023-05-25 10:35:21 +01:00
Tadeo Hepperle e63d618b28 Retain specific runtime APIs (#961)
* retain runtime apis

* fix ui test

* use boxed closures
2023-05-22 11:02:36 +02:00
Tadeo Hepperle 6087dc0002 speed up ui tests. (#944)
* speed up ui tests by reducing number of pallets

* add small and tiny versions of polkadot metadata

* change drop implementation fix test

* rust yml add nextest and merge tests

* 2

* 3

* 4

* 5

* 6

* add script for generating files

* Upgrade to `syn 2.0` (#875)

* WIP updating to syn 2.0.0

* WIP darling compat

* Update darling and syn workspace deps

* NestedMeta::parse_meta_list

* Rename attribute keyword type property to path

* Fmt

* Update more type to path

* Unused darling

* Cargo.lock

* Add missing syn features

* Metadata V15: Generate Runtime APIs (#918)

* Update frame-metadata to v15.1.0

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Enable V15 unstable metadata in frame-metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Move validation hashing to dedicated file

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use sp-metadata-ir from substrate to work with metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert using sp-metadata-ir in favor of conversion to v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Convert v14 to v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Use v15 for validation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Use v15 for codegen

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata/bench: Use v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust to v15 metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* force CI

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc: Fetch metadata at version

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* artifacts: Update polkadot.scale from commit 6dc9e84dde2

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Fetch V15 using the new API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Add runtime API interface

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Hash runtime API metadata for validation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Extract runtime API metadata wrapper from subxt::Metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Adjust hashing cache to reflect root+item keys

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc: Add raw state_call API method

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime_api: Add payload with static and dynamic variants

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Allow payloads to call into the runtime

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Add example to make a runtime API call both static and dynamic

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Simplify client fetching

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Address feedback and fallback to old API if needed

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime_api: Make mutability conditional on input params

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Regenerate polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Retain only pallets without runtime API info

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Retry via `Metadata_metadata` without conversion

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* payload: Remove `Decode` and change validation fn

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Retain runtime API types

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Runtime APIs documentation based on flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update examples/examples/custom_metadata_url.rs

Co-authored-by: James Wilson <james@jsdw.me>

* Update artifacts from polkadot-a6cfdb16e9

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update polkadot.rs with polkadot-a6cfdb16e9

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Generate input structures for runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime_api: Remove the static paylaod and use single impl

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* examples: Fetch account nonce

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Adjust build script to fetch latest metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Check account nonce from runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Fix doc generation for runtime types

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Rename `inputs` runtime calls module to `types`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Generate Calls structs inside the types module

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Check Alice account nonce before submitting the tx

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Add metadata version option flag supporting v14 and unstable

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Specify version to fetch

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Fallback to fetching latest stable metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* subxt: Add unstable-metadata feature to fetch the latest

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* RuntimeVersion with Latest and Version(u32)

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update polkadot.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Adjust fetch_metadata to inspect version list

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Adjust metadata to metadata_legacy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* events: Adjust docs to use metadata_legacy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* have a pass over fetch_metadata

* cargo fmt

* Option<String> when fetch metadata via latest API

* clippy

* fmt

* cli: Use the MetadataVersion from codegen

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Specify latest as default for MetadataVersion

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Remove version from metadata and use the one from file_or_url

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Decode metadata independently for different RPC calls

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>

* add small and tiny versions of polkadot metadata

* add script for generating files

* new metadata

* Runtime APIs; don't ask for validation hash anywhere except new_static (#947)

* remove validation hash where not wanted

* .into()

* fix example

* cargo fmt

* Subxt Guide (#890)

* WIP Starting to write book; extrinsics first pass done

* cargo fmt

* Ongoing work; events, constants, wip blocks

* at_latest() and wip blocks

* remove need to import parity-scale-codec crate with Subxt for macro to work

* More docs; expanding on setup guide and finish pass of main sections

* Tidy and remove example section for now

* format book lines to 100chars

* Fix example code

* cargo fmt

* cargo fmt

* fix example

* Fix typos

* fix broken doc links, pub mods

* Update Subxt macro docs

* can't link to Subxt here

* move macro docs to Subxt to make linking better and fix example code

* note on macro about docs

* cargo fmt

* document the no_default_derives macro feature

* Address feedback and remove redundant text

* address review comments; minor tweaks

* WIP add Runtime calls to book

* Improve Runtime API docs

* expose thing we forgot to expose and doc link fixes

* use new metadata files in book

* use mtadata full in docs

* regenerate polkadot.rs

* use small metadata in a couple of places

* Update scripts/artifacts.sh

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
2023-05-12 11:27:24 +02:00
Andrew Jones f4eb80e78d Upgrade to syn 2.0 (#875)
* WIP updating to syn 2.0.0

* WIP darling compat

* Update darling and syn workspace deps

* NestedMeta::parse_meta_list

* Rename attribute keyword type property to path

* Fmt

* Update more type to path

* Unused darling

* Cargo.lock

* Add missing syn features
2023-05-03 13:14:27 +01:00
Alexandru Vasile 59d195d4ad metadata: Use v15 internally (#912)
* Update frame-metadata to v15.1.0

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Enable V15 unstable metadata in frame-metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Move validation hashing to dedicated file

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Use sp-metadata-ir from substrate to work with metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert using sp-metadata-ir in favor of conversion to v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Convert v14 to v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Use v15 for validation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Use v15 for codegen

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata/bench: Use v15

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust to v15 metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* force CI

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Address feedback

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Use HASH_LEN

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadta: Remove `LatestRuntimeMetadata` type alias

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Remove `metadata_to_latest` to avoid pancis

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-04-20 17:59:28 +03:00
Alexandru Vasile 8a7c17289a Metadata: Retain a subset of metadata pallets (#879)
* Update cargo.lock to use scale-info v2.4.0

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Retain only a subset of the metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* codegen: Generate top level Event

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Only retain DispatchError

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Export just the retain method

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Retain pallets

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* metadata: Do not include extrinsic metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* retain: Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* test-runtime: Generate per pallet metadata and rs file

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ui-tests: Check per metadata generated files

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Revert "test-runtime: Generate per pallet metadata and rs file"

This reverts commit 725a2e5f8339a795892dbbd2df19e49330ae3a9b.

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ui-tests: Adjust path to metadata file

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* ui-tests: Change drop order to keep `PalletMetadata` around

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update metadata/src/retain.rs

Co-authored-by: James Wilson <james@jsdw.me>

* Address feedback

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* retain: Keep extrinsic type

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Introduce `MetadataSource`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Use `MetadataSource` helper

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Use `FileOrUrl` flatten command argument

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* retain: Do not include generic type params in retained metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust subxt to scale-info v2.5.0

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update scaleinfo to v2.5.0

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove deprecated fn

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* testing: Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* benches: Use inner fields of scale info

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* address nits, and strip RuntimeCall type instead of trying to filter out use of it for better overall wins/clarity

* fix UI test

* move utils out of commands folder and fix clippy etc

* address nits

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>
2023-04-04 16:16:51 +01:00
James Wilson a2b8dde5e6 Add 'Static' type and improve type substitution codegen to accept it (#886)
* Add Static type which defers to Encode/Decode and impls EncodeAsType/DecodeAsType

* rename to static_type and impl Deref/Mut

* Improve type substitution in codegen so that concrete types can be swapped in

* A couple of comment tweaks and no need for a macro export

* Extend type substitution logic to work recursively on destination type

* cargo fmt

* Fix a couple of comments

* update ui test outpuot

* Add docs and missing_docs lint

* Add test for replacing multiple of Ident

* Update codegen/src/error.rs

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* update copyright year and fix ui test

* simplify another error

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
2023-03-31 16:56:19 +01:00
James Wilson 7c252fccf7 cargo fmt with stable defaults (#876) 2023-03-21 16:53:47 +00:00
James Wilson c63ff6ec6d Use scale-encode and scale-decode to encode and decode based on metadata (#842)
* WIP EncodeAsType and DecodeAsType

* remove silly cli experiment code

* Get things finally compiling with EncodeAsType and DecodeAsType

* update codegen test and WrapperKeepOpaque proper impl (in case it shows up in codegen)

* fix tests

* accomodate scale-value changes

* starting to migrate to EncodeAsType/DecodeAsType

* static event decoding and tx encoding to use DecodeAsFields/EncodeAsFields

* some tidy up and add decode(skip) attrs where needed

* fix root event decoding

* #[codec(skip)] will do, and combine map_key stuff into storage_address since it's all specific to that

* fmt and clippy

* update Cargo.lock

* remove patched scale-encode

* bump scale-encode to 0.1 and remove unused dep in testing crate

* update deps and use released scale-decode

* update scale-value to latest to remove git branch

* Apply suggestions from code review

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

* remove sorting in derives/attr generation; spit them out in order given

* re-add derive sorting; it's a hashmap

* StaticTxPayload and DynamicTxPayload rolled into single Payload struct

* StaticStorageAddress and DynamicStorageAddress into single Address struct

* Fix storage address byte retrieval

* StaticConstantAddress and DynamicConstantAddress => Address

* Simplify storage codegen to fix test

* Add comments

* Alias to RuntimeEvent rather than making another, and prep for substituting call type

* remove unnecessary clone

* Fix docs and failing UI test

* root_bytes -> to_root_bytes

* document error case in StorageClient::address_bytes()

---------

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
2023-03-21 15:31:13 +00:00
Igor Matuszewski 977f2a3333 Allow for remapping type parameters in type substitutions (#735)
* feat!: Allow for remapping type parameters in type substitutions

* chore: cargo fmt

* chore: cargo clippy

* chore: Remove some old code

* a little tidy

* address comment nit

Co-authored-by: James Wilson <james@jsdw.me>
2023-01-19 10:49:57 +00:00
Alexandru Vasile a3713476da Update trybuild to the latest version (#785)
* Update to trybuild version 1.0.76

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* tests: Adjust testing to the new trybuild location

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-01-17 09:41:09 +00:00
Francisco Miguel García a80d6cfd30 Add --derive-for-type to cli (#708)
* Add `--derive-for-type` to cli

* Remove clippy warnings
2022-11-23 15:25:32 +00:00
Igor Matuszewski 8b46f5c786 Retain Rust code items from mod decorated with subxt attribute (#721)
* refactor: Simplify collecting type substitutes

* Simplify ItemMod::from

* Preserve inner Rust items when expanding the subxt::subxt macro

* No named lifetimes

* Update codegen/src/api/mod.rs

Co-authored-by: Andrew Jones <ascjones@gmail.com>

* Move passing UI tests under `ui_tests`

Co-authored-by: Andrew Jones <ascjones@gmail.com>
2022-11-23 12:28:49 +01:00
Andrew Jones 52d4762d13 Upgrade to scale-info 2.3 and fix errors (#704) 2022-11-01 10:45:06 +01:00
James Wilson d03e599684 Add runtime_metadata_url to pull metadata directly from a node (#689)
* Allow metadata to be pulled directly from a node with runtime_metadata_url

* Update docs

* https too, and abstract out block_on fn

* tweak a comment in the example
2022-10-19 11:07:12 +01:00
James Wilson c55b7ba3e3 dual license (#590) 2022-06-28 14:09:41 +01:00
Alexandru Vasile a9129863bd Validate absolute path for substitute_type (#577)
* Remove commented code

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Ensure substitue path is global absolute

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Absolute path for build.rs

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Add ui test that fails to compile

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2022-06-22 10:20:16 +01:00
James Wilson 9cf63bafac Move test crates into a "testing" folder and add a ui (trybuild) test and ui-test helpers (#567)
* move test crates into a testing folder and add a ui test and helpers

* undo wee mixup with another PR

* cargo fmt

* clippy

* tidy ui-tests a little

* test different DispatchError types

* refactor dispatch error stuff

* name ui tests

* duff => useless

* align versions and cargo fmt
2022-06-17 14:33:58 +01:00