Enrich metadata with type information (#8615)

* Cargo.lock after merge

* Restore scale-info feature

* Fully qualify TypeInfo derive

* Skip PendingSwap T

* Add missing skip_type_params attr

* metadata docs features

* Reduce pallet event attribute to struct

* Cargo.lock

* Update frame/balances/src/tests_composite.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Line widths check

* Cargo.lock

* Add scale-info/std

* Update frame/system/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Use `skip_type_params` to remove `TypeInfo` requirements on checks

* Revert "Remove unused Call metadata stuff"

This reverts commit 41311f85

* Skip BalanceSwapAction type parameter

* Remove unused event metadata macro

* Update frame-metadata

* Update primitives/npos-elections/compact/src/codec.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Manual TypeInfo for Header

* Remove TypeInfo requirement for consts in BoundedVec etc.

* Another TypeInfo bound removed

* review: fix indentation

* TypeInfo impls for Identity types

* Add some todos to add custom TypeInfo impls

* Update frame/support/procedural/src/pallet/expand/pallet_struct.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Add some todos to add custom TypeInfo impls

* Add a test for manual Data TypeInfo impl

* Add custom TypeInfo impl for Vote

* Era custom TypeInfo crimes

* Revert finality-grandpa version to 0.14.z

* review: renamed module to pallet_constants_metadata

* New line at end of file

* Add missing scale-info/std

* Update frame/support/src/storage/types/mod.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Remove StorageEntryType::Map unused flag

* Add missing scale-info dependency after merge

* SignedExtension::AdditionalSigned metadata

* Update frame-metadata, use abbreviated docs and args fields

* Update frame/example/Cargo.toml

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Add scale_info/std and remove unused scale-info dependency

* Remove scale-info dependency

* Remove treasury pallet::metadata

* Remove redundant Event test

* Add back scale-info as dev dependency

* fix error metadata when no error defined in decl_module

* Add Module3 to tests

* Fix metadata test

* Add docs feature to frame-support test

* WIP fixing pallet metadata test

* Remove redundant FunctionMetadata, FunctionArgumentMetadata as per https://github.com/paritytech/frame-metadata/pull/20

* Use main branch of frame-metadata

* Use patch of scale-info for latest changes

* Use latest patched scale-info

* Manual TypeInfo for DigestItem

* Manual TypeInfo for DigestItem

* Update scale-info

* Skip __Ignore variants for Error, depends on https://github.com/paritytech/scale-info/pull/117

* Named fields for FRAME v2 pallet Call variants

* Named fields for FRAME v1 pallet Call variants

* Add missing scale-info dependency

* WIP expand benchmark call variant

* fix benchmark with new function

create a new function for each variant of a pallet call.
This function is called by benchmarking macro in order not to break call
creation with unnamed argument

* fix tests

* more fix

* Fix staking tests

* Fix offchain workers calls

* Cherry pick rustfmt.toml from master

* cargo +nightly-2021-06-22 fmt --all

* Update to new call variant structs

* More call variant struct updates

* Remove unused import

* More call variant structs

* More call variant structs

* Even more call variant structs

* Mooar variant structs

* Evermore variant structs

* Call variant structs ad infinitum

* Fmt

* More call variants

* Last call variant

* Call variants all done?

* Fix SS58Prefix type

* Potential workaround for BitFlags<IdentityFields> TypeInfo

* Enable docs capturing for Call, Event, and Error types

* Fix IdentityFields TypeInfo

* Remove metadata-docs feature

* Add capture_docs = true for legacy Call, Event and Error types

* Fmt

* Fix metadata test type

* Update benchmarks with call struct variants

* Fmt

* More test fixes

* Fmt

* Fix benches

* Use latest capture_docs attr

* Latest scale_info

* Fmt

* review: change &Vec to &[]

* Remove pallet metadata attr

* review: remove commented out test code

* review: skip_type_params trailing comma suggestion

* Update to scale-info 0.10.0

* Update construct_runtime ui tests, different because of metadata TypeInfo impls

* Add some TypeInfo derives for UI tests

* Update storage ensure span ui stderrs

* Update call argument bound ui tests

Possibly changed because change from tuple to struct variants?

* Add scale-info dev dependency

* Update to latest finality-grandpa release

* review: missing newline

* review: missing scale-info/std

* review: remove duplicate scale-info/std

* review: remove fully qualified TypeInfo

* review: add missing scale-info/std

* review: remove unnecessary imports.

* Fmt

* Use crates.io RC version of frame-metadata

* Remove scale-info/std because it is a dev dependency

* Add missing scale_info dev-dependency for test

* Delete empty metadata folder

* Fix sp_std import

* review: improve manual UncheckedExtrinsic TypeInfo impl

* review: use full scale-info for dev-dependency

* Remove DefaultByteGetter impl

* review: derive TypeInfo for generic header

* Fmt

* Update primitives/runtime/src/generic/unchecked_extrinsic.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Update primitives/runtime/src/generic/unchecked_extrinsic.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Update bin/node/executor/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update frame/identity/src/types.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update frame/support/src/dispatch.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove redundant derive

* Simplify scale-info dependency

* Strip underscore prefix from call variant struct names

* Another underscore field

* More underscore fields

* Another underscore field

* Update to frame-metadata 14.0.0-rc.2 with combined StorageEntryType::Map

* Fmt

* Revert weights formatting

* Fix up some tests

* Fix up some tests for StorageEntryTypeMetadata

* scale-info dev dependency

* Fix test error

* Add missing TypeInfo derives

* Add back missing scale-info dependency

* Add back missing scale-info dependency

* Fix npos compact impls

* Cargo.lock

* Fmt

* Fix errors

* Fmt

* Fix renamed raw_solution field

* Fix error

* Fmt

* Fix some benchmarks

* Fmt

* Stray R

* Fix

* Add missing TypeInfos

* ui test fix

* Fix line widths

* Revert "ui test fix"

This reverts commit 2d15ec058a216e3f92d713f1174603a2bb1eac65.

* Upgrade to scale-info 0.11.0

* Revert "Upgrade to scale-info 0.11.0"

This reverts commit 047bb179085a0059c36cd20ab405f55cf0867e28.

* Add Runtime type

* Update to scale-info 0.12

* Update to scale-info 1.0

* Update frame-metadata to version 14.0.0

* Patch finality-grandpa until release available

* Fix metadata tests

* Fix metadata tests

* Fmt

* Remove patched finality-grandpa

* Fix tests, use scale_info imports

* Fix pallet tests

* Add BlockNumber TypeInfo bound

* ui test fix

* Cargo.lock

* Remove pallet metadata

* Cargo.lock

* Add missing scale-info dependency

* Remove pallet event metadata

* Fix error

* Fix collective errors

* Semicolol

* Fmt

* Remove another metadata attribute

* Add new variant to custom digest TypeInfo

* Fmt

* Cargo.lock from master

* Remove comma lol

* Fix example call error

* Fix example call error properly

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

substrate/frame/support/test/tests/pallet_compatibility.rs

@@ -99,7 +99,7 @@ mod pallet_old {
 #[frame_support::pallet]
 pub mod pallet {
 	use super::SomeAssociation;
-	use frame_support::pallet_prelude::*;
+	use frame_support::{pallet_prelude::*, scale_info};
 	use frame_system::{ensure_root, pallet_prelude::*};
 
 	#[pallet::config]
@@ -110,7 +110,8 @@ pub mod pallet {
 			+ Into<Weight>
 			+ Default
 			+ MaybeSerializeDeserialize
-			+ SomeAssociation;
+			+ SomeAssociation
+			+ scale_info::StaticTypeInfo;
 		#[pallet::constant]
 		type SomeConst: Get<Self::Balance>;
 		type Event: From<Event<Self>> + IsType<<Self as frame_system::Config>::Event>;
@@ -155,7 +156,6 @@ pub mod pallet {
 
 	#[pallet::event]
 	#[pallet::generate_deposit(fn deposit_event)]
-	#[pallet::metadata(T::Balance = "Balance")]
 	pub enum Event<T: Config> {
 		/// Dummy event, just here so there's a generic type that's used.
 		Dummy(T::Balance),
@@ -279,22 +279,71 @@ frame_support::construct_runtime!(
 mod test {
 	use super::{pallet, pallet_old, Runtime};
 	use codec::{Decode, Encode};
+	use scale_info::{form::PortableForm, Variant};
 
 	#[test]
 	fn metadata() {
 		let metadata = Runtime::metadata();
-		let modules = match metadata.1 {
-			frame_metadata::RuntimeMetadata::V13(frame_metadata::RuntimeMetadataV13 {
-				modules: frame_metadata::DecodeDifferent::Encode(m),
-				..
-			}) => m,
+		let (pallets, types) = match metadata.1 {
+			frame_support::metadata::RuntimeMetadata::V14(metadata) =>
+				(metadata.pallets, metadata.types),
 			_ => unreachable!(),
 		};
-		pretty_assertions::assert_eq!(modules[1].storage, modules[2].storage);
-		pretty_assertions::assert_eq!(modules[1].calls, modules[2].calls);
-		pretty_assertions::assert_eq!(modules[1].event, modules[2].event);
-		pretty_assertions::assert_eq!(modules[1].constants, modules[2].constants);
-		pretty_assertions::assert_eq!(modules[1].errors, modules[2].errors);
+
+		let assert_meta_types = |ty_id1, ty_id2| {
+			let ty1 = types.resolve(ty_id1).map(|ty| ty.type_def());
+			let ty2 = types.resolve(ty_id2).map(|ty| ty.type_def());
+			pretty_assertions::assert_eq!(ty1, ty2);
+		};
+
+		let get_enum_variants = |ty_id| match types.resolve(ty_id).map(|ty| ty.type_def()) {
+			Some(ty) => match ty {
+				scale_info::TypeDef::Variant(var) => var.variants(),
+				_ => panic!("Expected variant type"),
+			},
+			_ => panic!("No type found"),
+		};
+
+		let assert_enum_variants = |vs1: &[Variant<PortableForm>],
+		                            vs2: &[Variant<PortableForm>]| {
+			assert_eq!(vs1.len(), vs2.len());
+			for i in 0..vs1.len() {
+				let v1 = &vs2[i];
+				let v2 = &vs2[i];
+				assert_eq!(v1.fields().len(), v2.fields().len());
+				for f in 0..v1.fields().len() {
+					let f1 = &v1.fields()[f];
+					let f2 = &v2.fields()[f];
+					pretty_assertions::assert_eq!(f1.name(), f2.name());
+					pretty_assertions::assert_eq!(f1.ty(), f2.ty());
+				}
+			}
+		};
+
+		pretty_assertions::assert_eq!(pallets[1].storage, pallets[2].storage);
+
+		let calls1 = pallets[1].calls.as_ref().unwrap();
+		let calls2 = pallets[2].calls.as_ref().unwrap();
+		assert_meta_types(calls1.ty.id(), calls2.ty.id());
+
+		// event: check variants and fields but ignore the type name which will be different
+		let event1_variants = get_enum_variants(pallets[1].event.as_ref().unwrap().ty.id());
+		let event2_variants = get_enum_variants(pallets[2].event.as_ref().unwrap().ty.id());
+		assert_enum_variants(event1_variants, event2_variants);
+
+		let err1 = get_enum_variants(pallets[1].error.as_ref().unwrap().ty.id())
+			.iter()
+			.filter(|v| v.name() == "__Ignore")
+			.cloned()
+			.collect::<Vec<_>>();
+		let err2 = get_enum_variants(pallets[2].error.as_ref().unwrap().ty.id())
+			.iter()
+			.filter(|v| v.name() == "__Ignore")
+			.cloned()
+			.collect::<Vec<_>>();
+		assert_enum_variants(&err1, &err2);
+
+		pretty_assertions::assert_eq!(pallets[1].constants, pallets[2].constants);
 	}
 
 	#[test]
@@ -309,10 +358,10 @@ mod test {
 
 		assert_eq!(
 			pallet_old::Call::<Runtime>::decode(
-				&mut &pallet::Call::<Runtime>::set_dummy(10).encode()[..]
+				&mut &pallet::Call::<Runtime>::set_dummy { new_value: 10 }.encode()[..]
 			)
 			.unwrap(),
-			pallet_old::Call::<Runtime>::set_dummy(10),
+			pallet_old::Call::<Runtime>::set_dummy { new_value: 10 },
 		);
 	}
 }