substrate-test-runtime migrated to "pure" frame runtime (#13737)

* substrate-test-runtime migrated to pure-frame based

* test block builder: helpers added

* simple renaming

* basic_authorship test adjusted

* block_building storage_proof test adjusted

* babe: tests: should_panic expected added

* babe: tests adjusted

ConsensusLog::NextEpochData is now added by pallet_babe as
pallet_babe::SameAuthoritiesForever trigger is used in runtime config.

* beefy: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digests stored during block execution are
checked against header digests:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(MmrRoot / AuthoritiesChange) is used.

* grandpa: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digest logs stored during block execution are
checked against header digest logs:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(ScheduledChange / ForcedChange and DigestItem::Other) is used.

* network:bitswap: test adjusted

The size of unchecked extrinsic was increased. The pattern used in test will
be placed at the end of scale-encoded buffer.

* runtime apis versions adjusted

* storage keys used in runtime adjusted

* wasm vs native tests removed

* rpc tests: adjusted

Transfer transaction processing was slightly improved, test was
adjusted.

* tests: sizes adjusted

Runtime extrinsic size was increased. Size of data read during block
execution was also increased due to usage of new pallets in runtime.

Sizes were adjusted in tests.

* cargo.lock update

cargo update -p substrate-test-runtime -p substrate-test-runtime-client

* warnings fixed

* builders cleanup: includes / std

* extrinsic validation cleanup

* txpool: benches performance fixed

* fmt

* spelling

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Apply code review suggestions

* Apply code review suggestions

* get rid of 1063 const

* renaming: UncheckedExtrinsic -> Extrinsic

* test-utils-runtime: further step to pure-frame

* basic-authorship: tests OK

* CheckSubstrateCall added + tests fixes

* test::Transfer call removed

* priority / propagate / no sudo+root-testing

* fixing warnings + format

* cleanup: build2/nonce + format

* final tests fixes

all tests are passing

* logs/comments removal

* should_not_accept_old_signatures test removed

* make txpool benches work again

* Cargo.lock reset

* format

* sudo hack removed

* txpool benches fix+cleanup

* .gitignore reverted

* rebase fixing + unsigned cleanup

* Cargo.toml/Cargo.lock cleanup

* force-debug feature removed

* mmr tests fixed

* make cargo-clippy happy

* network sync test uses unsigned extrinsic

* cleanup

* ".git/.scripts/commands/fmt/fmt.sh"

* push_storage_change signed call remove

* GenesisConfig cleanup

* fix

* fix

* GenesisConfig simplified

* storage_keys_works: reworked

* storage_keys_works: expected keys in vec

* storage keys list moved to substrate-test-runtime

* substrate-test: some sanity tests + GenesisConfigBuilder rework

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

* Review suggestions

* fix

* fix

* beefy: generate_blocks_and_sync block_num sync with actaul value

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* cargo update -p sc-rpc -p sc-transaction-pool

* Review suggestions

* fix

* doc added

* slot_duration adjusted for Babe::slot_duration

* small doc fixes

* array_bytes::hex used instead of hex

* tiny -> medium name fix

* Apply suggestions from code review

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* TransferData::try_from_unchecked_extrinsic -> try_from

* Update Cargo.lock

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

substrate/client/rpc/src/state/tests.rs

@@ -29,9 +29,11 @@ use sc_block_builder::BlockBuilderProvider;
 use sc_rpc_api::DenyUnsafe;
 use sp_consensus::BlockOrigin;
 use sp_core::{hash::H256, storage::ChildInfo};
-use sp_io::hashing::blake2_256;
 use std::sync::Arc;
-use substrate_test_runtime_client::{prelude::*, runtime};
+use substrate_test_runtime_client::{
+	prelude::*,
+	runtime::{ExtrinsicBuilder, Transfer},
+};
 
 const STORAGE_KEY: &[u8] = b"child";
 
@@ -218,7 +220,7 @@ async fn should_notify_about_storage_changes() {
 		// Cause a change:
 		let mut builder = client.new_block(Default::default()).unwrap();
 		builder
-			.push_transfer(runtime::Transfer {
+			.push_transfer(Transfer {
 				from: AccountKeyring::Alice.into(),
 				to: AccountKeyring::Ferdie.into(),
 				amount: 42,
@@ -244,18 +246,26 @@ async fn should_send_initial_storage_changes_and_notifications() {
 		let mut client = Arc::new(substrate_test_runtime_client::new());
 		let (api, _child) = new_full(client.clone(), test_executor(), DenyUnsafe::No);
 
-		let alice_balance_key =
-			blake2_256(&runtime::system::balance_of_key(AccountKeyring::Alice.into()));
+		let alice_balance_key = [
+			sp_core::hashing::twox_128(b"System"),
+			sp_core::hashing::twox_128(b"Account"),
+			sp_core::hashing::blake2_128(&AccountKeyring::Alice.public()),
+		]
+		.concat()
+		.iter()
+		.chain(AccountKeyring::Alice.public().0.iter())
+		.cloned()
+		.collect::<Vec<u8>>();
 
 		let api_rpc = api.into_rpc();
 		let sub = api_rpc
-			.subscribe("state_subscribeStorage", [[StorageKey(alice_balance_key.to_vec())]])
+			.subscribe("state_subscribeStorage", [[StorageKey(alice_balance_key)]])
 			.await
 			.unwrap();
 
 		let mut builder = client.new_block(Default::default()).unwrap();
 		builder
-			.push_transfer(runtime::Transfer {
+			.push_transfer(Transfer {
 				from: AccountKeyring::Alice.into(),
 				to: AccountKeyring::Ferdie.into(),
 				amount: 42,
@@ -280,22 +290,42 @@ async fn should_query_storage() {
 	async fn run_tests(mut client: Arc<TestClient>) {
 		let (api, _child) = new_full(client.clone(), test_executor(), DenyUnsafe::No);
 
-		let mut add_block = |nonce| {
+		let mut add_block = |index| {
 			let mut builder = client.new_block(Default::default()).unwrap();
 			// fake change: None -> None -> None
-			builder.push_storage_change(vec![1], None).unwrap();
+			builder
+				.push(ExtrinsicBuilder::new_storage_change(vec![1], None).build())
+				.unwrap();
 			// fake change: None -> Some(value) -> Some(value)
-			builder.push_storage_change(vec![2], Some(vec![2])).unwrap();
+			builder
+				.push(ExtrinsicBuilder::new_storage_change(vec![2], Some(vec![2])).build())
+				.unwrap();
 			// actual change: None -> Some(value) -> None
 			builder
-				.push_storage_change(vec![3], if nonce == 0 { Some(vec![3]) } else { None })
+				.push(
+					ExtrinsicBuilder::new_storage_change(
+						vec![3],
+						if index == 0 { Some(vec![3]) } else { None },
+					)
+					.build(),
+				)
 				.unwrap();
 			// actual change: None -> Some(value)
 			builder
-				.push_storage_change(vec![4], if nonce == 0 { None } else { Some(vec![4]) })
+				.push(
+					ExtrinsicBuilder::new_storage_change(
+						vec![4],
+						if index == 0 { None } else { Some(vec![4]) },
+					)
+					.build(),
+				)
 				.unwrap();
 			// actual change: Some(value1) -> Some(value2)
-			builder.push_storage_change(vec![5], Some(vec![nonce as u8])).unwrap();
+			builder
+				.push(
+					ExtrinsicBuilder::new_storage_change(vec![5], Some(vec![index as u8])).build(),
+				)
+				.unwrap();
 			let block = builder.build().unwrap().block;
 			let hash = block.header.hash();
 			executor::block_on(client.import(BlockOrigin::Own, block)).unwrap();
@@ -482,12 +512,13 @@ async fn should_return_runtime_version() {
 	let client = Arc::new(substrate_test_runtime_client::new());
 	let (api, _child) = new_full(client.clone(), test_executor(), DenyUnsafe::No);
 
+	// it is basically json-encoded substrate_test_runtime_client::runtime::VERSION
 	let result = "{\"specName\":\"test\",\"implName\":\"parity-test\",\"authoringVersion\":1,\
 		\"specVersion\":2,\"implVersion\":2,\"apis\":[[\"0xdf6acb689907609b\",4],\
 		[\"0x37e397fc7c91f5e4\",2],[\"0xd2bc9897eed08f15\",3],[\"0x40fe3ad401f8959a\",6],\
-		[\"0xc6e9a76309f39b09\",1],[\"0xdd718d5cc53262d4\",1],[\"0xcbca25e39f142387\",2],\
-		[\"0xf78b278be53f454c\",2],[\"0xab3c0572291feb8b\",1],[\"0xbc9d89904f5b923f\",1]],\
-		\"transactionVersion\":1,\"stateVersion\":1}";
+		[\"0xbc9d89904f5b923f\",1],[\"0xc6e9a76309f39b09\",2],[\"0xdd718d5cc53262d4\",1],\
+		[\"0xcbca25e39f142387\",2],[\"0xf78b278be53f454c\",2],[\"0xab3c0572291feb8b\",1],\
+		[\"0xed99c5acb25eedf5\",3]],\"transactionVersion\":1,\"stateVersion\":1}";
 
 	let runtime_version = api.runtime_version(None.into()).unwrap();
 	let serialized = serde_json::to_string(&runtime_version).unwrap();