Offchain signing (#5182)

* New approach to offchain signing. * Use in im-online * Rewrite to use Account<T> * DRY signing. * Implement send_raw_unsigned_transaction * WiP * Expunge LocalCall * Expunge LocalCall * Fix compilation. * Solve call. * Make it compile again. * Finalize implementation. * Change CreateTransaction * Clear CreateTransaction. * Add price payload * Send raw transaction * Submit signed payload / unsigned transaction (WIP) * Supertrait requirements on T::Signature * Validate signature of payload on an unsigned transaction * Fix encoding - part 1 * Make it compile. * Fix compilation of unsigned validator. * Pass price payload to the transaction * Make block number part of the signed payload * Send signed transaction * Implement all_accounts, any_account * Fix formatting * Implement submit_transaction * Submit signed transaction (ForAll, ForAny) * Fix formatting * Implement CreateSignedTransaction * Move sign and verify to AppCrypto * Sign transaction * Call `use_encoded` * Remove SubmitAndSignTransaction * Implement runtime using new SigningTypes * Adapt offchain example to changes * Fix im-online pallet * Quick fix: rename AuthorityId2 * Fix offchain example tests * Add a comment on why keystore is required in unsigned transaction test * Use UintAuthorityId instead of u64 * WIP * Remove IdentifyAccount from UintAuthorityId * Implement PublicWrapper type * Fix im-online tests * Fix runtime test * Bump spec version * Fix executor tests * Rename ImOnlineAuthId -> ImOnlineAuthorityId and formatting * Fix merge * Documentation * Revert u64 -> UintAuthorityId conversion * Fix string errors * Document public members in offchain module * Introduce SubmitTransaction * Update pallets to use SubmitTransaction * WIP * Use SubmitTransaction in offchain * Use `submit_unsigned_transaction` * Fix tests * Update docs * Remove SigningTypes requirement from `SendTransactionTypes` * Fix tests * Update frame/system/src/offchain.rs Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/system/src/offchain.rs Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/example-offchain-worker/src/tests.rs Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/system/src/offchain.rs Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Update frame/system/src/offchain.rs Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com> * Remove leftover from previous iterations * Change enum to struct * Remove public * Move mock to node/executor/tests * Cleanup test-helpers * Make `application-crypto` `std` feature internal The macros should not generate code that requires that the calling crate has a feature with the name `std` defined. * Revert cargo lock update * Use TestAuthorityId from common * Restore members of account to public * Tidy up imports * Fix benchmarking pallet * Add tests demonstrating ForAll, ForAny on signer * Move definition of AppCrypto in example-offchain-worker from tests to mod::crypto * Cleanup stray comment * Fix ValidTransaction * Re-fix CreateSignedTransaction * Address PR feedback * Add can_sign method to signer * Propagate error * Improve documentation * Fix vec! macro not available * Document SendTransactiontypes * Add some docs. * Split signing examples * Add tests for signing examples * WIP can_sign - PR feedback * WIP * Split for_any / for_all into different calls * Verify payload and signature in test * Fix can_sign implementation * Fix impl_version * Import Box from sp_std * Create issues for TODOs * Ignore doctest. * Add test directly to system. Adjust UintTypes. * Add some tests to account filtering. * Remove code samples and point to example offchain worker * Fix doc links * Fix im-online tests using signatures. Co-authored-by: Tomasz Drwięga <tomasz@parity.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Bastian Köcher <git@kchr.de>
2026-04-27 10:27:59 +00:00 · 2020-04-21 14:55:05 +02:00
parent 798de8337b
commit 25751c0562
18 changed files with 1482 additions and 619 deletions
@@ -22,12 +22,12 @@
 //! Run `cargo doc --package pallet-example-offchain-worker --open` to view this module's
 //! documentation.
 //!
-//! - \[`pallet_example_offchain_worker::Trait`](./trait.Trait.html)
-//! - \[`Call`](./enum.Call.html)
-//! - \[`Module`](./struct.Module.html)
+//! - [`pallet_example_offchain_worker::Trait`](./trait.Trait.html)
+//! - [`Call`](./enum.Call.html)
+//! - [`Module`](./struct.Module.html)
 //!
 //!
-//! \## Overview
+//! ## Overview
 //!
 //! In this example we are going to build a very simplistic, naive and definitely NOT
 //! production-ready oracle for BTC/USD price.
@@ -40,15 +40,24 @@
 //! one unsigned transaction floating in the network.
 #![cfg_attr(not(feature = "std"), no_std)]

+use frame_system::{
+	self as system,
+	ensure_signed,
+	ensure_none,
+	offchain::{
+		AppCrypto, CreateSignedTransaction, SendUnsignedTransaction,
+		SignedPayload, SigningTypes, Signer, SubmitTransaction,
+	}
+};
 use frame_support::{
 	debug,
 	dispatch::DispatchResult, decl_module, decl_storage, decl_event,
 	traits::Get,
 	weights::{SimpleDispatchInfo, MINIMUM_WEIGHT},
 };
-use frame_system::{self as system, ensure_signed, ensure_none, offchain};
 use sp_core::crypto::KeyTypeId;
 use sp_runtime::{
+	RuntimeDebug,
 	offchain::{http, Duration, storage::StorageValueRef},
 	traits::Zero,
 	transaction_validity::{
@@ -56,6 +65,7 @@ use sp_runtime::{
 		TransactionPriority,
 	},
 };
+use codec::{Encode, Decode};
 use sp_std::vec::Vec;
 use lite_json::json::JsonValue;

@@ -76,18 +86,25 @@ pub const KEY_TYPE: KeyTypeId = KeyTypeId(*b"btc!");
 /// the types with this pallet-specific identifier.
 pub mod crypto {
 	use super::KEY_TYPE;
-	use sp_runtime::app_crypto::{app_crypto, sr25519};
+	use sp_runtime::{
+		app_crypto::{app_crypto, sr25519},
+		traits::Verify,
+	};
+	use sp_core::sr25519::Signature as Sr25519Signature;
 	app_crypto!(sr25519, KEY_TYPE);
+
+	pub struct TestAuthId;
+	impl frame_system::offchain::AppCrypto<<Sr25519Signature as Verify>::Signer, Sr25519Signature> for TestAuthId {
+		type RuntimeAppPublic = Public;
+		type GenericSignature = sp_core::sr25519::Signature;
+		type GenericPublic = sp_core::sr25519::Public;
+	}
 }

 /// This pallet's configuration trait
-pub trait Trait: frame_system::Trait {
-	/// The type to sign and submit transactions.
-	type SubmitSignedTransaction:
-		offchain::SubmitSignedTransaction<Self, <Self as Trait>::Call>;
-	/// The type to submit unsigned transactions.
-	type SubmitUnsignedTransaction:
-		offchain::SubmitUnsignedTransaction<Self, <Self as Trait>::Call>;
+pub trait Trait: CreateSignedTransaction<Call<Self>> {
+	/// The identifier type for an offchain worker.
+	type AuthorityId: AppCrypto<Self::Public, Self::Signature>;

 	/// The overarching event type.
 	type Event: From<Event<Self>> + Into<<Self as frame_system::Trait>::Event>;
@@ -115,6 +132,21 @@ pub trait Trait: frame_system::Trait {
 	type UnsignedPriority: Get<TransactionPriority>;
 }

+/// Payload used by this example crate to hold price
+/// data required to submit a transaction.
+#[derive(Encode, Decode, Clone, PartialEq, Eq, RuntimeDebug)]
+pub struct PricePayload<Public, BlockNumber> {
+	block_number: BlockNumber,
+	price: u32,
+	public: Public,
+}
+
+impl<T: SigningTypes> SignedPayload<T> for PricePayload<T::Public, T::BlockNumber> {
+	fn public(&self) -> T::Public {
+		self.public.clone()
+	}
+}
+
 decl_storage! {
 	trait Store for Module<T: Trait> as ExampleOffchainWorker {
 		/// A vector of recently submitted prices.
@@ -196,6 +228,22 @@ decl_module! {
 			Ok(())
 		}

+		#[weight = SimpleDispatchInfo::FixedNormal(10_000)]
+		pub fn submit_price_unsigned_with_signed_payload(
+			origin,
+			price_payload: PricePayload<T::Public, T::BlockNumber>,
+			_signature: T::Signature,
+		) -> DispatchResult {
+			// This ensures that the function can only be called via unsigned transaction.
+			ensure_none(origin)?;
+			// Add the price to the on-chain list, but mark it as coming from an empty address.
+			Self::add_price(Default::default(), price_payload.price);
+			// now increment the block number at which we expect next unsigned transaction.
+			let current_block = <system::Module<T>>::block_number();
+			<NextUnsignedAt<T>>::put(current_block + T::UnsignedInterval::get());
+			Ok(())
+		}
+
 		/// Offchain Worker entry point.
 		///
 		/// By implementing `fn offchain_worker` within `decl_module!` you declare a new offchain
@@ -236,7 +284,9 @@ decl_module! {
 			let should_send = Self::choose_transaction_type(block_number);
 			let res = match should_send {
 				TransactionType::Signed => Self::fetch_price_and_send_signed(),
-				TransactionType::Unsigned => Self::fetch_price_and_send_unsigned(block_number),
+				TransactionType::UnsignedForAny => Self::fetch_price_and_send_unsigned_for_any_account(block_number),
+				TransactionType::UnsignedForAll => Self::fetch_price_and_send_unsigned_for_all_accounts(block_number),
+				TransactionType::Raw => Self::fetch_price_and_send_raw_unsigned(block_number),
 				TransactionType::None => Ok(()),
 			};
 			if let Err(e) = res {
@@ -248,7 +298,9 @@ decl_module! {

 enum TransactionType {
 	Signed,
-	Unsigned,
+	UnsignedForAny,
+	UnsignedForAll,
+	Raw,
 	None,
 }

@@ -311,12 +363,11 @@ impl<T: Trait> Module<T> {
 				// transactions in a row. If a strict order is desired, it's better to use
 				// the storage entry for that. (for instance store both block number and a flag
 				// indicating the type of next transaction to send).
-				let send_signed = block_number % 2.into() == Zero::zero();
-				if send_signed {
-					TransactionType::Signed
-				} else {
-					TransactionType::Unsigned
-				}
+				let transaction_type = block_number % 3.into();
+				if transaction_type == Zero::zero() { TransactionType::Signed }
+				else if transaction_type == T::BlockNumber::from(1) { TransactionType::UnsignedForAny }
+				else if transaction_type == T::BlockNumber::from(2) { TransactionType::UnsignedForAll }
+				else { TransactionType::Raw }
 			},
 			// We are in the grace period, we should not send a transaction this time.
 			Err(RECENTLY_SENT) => TransactionType::None,
@@ -331,44 +382,43 @@ impl<T: Trait> Module<T> {

 	/// A helper function to fetch the price and send signed transaction.
 	fn fetch_price_and_send_signed() -> Result<(), &'static str> {
-		use system::offchain::SubmitSignedTransaction;
-		// Firstly we check if there are any accounts in the local keystore that are capable of
-		// signing the transaction.
-		// If not it doesn't even make sense to make external HTTP requests, since we won't be able
-		// to put the results back on-chain.
-		if !T::SubmitSignedTransaction::can_sign() {
+		use frame_system::offchain::SendSignedTransaction;
+
+		let signer = Signer::<T, T::AuthorityId>::all_accounts();
+		if !signer.can_sign() {
 			return Err(
 				"No local accounts available. Consider adding one via `author_insertKey` RPC."
 			)?
 		}
-
 		// Make an external HTTP request to fetch the current price.
 		// Note this call will block until response is received.
 		let price = Self::fetch_price().map_err(|_| "Failed to fetch price")?;

-		// Received price is wrapped into a call to `submit_price` public function of this pallet.
-		// This means that the transaction, when executed, will simply call that function passing
-		// `price` as an argument.
-		let call = Call::submit_price(price);
-
-		// Using `SubmitSignedTransaction` associated type we create and submit a transaction
+		// Using `send_signed_transaction` associated type we create and submit a transaction
 		// representing the call, we've just created.
 		// Submit signed will return a vector of results for all accounts that were found in the
 		// local keystore with expected `KEY_TYPE`.
-		let results = T::SubmitSignedTransaction::submit_signed(call);
+		let results = signer.send_signed_transaction(
+			|_account| {
+				// Received price is wrapped into a call to `submit_price` public function of this pallet.
+				// This means that the transaction, when executed, will simply call that function passing
+				// `price` as an argument.
+				Call::submit_price(price)
+			}
+		);
+
 		for (acc, res) in &results {
 			match res {
-				Ok(()) => debug::info!("[{:?}] Submitted price of {} cents", acc, price),
-				Err(e) => debug::error!("[{:?}] Failed to submit transaction: {:?}", acc, e),
+				Ok(()) => debug::info!("[{:?}] Submitted price of {} cents", acc.id, price),
+				Err(e) => debug::error!("[{:?}] Failed to submit transaction: {:?}", acc.id, e),
 			}
 		}

 		Ok(())
 	}

-	/// A helper function to fetch the price and send unsigned transaction.
-	fn fetch_price_and_send_unsigned(block_number: T::BlockNumber) -> Result<(), &'static str> {
-		use system::offchain::SubmitUnsignedTransaction;
+	/// A helper function to fetch the price and send a raw unsigned transaction.
+	fn fetch_price_and_send_raw_unsigned(block_number: T::BlockNumber) -> Result<(), &'static str> {
 		// Make sure we don't fetch the price if unsigned transaction is going to be rejected
 		// anyway.
 		let next_unsigned_at = <NextUnsignedAt<T>>::get();
@@ -385,14 +435,101 @@ impl<T: Trait> Module<T> {
 		// passing `price` as an argument.
 		let call = Call::submit_price_unsigned(block_number, price);

-		// Now let's create an unsigned transaction out of this call and submit it to the pool.
+		// Now let's create a transaction out of this call and submit it to the pool.
+		// Here we showcase two ways to send an unsigned transaction / unsigned payload (raw)
+		//
 		// By default unsigned transactions are disallowed, so we need to whitelist this case
 		// by writing `UnsignedValidator`. Note that it's EXTREMELY important to carefuly
 		// implement unsigned validation logic, as any mistakes can lead to opening DoS or spam
 		// attack vectors. See validation logic docs for more details.
-		T::SubmitUnsignedTransaction::submit_unsigned(call)
-			.map_err(|()| "Unable to submit unsigned transaction.".into())
+		//
+		SubmitTransaction::<T, Call<T>>::submit_unsigned_transaction(call.into())
+			.map_err(|()| "Unable to submit unsigned transaction.")?;

+		Ok(())
+	}
+
+	/// A helper function to fetch the price, sign payload and send an unsigned transaction
+	fn fetch_price_and_send_unsigned_for_any_account(block_number: T::BlockNumber) -> Result<(), &'static str> {
+		// Make sure we don't fetch the price if unsigned transaction is going to be rejected
+		// anyway.
+		let next_unsigned_at = <NextUnsignedAt<T>>::get();
+		if next_unsigned_at > block_number {
+			return Err("Too early to send unsigned transaction")
+		}
+
+		// Make an external HTTP request to fetch the current price.
+		// Note this call will block until response is received.
+		let price = Self::fetch_price().map_err(|_| "Failed to fetch price")?;
+
+		// Received price is wrapped into a call to `submit_price_unsigned` public function of this
+		// pallet. This means that the transaction, when executed, will simply call that function
+		// passing `price` as an argument.
+		let call = Call::submit_price_unsigned(block_number, price);
+
+		// Now let's create a transaction out of this call and submit it to the pool.
+		// Here we showcase two ways to send an unsigned transaction with a signed payload
+		SubmitTransaction::<T, Call<T>>::submit_unsigned_transaction(call.into())
+			.map_err(|()| "Unable to submit unsigned transaction.")?;
+
+		// -- Sign using any account
+		let (_, result) = Signer::<T, T::AuthorityId>::any_account().send_unsigned_transaction(
+			|account| PricePayload {
+				price,
+				block_number,
+				public: account.public.clone()
+			},
+			|payload, signature| {
+				Call::submit_price_unsigned_with_signed_payload(payload, signature)
+			}
+		).ok_or("No local accounts accounts available.")?;
+		result.map_err(|()| "Unable to submit transaction")?;
+
+		Ok(())
+	}
+
+	/// A helper function to fetch the price, sign payload and send an unsigned transaction
+	fn fetch_price_and_send_unsigned_for_all_accounts(block_number: T::BlockNumber) -> Result<(), &'static str> {
+		// Make sure we don't fetch the price if unsigned transaction is going to be rejected
+		// anyway.
+		let next_unsigned_at = <NextUnsignedAt<T>>::get();
+		if next_unsigned_at > block_number {
+			return Err("Too early to send unsigned transaction")
+		}
+
+		// Make an external HTTP request to fetch the current price.
+		// Note this call will block until response is received.
+		let price = Self::fetch_price().map_err(|_| "Failed to fetch price")?;
+
+		// Received price is wrapped into a call to `submit_price_unsigned` public function of this
+		// pallet. This means that the transaction, when executed, will simply call that function
+		// passing `price` as an argument.
+		let call = Call::submit_price_unsigned(block_number, price);
+
+		// Now let's create a transaction out of this call and submit it to the pool.
+		// Here we showcase two ways to send an unsigned transaction with a signed payload
+		SubmitTransaction::<T, Call<T>>::submit_unsigned_transaction(call.into())
+			.map_err(|()| "Unable to submit unsigned transaction.")?;
+
+		// -- Sign using all accounts
+		let transaction_results = Signer::<T, T::AuthorityId>::all_accounts()
+			.send_unsigned_transaction(
+				|account| PricePayload {
+					price,
+					block_number,
+					public: account.public.clone()
+				},
+				|payload, signature| {
+					Call::submit_price_unsigned_with_signed_payload(payload, signature)
+				}
+			);
+		for (_account_id, result) in transaction_results.into_iter() {
+			if result.is_err() {
+				return Err("Unable to submit transaction");
+			}
+		}
+
+		Ok(())
 	}

 	/// Fetch current price and return the result in cents.
@@ -507,6 +644,58 @@ impl<T: Trait> Module<T> {
 			Some(prices.iter().fold(0_u32, |a, b| a.saturating_add(*b)) / prices.len() as u32)
 		}
 	}
+
+	fn validate_transaction_parameters(
+		block_number: &T::BlockNumber,
+		new_price: &u32,
+	) -> TransactionValidity {
+		// Now let's check if the transaction has any chance to succeed.
+		let next_unsigned_at = <NextUnsignedAt<T>>::get();
+		if &next_unsigned_at > block_number {
+			return InvalidTransaction::Stale.into();
+		}
+		// Let's make sure to reject transactions from the future.
+		let current_block = <system::Module<T>>::block_number();
+		if &current_block < block_number {
+			return InvalidTransaction::Future.into();
+		}
+
+		// We prioritize transactions that are more far away from current average.
+		//
+		// Note this doesn't make much sense when building an actual oracle, but this example
+		// is here mostly to show off offchain workers capabilities, not about building an
+		// oracle.
+		let avg_price = Self::average_price()
+			.map(|price| if &price > new_price { price - new_price } else { new_price - price })
+			.unwrap_or(0);
+
+		ValidTransaction::with_tag_prefix("ExampleOffchainWorker")
+			// We set base priority to 2**20 and hope it's included before any other
+			// transactions in the pool. Next we tweak the priority depending on how much
+			// it differs from the current average. (the more it differs the more priority it
+			// has).
+			.priority(T::UnsignedPriority::get().saturating_add(avg_price as _))
+			// This transaction does not require anything else to go before into the pool.
+			// In theory we could require `previous_unsigned_at` transaction to go first,
+			// but it's not necessary in our case.
+			//.and_requires()
+			// We set the `provides` tag to be the same as `next_unsigned_at`. This makes
+			// sure only one transaction produced after `next_unsigned_at` will ever
+			// get to the transaction pool and will end up in the block.
+			// We can still have multiple transactions compete for the same "spot",
+			// and the one with higher priority will replace other one in the pool.
+			.and_provides(next_unsigned_at)
+			// The transaction is only valid for next 5 blocks. After that it's
+			// going to be revalidated by the pool.
+			.longevity(5)
+			// It's fine to propagate that transaction to other peers, which means it can be
+			// created even by nodes that don't produce blocks.
+			// Note that sometimes it's better to keep it for yourself (if you are the block
+			// producer), since for instance in some schemes others may copy your solution and
+			// claim a reward.
+			.propagate(true)
+			.build()
+	}
 }

 #[allow(deprecated)] // ValidateUnsigned
@@ -523,54 +712,16 @@ impl<T: Trait> frame_support::unsigned::ValidateUnsigned for Module<T> {
 		call: &Self::Call,
 	) -> TransactionValidity {
 		// Firstly let's check that we call the right function.
-		if let Call::submit_price_unsigned(block_number, new_price) = call {
-			// Now let's check if the transaction has any chance to succeed.
-			let next_unsigned_at = <NextUnsignedAt<T>>::get();
-			if &next_unsigned_at > block_number {
-				return InvalidTransaction::Stale.into();
+		if let Call::submit_price_unsigned_with_signed_payload(
+			ref payload, ref signature
+		) = call {
+			let signature_valid = SignedPayload::<T>::verify::<T::AuthorityId>(payload, signature.clone());
+			if !signature_valid {
+				return InvalidTransaction::BadProof.into();
 			}
-			// Let's make sure to reject transactions from the future.
-			let current_block = <system::Module<T>>::block_number();
-			if &current_block < block_number {
-				return InvalidTransaction::Future.into();
-			}
-
-			// We prioritize transactions that are more far away from current average.
-			//
-			// Note this doesn't make much sense when building an actual oracle, but this example
-			// is here mostly to show off offchain workers capabilities, not about building an
-			// oracle.
-			let avg_price = Self::average_price()
-				.map(|price| if &price > new_price { price - new_price } else { new_price - price })
-				.unwrap_or(0);
-
-			ValidTransaction::with_tag_prefix("ExampleOffchainWorker")
-				// We set base priority to 2**20 to make sure it's included before any other
-				// transactions in the pool. Next we tweak the priority depending on how much
-				// it differs from the current average. (the more it differs the more priority it
-				// has).
-				.priority(T::UnsignedPriority::get().saturating_add(avg_price as _))
-				// This transaction does not require anything else to go before into the pool.
-				// In theory we could require `previous_unsigned_at` transaction to go first,
-				// but it's not necessary in our case.
-				//.and_requires()
-
-				// We set the `provides` tag to be the same as `next_unsigned_at`. This makes
-				// sure only one transaction produced after `next_unsigned_at` will ever
-				// get to the transaction pool and will end up in the block.
-				// We can still have multiple transactions compete for the same "spot",
-				// and the one with higher priority will replace other one in the pool.
-				.and_provides(next_unsigned_at)
-				// The transaction is only valid for next 5 blocks. After that it's
-				// going to be revalidated by the pool.
-				.longevity(5)
-				// It's fine to propagate that transaction to other peers, which means it can be
-				// created even by nodes that don't produce blocks.
-				// Note that sometimes it's better to keep it for yourself (if you are the block
-				// producer), since for instance in some schemes others may copy your solution and
-				// claim a reward.
-				.propagate(true)
-				.build()
+			Self::validate_transaction_parameters(&payload.block_number, &payload.price)
+		} else if let Call::submit_price_unsigned(block_number, new_price) = call {
+			Self::validate_transaction_parameters(block_number, new_price)
 		} else {
 			InvalidTransaction::Call.into()
 		}