Add special tag to exclude runtime storage items from benchmarking (#12205)

* initial setup

* add WhitelistedStorageKeys trait

* add (A, B) tuple implementation for whitelisted_storage_keys()

* fix formatting

* implement WhitelistedStorageKeys for all tuple combinations

* impl_for_tuples up to 128 for WhitelistedStorageKeys

* refactor to #[benchmarking(cached)]

* tweak error message and mark BlockNumber as cached

* add benchmarking(cached) to the other default types

* add docs for benchmarking(cached)

* properly parse storage type declaration

* make storage_alias structs public so we can use them in this macro

* use BTreeMap since TrackedStorageKey missing Ord outside of std

* make WhitelistedStorageKeys accessible

* basic detection of benchmarking(cached) 💥

* proper parsing of #[benchmarking(cached)] from pallet parse macro

* store presence of #[benchmarking(cached)] macro on StorageDef

* will be used for later expansion

* compiling blank impl for WhitelistedStorageKeys

* move impl to expand_pallet_struct

* use frame_support::sp_std::vec::Vec properly

* successfully compiling with storage info loaded into a variable 💥

* plausible implementation for whitelisted_storage_keys()

* depends on the assumption that storage_info.encode() can be loaded
  into TrackedStorageKey::new(..)

* use Pallet::whitelisted_storage_keys() instead of hard-coded list

* AllPallets::whitelisted_storage_keys() properly working 💥

* collect storage names

* whitelisted_storage_keys() impl working 💥

* clean up

* fix compiler error

* just one compiler error

* fix doc compiler error

* use better import path

* fix comment

* whoops

* whoops again

* fix macro import issue

* cargo fmt

* mark example as ignore

* use keyword tokens instead of string parsing

* fix keyword-based parsing of benchmarking(cached)

* preliminary spec for check_whitelist()

* add additional test for benchmarking whitelist

* add TODO note

* remove irrelevant line from example

* use filter_map instead of filter and map

* simplify syntax

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* clean up

* fix test

* fix tests

* use keyword parsing instead of string parsing

* use collect() instead of a for loop

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* fix compiler error

* clean up benchmarking(cached) marking code

* use cloned()

* refactor to not use panic! and remove need for pub types in storage_alias

* remove unneeded use

Co-authored-by: Bastian Köcher <info@kchr.de>

* remove unneeded visibility changes

* don't manually hard code hash for treasury account as hex

* proper Ord, PartialOrd, and Hash impls for TrackedStorageKey

* now based just on key, and available in no-std

* use BTreeSet instead of BTreeMap

* fix comments

* cargo fmt

* switch to pallet::whitelist and re-do it basti's way :D

* make PartialOrd for TrackedStorageKey consistent with Ord

* more correct implementation of hash-related traits for TrackedStorageKey

* fix integration test

* update TODO

* remove unused keyword

* remove more unused keywords

* use into_iter()

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Update frame/support/procedural/src/pallet/parse/mod.rs

Co-authored-by: Bastian Köcher <info@kchr.de>

* add docs for whitelisted

* fix comment

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>

substrate/frame/support/src/lib.rs

@@ -1858,6 +1858,21 @@ pub mod pallet_prelude {
 /// pub(super) type MyStorage<T> = StorageValue<Value = u32>;
 /// ```
 ///
+/// The optional attribute `#[pallet::whitelist_storage]` will declare the
+/// storage as whitelisted from benchmarking. Doing so will exclude reads of
+/// that value's storage key from counting towards weight calculations during
+/// benchmarking.
+///
+/// This attribute should only be attached to storages that are known to be
+/// read/used in every block. This will result in a more accurate benchmarking weight.
+///
+/// ### Example
+/// ```ignore
+/// #[pallet::storage]
+/// #[pallet::whitelist_storage]
+/// pub(super) type Number<T: Config> = StorageValue<_, T::BlockNumber, ValueQuery>;
+/// ```
+///
 /// All the `cfg` attributes are automatically copied to the items generated for the storage,
 /// i.e. the getter, storage prefix, and the metadata element etc.
 ///

substrate/frame/support/src/traits.rs

@@ -89,6 +89,7 @@ pub mod schedule;
 mod storage;
 pub use storage::{
 	Instance, PartialStorageInfoTrait, StorageInfo, StorageInfoTrait, StorageInstance,
+	TrackedStorageKey, WhitelistedStorageKeys,
 };
 
 mod dispatch;

substrate/frame/support/src/traits/storage.rs

@@ -17,7 +17,9 @@
 
 //! Traits for encoding data related to pallet's storage items.
 
+use crate::sp_std::collections::btree_set::BTreeSet;
 use impl_trait_for_tuples::impl_for_tuples;
+pub use sp_core::storage::TrackedStorageKey;
 use sp_std::prelude::*;
 
 /// An instance of a pallet in the storage.
@@ -90,3 +92,29 @@ impl StorageInfoTrait for Tuple {
 pub trait PartialStorageInfoTrait {
 	fn partial_storage_info() -> Vec<StorageInfo>;
 }
+
+/// Allows a pallet to specify storage keys to whitelist during benchmarking.
+/// This means those keys will be excluded from the benchmarking performance
+/// calculation.
+pub trait WhitelistedStorageKeys {
+	/// Returns a [`Vec<TrackedStorageKey>`] indicating the storage keys that
+	/// should be whitelisted during benchmarking. This means that those keys
+	/// will be excluded from the benchmarking performance calculation.
+	fn whitelisted_storage_keys() -> Vec<TrackedStorageKey>;
+}
+
+#[cfg_attr(all(not(feature = "tuples-96"), not(feature = "tuples-128")), impl_for_tuples(64))]
+#[cfg_attr(all(feature = "tuples-96", not(feature = "tuples-128")), impl_for_tuples(96))]
+#[cfg_attr(feature = "tuples-128", impl_for_tuples(128))]
+impl WhitelistedStorageKeys for Tuple {
+	fn whitelisted_storage_keys() -> Vec<TrackedStorageKey> {
+		// de-duplicate the storage keys
+		let mut combined_keys: BTreeSet<TrackedStorageKey> = BTreeSet::new();
+		for_tuples!( #(
+			for storage_key in Tuple::whitelisted_storage_keys() {
+				combined_keys.insert(storage_key);
+			}
+		 )* );
+		combined_keys.into_iter().collect::<Vec<_>>()
+	}
+}