pallet-vesting: Support multiple, merge-able vesting schedules (#9202)

* Support multiple, mergable vesting schedules

* Update node runtime

* Remove some TODO design questions and put them as commennts

* Update frame/vesting/src/benchmarking.rs

* Syntax and comment clean up

* Create filter enum for removing schedules

* Dry vesting calls with do_vest

* Improve old benchmarks to account for max schedules

* Update WeightInfo trait and make dummy fns

* Add merge_schedule weights

* Explicitly test multiple vesting scheudles

* Make new vesting tests more more clear

* Apply suggestions from code review

* Update remove_vesting_schedule to error with no index

* Try reduce spacing diff

* Apply suggestions from code review

* Use get on vesting for bounds check; check origin first

* No filter tuple; various simplifications

* unwrap or default when getting user schedules

* spaces be gone

* ReadMe fixes

* Update frame/vesting/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* address some comments for docs

* merge sched docs

* Apply suggestions from code review

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* log error when trying to push to vesting vec

* use let Some, not is_some

* remove_vesting_schedule u32, not optin

* new not try_new, create validate builder; VestingInfo

* Merge prep: break out tests and mock

* Add files forgot to include in merge

* revert some accidental changes to merged files

* Revert remaining accidental file changes

* More revert of accidental file change

* Try to reduce diff on tests

* namespace Vesting; check key when key should not exist;

* ending_block throws error on per_block of 0

* Try improve merge vesting info comment

* Update frame/vesting/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* add validate + correct; handle duration > blocknumber

* Move vesting_info module to its own file

* Seperate Vesting/locks updates from writing

* Add can_add_vesting schedule

* Adjust min vested transfer to be greater than all ED

* Initial integrity test impl

* merge_finished_and_yet_to_be_started_schedules

* Make sure to assert storage items are cleaned up

* Migration initial impl (not tested)

* Correct try-runtime hooks

* Apply suggestions from code review

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* header

* WIP: improve benchmarks

* Benchmarking working

* benchmarking: step over max schedules

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Simplify APIs by accepting vec; convert to bounded on write

* Test:  build_genesis_has_storage_version_v1

* Test more error cases

* Hack to get polkadot weights to work; should revert later

* Improve benchmarking; works on polkadot

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* WIP override storage

* Set storage not working example

* Remove unused tests

* VestingInfo: make public, derive MaxEndcodedLen

* Rename ending_block to ending_block_as_balance

* Superificial improvements

* Check for end block infinite, not just duration

* More superficial update

* Update tests

* Test vest with multi schedule

* Don't use half max balance in benchmarks

* Use debug_assert when locked is unexpected 0

* Implement exec_action

* Simplify per_block calc in vesting_info

* VestingInfo.validate in add_vesting_schedule & can_add_vesting_schedule

* Simplify post migrate check

* Remove merge event

* Minor benchmarking updates

* Remove VestingInfo.correct

* per_block accesor max with 1

* Improve comment

* Remoe debug

* Fix add schedule comment

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* no ref for should_remove param

* Remove unused vestingaction derive

* Asserts to show balance unlock in merge benchmark

* Remove unused imports

* trivial

* Fix benchmark asserts to handle non-multiple of 20 locked

* Add generate_storage_info

* migration :facepalm

* Remove per_block 0 logic

* Update frame/vesting/src/lib.rs

* Do not check for ending later than greatest block

* Apply suggestions from code review

* Benchmarks: simplify vesting schedule creation

* Add log back for migration

* Add note in ext docs explaining that all schedules will vest

* Make integrity test work

* Improve integrity test

* Remove unnescary type param from VestingInfo::new

* Remove unnescary resut for ending_block_as_balance

* Remove T param from ending_block_as_balance

* Reduce visibility of raw_per_block

* Remove unused type param for validate

* update old comment

* Make log a dep; log warn in migrate

* VestingInfo.validate returns Err(()), no T type param

* Try improve report_schedule_updates

* is_valid, not validate

* revert node runtime reorg;

* change schedule validity check to just warning

* Simplify merge_vesting_info return type

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add warning for migration

* Fix indentation

* Delete duplicate warnings

* Reduce diff in node runtime

* Fix benchmark build

* Upgrade cargo.toml to use 4.0.0-dev

* Cleanup

* MaxVestingSchedulesGetter initial impl

* MinVestedTransfer getter inintial impl

* Test MaxVestingSchedules & MinVestedTransfer getters; use getters in benchmarks

* Run cargo fmt

* Revert MinVestedTransfer & MaxVestingSchedules getters; Add integrity test

* Make MAX_VESTING_SCHEDULES a const

* fmt

* WIP: benchmark improvements

* Finish benchmark update

* Add test for transfer to account with less than ed

* Rm min_new_account_transfer; move sp-io to dev-dep

* Reduce cargo.toml diff

* Explain MAX_VESTING_SCHEDULES choice

* Fix after merge

* Try fix CI complaints

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fmt

* trigger

* fmt

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>

substrate/frame/vesting/src/migrations.rs

@@ -0,0 +1,95 @@
+// This file is part of Substrate.
+
+// Copyright (C) 2017-2021 Parity Technologies (UK) Ltd.
+// SPDX-License-Identifier: Apache-2.0
+
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// 	http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Storage migrations for the vesting pallet.
+
+use super::*;
+
+// Migration from single schedule to multiple schedules.
+pub(crate) mod v1 {
+	use super::*;
+
+	#[cfg(feature = "try-runtime")]
+	pub(crate) fn pre_migrate<T: Config>() -> Result<(), &'static str> {
+		assert!(StorageVersion::<T>::get() == Releases::V0, "Storage version too high.");
+
+		log::debug!(
+			target: "runtime::vesting",
+			"migration: Vesting storage version v1 PRE migration checks succesful!"
+		);
+
+		Ok(())
+	}
+
+	/// Migrate from single schedule to multi schedule storage.
+	/// WARNING: This migration will delete schedules if `MaxVestingSchedules < 1`.
+	pub(crate) fn migrate<T: Config>() -> Weight {
+		let mut reads_writes = 0;
+
+		Vesting::<T>::translate::<VestingInfo<BalanceOf<T>, T::BlockNumber>, _>(
+			|_key, vesting_info| {
+				reads_writes += 1;
+				let v: Option<
+					BoundedVec<
+						VestingInfo<BalanceOf<T>, T::BlockNumber>,
+						MaxVestingSchedulesGet<T>,
+					>,
+				> = vec![vesting_info].try_into().ok();
+
+				if v.is_none() {
+					log::warn!(
+						target: "runtime::vesting",
+						"migration: Failed to move a vesting schedule into a BoundedVec"
+					);
+				}
+
+				v
+			},
+		);
+
+		T::DbWeight::get().reads_writes(reads_writes, reads_writes)
+	}
+
+	#[cfg(feature = "try-runtime")]
+	pub(crate) fn post_migrate<T: Config>() -> Result<(), &'static str> {
+		assert_eq!(StorageVersion::<T>::get(), Releases::V1);
+
+		for (_key, schedules) in Vesting::<T>::iter() {
+			assert!(
+				schedules.len() == 1,
+				"A bounded vec with incorrect count of items was created."
+			);
+
+			for s in schedules {
+				// It is ok if this does not pass, but ideally pre-existing schedules would pass
+				// this validation logic so we can be more confident about edge cases.
+				if !s.is_valid() {
+					log::warn!(
+						target: "runtime::vesting",
+						"migration: A schedule does not pass new validation logic.",
+					)
+				}
+			}
+		}
+
+		log::debug!(
+			target: "runtime::vesting",
+			"migration: Vesting storage version v1 POST migration checks successful!"
+		);
+		Ok(())
+	}
+}