Elastic scaling: runtime dependency tracking and enactment (#3479)

Changes needed to implement the runtime part of elastic scaling:
https://github.com/paritytech/polkadot-sdk/issues/3131,
https://github.com/paritytech/polkadot-sdk/issues/3132,
https://github.com/paritytech/polkadot-sdk/issues/3202

Also fixes https://github.com/paritytech/polkadot-sdk/issues/3675

TODOs:

- [x] storage migration
- [x] optimise process_candidates from O(N^2)
- [x] drop backable candidates which form cycles
- [x] fix unit tests
- [x] add more unit tests
- [x] check the runtime APIs which use the pending availability storage.
We need to expose all of them, see
https://github.com/paritytech/polkadot-sdk/issues/3576
- [x] optimise the candidate selection. we're currently picking randomly
until we satisfy the weight limit. we need to be smart about not
breaking candidate chains while being fair to all paras -
https://github.com/paritytech/polkadot-sdk/pull/3573

Relies on the changes made in
https://github.com/paritytech/polkadot-sdk/pull/3233 in terms of the
inclusion policy and the candidate ordering

---------

Signed-off-by: alindima <alin@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>

polkadot/runtime/parachains/src/util.rs

@@ -18,7 +18,7 @@
 //! on all modules.
 
 use frame_system::pallet_prelude::BlockNumberFor;
-use primitives::{Id as ParaId, PersistedValidationData, ValidatorIndex};
+use primitives::{HeadData, Id as ParaId, PersistedValidationData, ValidatorIndex};
 use sp_std::{collections::btree_set::BTreeSet, vec::Vec};
 
 use crate::{configuration, hrmp, paras};
@@ -42,6 +42,23 @@ pub fn make_persisted_validation_data<T: paras::Config + hrmp::Config>(
 	})
 }
 
+/// Make the persisted validation data for a particular parachain, a specified relay-parent, its
+/// storage root and parent head data.
+pub fn make_persisted_validation_data_with_parent<T: configuration::Config>(
+	relay_parent_number: BlockNumberFor<T>,
+	relay_parent_storage_root: T::Hash,
+	parent_head: HeadData,
+) -> PersistedValidationData<T::Hash, BlockNumberFor<T>> {
+	let config = <configuration::Pallet<T>>::config();
+
+	PersistedValidationData {
+		parent_head,
+		relay_parent_number,
+		relay_parent_storage_root,
+		max_pov_size: config.max_pov_size,
+	}
+}
+
 /// Take an active subset of a set containing all validators.
 ///
 /// First item in pair will be all items in set have indices found in the `active` indices set (in