Asynchronous backing PR (#2300)

* Update substrate & polkadot

* min changes to make async backing compile

* (async backing) parachain-system: track limitations for unincluded blocks (#2438)

* unincluded segment draft

* read para head from storage proof

* read_para_head -> read_included_para_head

* Provide pub interface

* add errors

* fix unincluded segment update

* BlockTracker -> Ancestor

* add a dmp limit

* Read para head depending on the storage switch

* doc comments

* storage items docs

* add a sanity check on block initialize

* Check watermark

* append to the segment on block finalize

* Move segment update into set_validation_data

* Resolve para head todo

* option watermark

* fix comment

* Drop dmq check

* fix weight

* doc-comments on inherent invariant

* Remove TODO

* add todo

* primitives tests

* pallet tests

* doc comments

* refactor unincluded segment length into a ConsensusHook (#2501)

* refactor unincluded segment length into a ConsensusHook

* add docs

* refactor bandwidth_out calculation

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* test for limits from impl

* fmt

* make tests compile

* update comment

* uncomment test

* fix collator test by adding parent to state proof

* patch HRMP watermark rules for unincluded segment

* get consensus-common tests to pass, using unincluded segment

* fix unincluded segment tests

* get all tests passing

* fmt

* rustdoc CI

* aura-ext: limit the number of authored blocks per slot (#2551)

* aura_ext consensus hook

* reverse dependency

* include weight into hook

* fix tests

* remove stray println

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* fix test warning

* fix doc link

---------

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>

* parachain-system: ignore go ahead signal once upgrade is processed (#2594)

* handle goahead signal for unincluded segment

* doc comment

* add test

* parachain-system: drop processed messages from inherent data (#2590)

* implement `drop_processed_messages`

* drop messages based on relay parent number

* adjust tests

* drop changes to mqc

* fix comment

* drop test

* drop more dead code

* clippy

* aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658)

* aura-ext: check slot in consensus hook

* convert relay chain slot

* Make relay chain slot duration generic

* use fixed velocity hook for pallets with aura

* purge timestamp inherent

* fix warning

* adjust runtime tests

* fix slots in tests

* Make `xcm-emulator` test pass for new consensus hook (#2722)

* add pallets on_initialize

* tests pass

* add AuraExt on_init

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: command-bot <>

---------

Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>

* update polkadot git refs

* CollationGenerationConfig closure is now optional (#2772)

* CollationGenerationConfig closure is now optional

* fix test

* propagate network-protocol-staging feature (#2899)

* Feature Flagging Consensus Hook Type Parameter (#2911)

* First pass

* fmt

* Added as default feature in tomls

* Changed to direct dependency feature

* Dealing with clippy error

* Update pallets/parachain-system/src/lib.rs

Co-authored-by: asynchronous rob <rphmeier@gmail.com>

---------

Co-authored-by: asynchronous rob <rphmeier@gmail.com>

* fmt

* bump deps and remove warning

* parachain-system: update RelevantMessagingState according to the unincluded segment (#2948)

* mostly address 2471 with a bug introduced

* adjust relevant messaging state after computing total

* fmt

* max -> min

* fix test implementation of xcmp source

* add test

* fix test message sending logic

* fix + test

* add more to unincluded segment test

* fmt

---------

Co-authored-by: Chris Sosnin <chris125_@live.com>

* Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864)

* add a comment

* refactor client/service utilities

* deprecate start_collator

* update parachain-template

* update test-service in the same way

* update polkadot-parachain crate

* fmt

* wire up new SubmitCollation message

* some runtime utilities for implementing unincluded segment runtime APIs

* allow parachains to configure their level of sybil-resistance when starting the network

* make aura-ext compile

* update to specify sybil resistance levels

* fmt

* specify relay chain slot duration in milliseconds

* update Aura to explicitly produce Send futures

also, make relay_chain_slot_duration a Duration

* add authoring duration to basic collator and document params

* integrate new basic collator into parachain-template

* remove assert_send used for testing

* basic-aura: only author when parent included

* update polkadot-parachain-bin

* fmt

* some fixes

* fixes

* add a RelayNumberMonotonicallyIncreases

* add a utility function for initializing subsystems

* some logging for timestamp adjustment

* fmt

* some fixes for lookahead collator

* add a log

* update `find_potential_parents` to account for sessions

* bound the loop

* restore & deprecate old start_collator and start_full_node functions.

* remove unnecessary await calls

* fix warning

* clippy

* more clippy

* remove unneeded logic

* ci

* update comment

Co-authored-by: Marcin S. <marcin@bytedude.com>

* (async backing) restore `CheckInherents` for backwards-compatibility (#2977)

* bring back timestamp

* Restore CheckInherents

* revert to empty CheckInherents

* make CheckInherents optional

* attempt

* properly end system blocks

* add some more comments

* ignore failing system parachain tests

* update refs after main feature branch merge

* comment out the offending tests because CI runs ignored tests

* fix warnings

* fmt

* revert to polkadot master

* cargo update -p polkadot-primitives -p sp-io

---------

Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: Andronik <write@reusable.software>
This commit is contained in:
Chris Sosnin
2023-08-18 19:57:34 +03:00
committed by GitHub
parent 4f699c70a4
commit 6ef1117685
105 changed files with 4707 additions and 2359 deletions
@@ -21,7 +21,7 @@ sp-std = { git = "https://github.com/paritytech/substrate", default-features = f
sp-core = { git = "https://github.com/paritytech/substrate", default-features = false, branch = "master" }
# Cumulus
cumulus-pallet-parachain-system = { path = "../../../../pallets/parachain-system", default-features = false }
cumulus-pallet-parachain-system = { path = "../../../../pallets/parachain-system", default-features = false, features = ["parameterized-consensus-hook",] }
cumulus-pallet-xcmp-queue = { path = "../../../../pallets/xcmp-queue", default-features = false }
cumulus-pallet-dmp-queue = { path = "../../../../pallets/dmp-queue", default-features = false }
pallet-collator-selection = { path = "../../../../pallets/collator-selection", default-features = false }
@@ -18,13 +18,14 @@
use codec::Encode;
use frame_support::{
assert_noop, assert_ok,
traits::{fungibles::InspectEnumerable, Get, OriginTrait},
traits::{fungibles::InspectEnumerable, Get, OnFinalize, OnInitialize, OriginTrait},
weights::Weight,
};
use parachains_common::Balance;
use frame_system::pallet_prelude::BlockNumberFor;
use parachains_common::{AccountId, Balance};
use parachains_runtimes_test_utils::{
assert_metadata, assert_total, AccountIdOf, BalanceOf, CollatorSessionKeys, ExtBuilder,
RuntimeHelper, ValidatorIdOf, XcmReceivedFrom,
ValidatorIdOf, XcmReceivedFrom,
};
use sp_runtime::{
traits::{MaybeEquivalence, StaticLookup, Zero},
@@ -33,6 +34,9 @@ use sp_runtime::{
use xcm::latest::prelude::*;
use xcm_executor::{traits::ConvertLocation, XcmExecutor};
type RuntimeHelper<Runtime, AllPalletsWithoutSystem = ()> =
parachains_runtimes_test_utils::RuntimeHelper<Runtime, AllPalletsWithoutSystem>;
// Re-export test_case from `parachains-runtimes-test-utils`
pub use parachains_runtimes_test_utils::test_cases::change_storage_constant_by_governance_works;
@@ -40,6 +44,7 @@ pub use parachains_runtimes_test_utils::test_cases::change_storage_constant_by_g
/// and can teleport it back and to the other parachains
pub fn teleports_for_native_asset_works<
Runtime,
AllPalletsWithoutSystem,
XcmConfig,
CheckingAccount,
WeightToFee,
@@ -62,6 +67,8 @@ pub fn teleports_for_native_asset_works<
+ pallet_collator_selection::Config
+ cumulus_pallet_parachain_system::Config
+ cumulus_pallet_xcmp_queue::Config,
AllPalletsWithoutSystem:
OnInitialize<BlockNumberFor<Runtime>> + OnFinalize<BlockNumberFor<Runtime>>,
AccountIdOf<Runtime>: Into<[u8; 32]>,
ValidatorIdOf<Runtime>: From<AccountIdOf<Runtime>>,
BalanceOf<Runtime>: From<Balance> + Into<u128>,
@@ -84,6 +91,13 @@ pub fn teleports_for_native_asset_works<
.with_para_id(runtime_para_id.into())
.build()
.execute_with(|| {
let mut alice = [0u8; 32];
alice[0] = 1;
let included_head = RuntimeHelper::<Runtime, AllPalletsWithoutSystem>::run_to_block(
2,
AccountId::from(alice),
);
// check Balances before
assert_eq!(<pallet_balances::Pallet<Runtime>>::free_balance(&target_account), 0.into());
assert_eq!(
@@ -166,6 +180,8 @@ pub fn teleports_for_native_asset_works<
dest_beneficiary,
(native_asset_id, native_asset_to_teleport_away.into()),
None,
included_head.clone(),
&alice,
));
// check balances
assert_eq!(
@@ -211,6 +227,8 @@ pub fn teleports_for_native_asset_works<
dest_beneficiary,
(native_asset_id, native_asset_to_teleport_away.into()),
Some((runtime_para_id, other_para_id)),
included_head,
&alice,
));
// check balances
@@ -240,6 +258,7 @@ pub fn teleports_for_native_asset_works<
macro_rules! include_teleports_for_native_asset_works(
(
$runtime:path,
$all_pallets_without_system:path,
$xcm_config:path,
$checking_account:path,
$weight_to_fee:path,
@@ -257,6 +276,7 @@ macro_rules! include_teleports_for_native_asset_works(
$crate::test_cases::teleports_for_native_asset_works::<
$runtime,
$all_pallets_without_system,
$xcm_config,
$checking_account,
$weight_to_fee,
@@ -277,6 +297,7 @@ macro_rules! include_teleports_for_native_asset_works(
/// chain
pub fn teleports_for_foreign_assets_works<
Runtime,
AllPalletsWithoutSystem,
XcmConfig,
CheckingAccount,
WeightToFee,
@@ -302,6 +323,8 @@ pub fn teleports_for_foreign_assets_works<
+ cumulus_pallet_parachain_system::Config
+ cumulus_pallet_xcmp_queue::Config
+ pallet_assets::Config<ForeignAssetsPalletInstance>,
AllPalletsWithoutSystem:
OnInitialize<BlockNumberFor<Runtime>> + OnFinalize<BlockNumberFor<Runtime>>,
AccountIdOf<Runtime>: Into<[u8; 32]>,
ValidatorIdOf<Runtime>: From<AccountIdOf<Runtime>>,
BalanceOf<Runtime>: From<Balance>,
@@ -363,6 +386,13 @@ pub fn teleports_for_foreign_assets_works<
.with_tracing()
.build()
.execute_with(|| {
let mut alice = [0u8; 32];
alice[0] = 1;
let included_head = RuntimeHelper::<Runtime, AllPalletsWithoutSystem>::run_to_block(
2,
AccountId::from(alice),
);
// checks target_account before
assert_eq!(
<pallet_balances::Pallet<Runtime>>::free_balance(&target_account),
@@ -514,6 +544,8 @@ pub fn teleports_for_foreign_assets_works<
dest_beneficiary,
(foreign_asset_id_multilocation, asset_to_teleport_away),
Some((runtime_para_id, foreign_para_id)),
included_head,
&alice,
));
// check balances
@@ -558,6 +590,7 @@ pub fn teleports_for_foreign_assets_works<
macro_rules! include_teleports_for_foreign_assets_works(
(
$runtime:path,
$all_pallets_without_system:path,
$xcm_config:path,
$checking_account:path,
$weight_to_fee:path,
@@ -578,6 +611,7 @@ macro_rules! include_teleports_for_foreign_assets_works(
$crate::test_cases::teleports_for_foreign_assets_works::<
$runtime,
$all_pallets_without_system,
$xcm_config,
$checking_account,
$weight_to_fee,