pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-05-30 11:41:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Reduce the inclusion inherent's actual weight if the block is already heavy (#2060)

Browse Source 
* don't modify inherent data on heavy block

* write up current thinking on block weight detection

* extract inherent inclusion check into its own function

* put heavy block check into runtime

* the `inclusion` inherent call is Operational, not Mandatory

This resolves a lot of the trickiness about this issue, because
we no longer need to override or supplant any existing proposer
logic; the existing logic should exhibit these behaviors:

- the `inclusion` inherent is prioritized over standard transactions
- but if it's too heavy, i.e. in case of runtime upgrade, it'll be
  dropped in favor of that.

It is my belief that allowing the proposer to just not include
this data won't have any adverse effects: it's equivalent to replacing
them with empty versions of themselves, which the `ProvideInherent`
impl already does.

* Revert "the `inclusion` inherent call is Operational, not Mandatory"

This reverts commit e58858d109b18b84e7af3ac47981c6900b2d9a3e.

* Revert "write up current thinking on block weight detection"

This reverts commit fd587b80c46761b2a2b62448193348237863f99f.

* Revert "don't modify inherent data on heavy block"

This reverts commit 38299d3c23e9efb5a354d8cfa658e62a5c8c7ddf.

* add backed candidate block weight assumption to configuration

* Limit backed candidates according to a candidate weight heuristic.

This approach replaces making the inclusion inherent non-mandatory.
It's still not ideal in that we have to configure a heuristic for
how much each backed candidate 'weighs', instead of directly
measuring it somehow.

This approach also never truncates the signed bitfields. The
rationale for that depends on some assumptions:

- processing the signed bitfields is cheap compared to the
  backed candidates
- it is beneficial to the progress of the relay chain
  to update the signed bitfields even if not all backed candidates
  are updated

* simplify limit_backed_candidates and weight assumption

* don't trust the provisioner to fairly distribute candidates

* use saturating subtraction

* empty commit to restart ci

* use new mechanism for getting max block weight

* apply weight refunds to the inclusion inherent

This makes some assumptions about fundamental weights, which are
encapsulated as constants. From there, it lets Substrate know
what the actual computed weight of the inherent is.

* use a correct fixed weight for the inclusion inherent

Co-authored-by:  Guillaume Thiolliere <gui.thiolliere@gmail.com>

* use dynamic inclusion weight so we reduce calculated weight when excluding candidates

* don't double-count this intrinsic's weight in the block weight

* add unit tests of fn limit_backed_candidates

* add tests that the inclusion inherent's weight correctly updates

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
This commit is contained in:
Peter Goodspeed-Niklaus
2021-01-05 16:02:06 +01:00
committed by GitHub
parent 8d05d228fe
commit fcc0fca161
3 changed files with 197 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
polkadot/runtime/parachains/src/inclusion_inherent.rs
+194 -7
View File
@@ -25,10 +25,9 @@ use sp_std::prelude::*;
use primitives::v1::{
BackedCandidate, SignedAvailabilityBitfields, INCLUSION_INHERENT_IDENTIFIER, Header,
};
use sp_runtime::traits::One;
use frame_support::{
decl_error, decl_module, decl_storage, ensure,
dispatch::DispatchResult,
dispatch::DispatchResultWithPostInfo,
weights::{DispatchClass, Weight},
traits::Get,
};
@@ -40,6 +39,12 @@ use crate::{
};
use inherents::{InherentIdentifier, InherentData, MakeFatalError, ProvideInherent};
// In the future, we should benchmark these consts; these are all untested assumptions for now.
const BACKED_CANDIDATE_WEIGHT: Weight = 100_000;
const INCLUSION_INHERENT_CLAIMED_WEIGHT: Weight = 1_000_000_000;
// we assume that 75% of an inclusion inherent's weight is used processing backed candidates
const MINIMAL_INCLUSION_INHERENT_WEIGHT: Weight = INCLUSION_INHERENT_CLAIMED_WEIGHT / 4;
pub trait Config: inclusion::Config + scheduler::Config {}
decl_storage! {
@@ -80,19 +85,21 @@ decl_module! {
}
/// Include backed candidates and bitfields.
#[weight = (1_000_000_000, DispatchClass::Mandatory)]
#[weight = (
MINIMAL_INCLUSION_INHERENT_WEIGHT + backed_candidates.len() as Weight * BACKED_CANDIDATE_WEIGHT,
DispatchClass::Mandatory,
)]
pub fn inclusion(
origin,
signed_bitfields: SignedAvailabilityBitfields,
backed_candidates: Vec<BackedCandidate<T::Hash>>,
parent_header: Header,
) -> DispatchResult {
) -> DispatchResultWithPostInfo {
ensure_none(origin)?;
ensure!(!<Included>::exists(), Error::<T>::TooManyInclusionInherents);
// Check that the submitted parent header indeed corresponds to the previous block hash.
let now = <frame_system::Module<T>>::block_number();
let parent_hash = <frame_system::Module<T>>::block_hash(now - One::one());
let parent_hash = <frame_system::Module<T>>::parent_hash();
ensure!(
parent_header.hash().as_ref() == parent_hash.as_ref(),
Error::<T>::InvalidParentHeader,
@@ -119,6 +126,9 @@ decl_module! {
<scheduler::Module<T>>::schedule(freed);
let backed_candidates = limit_backed_candidates::<T>(backed_candidates);
let backed_candidates_len = backed_candidates.len() as Weight;
// Process backed candidates according to scheduled cores.
let parent_storage_root = parent_header.state_root;
let occupied = <inclusion::Module<T>>::process_candidates(
@@ -137,11 +147,35 @@ decl_module! {
// And track that we've finished processing the inherent for this block.
Included::set(Some(()));
Ok(())
Ok(Some(
MINIMAL_INCLUSION_INHERENT_WEIGHT +
(backed_candidates_len * BACKED_CANDIDATE_WEIGHT)
).into())
}
}
}
/// Limit the number of backed candidates processed in order to stay within block weight limits.
///
/// Use a configured assumption about the weight required to process a backed candidate and the
/// current block weight as of the execution of this function to ensure that we don't overload
/// the block with candidate processing.
///
/// If the backed candidates exceed the available block weight remaining, then skips all of them.
/// This is somewhat less desirable than attempting to fit some of them, but is more fair in the
/// even that we can't trust the provisioner to provide a fair / random ordering of candidates.
fn limit_backed_candidates<T: Config>(
backed_candidates: Vec<BackedCandidate<T::Hash>>,
) -> Vec<BackedCandidate<T::Hash>> {
// the weight of the inclusion inherent is already included in the current block weight,
// so our operation is simple: if the block is currently overloaded, make this intrinsic smaller
if frame_system::Module::<T>::block_weight().total() > <T as frame_system::Config>::BlockWeights::get().max_block {
Vec::new()
} else {
backed_candidates
}
}
impl<T: Config> ProvideInherent for Module<T> {
type Call = Call<T>;
type Error = MakeFatalError<()>;
@@ -174,3 +208,156 @@ impl<T: Config> ProvideInherent for Module<T> {
)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::mock::{
new_test_ext, System, GenesisConfig as MockGenesisConfig, Test
};
mod limit_backed_candidates {
use super::*;
#[test]
fn does_not_truncate_on_empty_block() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let backed_candidates = vec![BackedCandidate::default()];
System::set_block_consumed_resources(0, 0);
assert_eq!(limit_backed_candidates::<Test>(backed_candidates).len(), 1);
});
}
#[test]
fn does_not_truncate_on_exactly_full_block() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let backed_candidates = vec![BackedCandidate::default()];
let max_block_weight = <Test as frame_system::Config>::BlockWeights::get().max_block;
// if the consumed resources are precisely equal to the max block weight, we do not truncate.
System::set_block_consumed_resources(max_block_weight, 0);
assert_eq!(limit_backed_candidates::<Test>(backed_candidates).len(), 1);
});
}
#[test]
fn truncates_on_over_full_block() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let backed_candidates = vec![BackedCandidate::default()];
let max_block_weight = <Test as frame_system::Config>::BlockWeights::get().max_block;
// if the consumed resources are precisely equal to the max block weight, we do not truncate.
System::set_block_consumed_resources(max_block_weight + 1, 0);
assert_eq!(limit_backed_candidates::<Test>(backed_candidates).len(), 0);
});
}
#[test]
fn all_backed_candidates_get_truncated() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let backed_candidates = vec![BackedCandidate::default(); 10];
let max_block_weight = <Test as frame_system::Config>::BlockWeights::get().max_block;
// if the consumed resources are precisely equal to the max block weight, we do not truncate.
System::set_block_consumed_resources(max_block_weight + 1, 0);
assert_eq!(limit_backed_candidates::<Test>(backed_candidates).len(), 0);
});
}
}
mod inclusion_inherent_weight {
use super::*;
use crate::mock::{
new_test_ext, System, GenesisConfig as MockGenesisConfig, Test
};
use frame_support::traits::UnfilteredDispatchable;
fn default_header() -> Header {
Header {
parent_hash: Default::default(),
number: 0,
state_root: Default::default(),
extrinsics_root: Default::default(),
digest: Default::default(),
}
}
/// We expect the weight of the inclusion inherent not to change when no truncation occurs:
/// its weight is dynamically computed from the size of the backed candidates list, and is
/// already incorporated into the current block weight when it is selected by the provisioner.
#[test]
fn weight_does_not_change_on_happy_path() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let header = default_header();
System::set_block_number(1);
System::set_parent_hash(header.hash());
// number of bitfields doesn't affect the inclusion inherent weight, so we can mock it with an empty one
let signed_bitfields = Vec::new();
// backed candidates must not be empty, so we can demonstrate that the weight has not changed
let backed_candidates = vec![BackedCandidate::default(); 10];
// the expected weight can always be computed by this formula
let expected_weight = MINIMAL_INCLUSION_INHERENT_WEIGHT +
(backed_candidates.len() as Weight * BACKED_CANDIDATE_WEIGHT);
// we've used half the block weight; there's plenty of margin
let max_block_weight = <Test as frame_system::Config>::BlockWeights::get().max_block;
let used_block_weight = max_block_weight / 2;
System::set_block_consumed_resources(used_block_weight, 0);
// execute the inclusion inherent
let post_info = Call::<Test>::inclusion(signed_bitfields, backed_candidates, default_header())
.dispatch_bypass_filter(None.into()).unwrap_err().post_info;
// we don't directly check the block's weight post-call. Instead, we check that the
// call has returned the appropriate post-dispatch weight for refund, and trust
// Substrate to do the right thing with that information.
//
// In this case, the weight system can update the actual weight with the same amount,
// or return `None` to indicate that the pre-computed weight should not change.
// Either option is acceptable for our purposes.
if let Some(actual_weight) = post_info.actual_weight {
assert_eq!(actual_weight, expected_weight);
}
});
}
/// We expect the weight of the inclusion inherent to change when truncation occurs: its
/// weight was initially dynamically computed from the size of the backed candidates list,
/// but was reduced by truncation.
#[test]
fn weight_changes_when_backed_candidates_are_truncated() {
new_test_ext(MockGenesisConfig::default()).execute_with(|| {
let header = default_header();
System::set_block_number(1);
System::set_parent_hash(header.hash());
// number of bitfields doesn't affect the inclusion inherent weight, so we can mock it with an empty one
let signed_bitfields = Vec::new();
// backed candidates must not be empty, so we can demonstrate that the weight has not changed
let backed_candidates = vec![BackedCandidate::default(); 10];
// the expected weight with no blocks is just the minimum weight
let expected_weight = MINIMAL_INCLUSION_INHERENT_WEIGHT;
// oops, looks like this mandatory call pushed the block weight over the limit
let max_block_weight = <Test as frame_system::Config>::BlockWeights::get().max_block;
let used_block_weight = max_block_weight + 1;
System::set_block_consumed_resources(used_block_weight, 0);
// execute the inclusion inherent
let post_info = Call::<Test>::inclusion(signed_bitfields, backed_candidates, header)
.dispatch_bypass_filter(None.into()).unwrap();
// we don't directly check the block's weight post-call. Instead, we check that the
// call has returned the appropriate post-dispatch weight for refund, and trust
// Substrate to do the right thing with that information.
assert_eq!(
post_info.actual_weight.unwrap(),
expected_weight,
);
});
}
}
}