Implement Approval Voting Subsystem (#2112)

* skeleton

* skeleton aux-schema module

* start approval types

* start aux schema with aux store

* doc

* finish basic types

* start approval types

* doc

* finish basic types

* write out schema types

* add debug and codec impls to approval types

* add debug and codec impls to approval types

also add some key computation

* add debug and codec impls to approval types

* getters for block and candidate entries

* grumbles

* remove unused AssignmentId

* load_decode utility

* implement DB clearing

* function for adding new block entry to aux store

* start `canonicalize` implementation

* more skeleton

* finish implementing canonicalize

* tag TODO

* implement a test AuxStore

* add allow(unused)

* basic loading and deleting test

* block_entry test function

* add a test for `add_block_entry`

* ensure range is exclusive at end

* test clear()

* test that add_block sets children

* add a test for canonicalize

* extract Pre-digest from header

* utilities for extracting RelayVRFStory from the header-chain

* add approval voting message types

* approval distribution message type

* subsystem skeleton

* state struct

* add futures-timer

* prepare service for babe slot duration

* more skeleton

* better integrate AuxStore

* RelayVRF -> RelayVRFStory

* canonicalize

* implement some tick functionality

* guide: tweaks

* check_approval

* more tweaks and helpers

* guide: add core index to candidate event

* primitives: add core index to candidate event

* runtime: add core index to candidate events

* head handling (session window)

* implement `determine_new_blocks`

* add TODO

* change error type on functions

* compute RelayVRFModulo assignments

* compute RelayVRFDelay assignments

* fix delay tranche calc

* assignment checking

* pluralize

* some dummy code for fetching assignments

* guide: add babe epoch runtime API

* implement a current_epoch() runtime API

* compute assignments

* candidate events get backing group

* import blocks and assignments into DB

* push block approval meta

* add message types, no overseer integration yet

* notify approval distribution of new blocks

* refactor import into separate functions

* impl tranches_to_approve

* guide: improve function signatures

* guide: remove Tick from ApprovalEntry

* trigger and broadcast assignment

* most of approval launching

* remove byteorder crate

* load blocks back to finality, except on startup

* check unchecked assignments

* add claimed core to approval voting message

* fix checks

* assign only to backing group

* remove import_checked_assignment from guide

* newline

* import assignments

* abstract out a bit

* check and import approvals

* check full approvals from assignment import too

* comment

* create a Transaction utility

* must_use

* use transaction in `check_full_approvals`

* wire up wakeups

* add Ord to CandidateHash

* wakeup refactoring

* return candidate info from add_block_entry

* schedule wakeups

* background task: do candidate validation

* forward candidate validation requests

* issue approval votes when requested

* clean up a couple TODOs

* fix up session caching

* clean up last unimplemented!() items

* fix remaining warnings

* remove TODO

* implement handle_approved_ancestor

* update Cargo.lock

* fix runtime API tests

* guide: cleanup assignment checking

* use claimed candidate index instead of core

* extract time to a trait

* tests module

* write a mock clock for testing

* allow swapping out the clock

* make abstract over assignment criteria

* add some skeleton tests and simplify params

* fix backing group check

* do backing group check inside check_assignment_cert

* write some empty test functions to implement

* add a test for non-backing

* test that produced checks pass

* some empty test ideas

* runtime/inclusion: remove outdated TODO

* fix compilation

* av-store: fix tests

* dummy cert

* criteria tests

* move `TestStore` to main tests file

* fix unused warning

* test harness beginnings

* resolve slots renaming fallout

* more compilation fixes

* wip: extract pure data into a separate module

* wip: extract pure data into a separate module

* move types completely to v1

* add persisted_entries

* add conversion trait impls

* clean up some warnings

* extract import logic to own module

* schedule wakeups

* experiment with Actions

* uncomment approval-checking

* separate module for approval checking utilities

* port more code to use actions

* get approval pipeline using actions

* all logic is uncommented

* main loop processes actions

* all loop logic uncommented

* separate function for handling actions

* remove last unimplemented item

* clean up warnings

* State gives read-only access to underlying DB

* tests for approval checking

* tests for approval criteria

* skeleton test module for import

* list of import tests to do

* some test glue code

* test reject bad assignment

* test slot too far in future

* test reject assignment with unknown candidate

* remove loads_blocks tests

* determine_new_blocks back to finalized & harness

* more coverage for determining new blocks

* make `imported_block_info` have less reliance on State

* candidate_info tests

* tests for session caching

* remove println

* extricate DB and main TestStores

* rewrite approval checking logic to counteract early delays

* move state out of function

* update approval-checking tests

* tweak wakeups & scheduling logic

* rename check_full_approvals

* test that assignment import updates candidate

* some approval import tests

* some tests for check_and_apply_approval

* add 'full' qualifier to avoid confusion

* extract should-trigger logic to separate function

* some tests for all triggering

* tests for when we trigger assignments

* test wakeups

* add block utilities for testing

* some more tests for approval updates

* approved_ancestor tests

* new action type for launch approval

* process-wakeup tests

* clean up some warnings

* fix in_future test

* approval checking tests

* tighten up too-far-in-future

* special-case genesis when caching sessions

* fix bitfield len

Co-authored-by: Andronik Ordian <write@reusable.software>
This commit is contained in:
Robert Habermeier
2021-02-11 10:21:47 -06:00
committed by GitHub
parent 09eadfc979
commit e48c687504
29 changed files with 7505 additions and 132 deletions
+23 -1
View File
@@ -5235,16 +5235,33 @@ dependencies = [
name = "polkadot-node-core-approval-voting"
version = "0.1.0"
dependencies = [
"assert_matches",
"bitvec",
"futures 0.3.12",
"futures-timer 3.0.2",
"maplit",
"merlin",
"parity-scale-codec",
"parking_lot 0.11.1",
"polkadot-node-primitives",
"polkadot-node-subsystem",
"polkadot-node-subsystem-test-helpers",
"polkadot-overseer",
"polkadot-primitives",
"rand_core 0.5.1",
"sc-client-api",
"sc-keystore",
"schnorrkel",
"sp-application-crypto",
"sp-blockchain",
"sp-consensus-babe",
"sp-consensus-slots",
"sp-core",
"sp-keyring",
"sp-keystore",
"sp-runtime",
"tracing",
"tracing-futures",
]
[[package]]
@@ -5412,11 +5429,13 @@ dependencies = [
"futures 0.3.12",
"memory-lru",
"parity-util-mem",
"polkadot-node-primitives",
"polkadot-node-subsystem",
"polkadot-node-subsystem-test-helpers",
"polkadot-node-subsystem-util",
"polkadot-primitives",
"sp-api",
"sp-consensus-babe",
"sp-core",
"tracing",
"tracing-futures",
@@ -5460,10 +5479,13 @@ dependencies = [
"parity-scale-codec",
"polkadot-primitives",
"polkadot-statement-table",
"sp-consensus-slots",
"schnorrkel",
"sp-application-crypto",
"sp-consensus-babe",
"sp-consensus-vrf",
"sp-core",
"sp-runtime",
"thiserror",
]
[[package]]
+19 -2
View File
@@ -6,16 +6,33 @@ edition = "2018"
[dependencies]
futures = "0.3.8"
futures-timer = "3.0.2"
parity-scale-codec = { version = "2.0.0", default-features = false, features = ["bit-vec", "derive"] }
tracing = "0.1.22"
tracing-futures = "0.2.4"
bitvec = { version = "0.20.1", default-features = false, features = ["alloc"] }
merlin = "2.0"
schnorrkel = "0.9.1"
polkadot-subsystem = { package = "polkadot-node-subsystem", path = "../../subsystem" }
polkadot-overseer = { path = "../../overseer" }
polkadot-primitives = { path = "../../../primitives" }
polkadot-node-primitives = { path = "../../primitives" }
bitvec = "0.20.1"
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sc-keystore = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-consensus-slots = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-blockchain = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-application-crypto = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false, features = ["full_crypto"] }
sp-runtime = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
[dev-dependencies]
[dev-dependencies]
parking_lot = "0.11.1"
rand_core = "0.5.1" # should match schnorrkel
sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-keystore = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-consensus-babe = { git = "https://github.com/paritytech/substrate", branch = "master" }
maplit = "1.0.2"
polkadot-node-subsystem-test-helpers = { path = "../../subsystem-test-helpers" }
assert_matches = "1.4.0"
@@ -0,0 +1,879 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Utilities for checking whether a candidate has been approved under a given block.
use polkadot_node_primitives::approval::DelayTranche;
use bitvec::slice::BitSlice;
use bitvec::order::Lsb0 as BitOrderLsb0;
use crate::persisted_entries::{ApprovalEntry, CandidateEntry};
use crate::time::Tick;
/// The required tranches of assignments needed to determine whether a candidate is approved.
#[derive(Debug, PartialEq, Clone)]
pub enum RequiredTranches {
/// All validators appear to be required, based on tranches already taken and remaining
/// no-shows.
All,
/// More tranches required - We're awaiting more assignments.
Pending {
/// The highest considered delay tranche when counting assignments.
considered: DelayTranche,
/// The tick at which the next no-show, of the assignments counted, would occur.
next_no_show: Option<Tick>,
/// The highest tranche to consider when looking to broadcast own assignment.
/// This should be considered along with the clock drift to avoid broadcasting
/// assignments that are before the local time.
maximum_broadcast: DelayTranche,
/// The clock drift, in ticks, to apply to the local clock when determining whether
/// to broadcast an assignment or when to schedule a wakeup. The local clock should be treated
/// as though it is `clock_drift` ticks earlier.
clock_drift: Tick,
},
/// An exact number of required tranches and a number of no-shows. This indicates that
/// at least the amount of `needed_approvals` are assigned and additionally all no-shows
/// are covered.
Exact {
/// The tranche to inspect up to.
needed: DelayTranche,
/// The amount of missing votes that should be tolerated.
tolerated_missing: usize,
/// When the next no-show would be, if any. This is used to schedule the next wakeup in the
/// event that there are some assignments that don't have corresponding approval votes. If this
/// is `None`, all assignments have approvals.
next_no_show: Option<Tick>,
}
}
/// Check the approval of a candidate.
pub fn check_approval(
candidate: &CandidateEntry,
approval: &ApprovalEntry,
required: RequiredTranches,
) -> bool {
match required {
RequiredTranches::Pending { .. } => false,
RequiredTranches::All => {
let approvals = candidate.approvals();
3 * approvals.count_ones() > 2 * approvals.len()
}
RequiredTranches::Exact { needed, tolerated_missing, .. } => {
// whether all assigned validators up to `needed` less no_shows have approved.
// e.g. if we had 5 tranches and 1 no-show, we would accept all validators in
// tranches 0..=5 except for 1 approving. In that example, we also accept all
// validators in tranches 0..=5 approving, but that would indicate that the
// RequiredTranches value was incorrectly constructed, so it is not realistic.
// If there are more missing approvals than there are no-shows, that indicates
// that there are some assignments which are not yet no-shows, but may become
// no-shows.
let mut assigned_mask = approval.assignments_up_to(needed);
let approvals = candidate.approvals();
let n_assigned = assigned_mask.count_ones();
// Filter the amount of assigned validators by those which have approved.
assigned_mask &= approvals.iter().by_val();
let n_approved = assigned_mask.count_ones();
// note: the process of computing `required` only chooses `exact` if
// that will surpass a minimum amount of checks.
// shouldn't typically go above, since all no-shows are supposed to be covered.
n_approved + tolerated_missing >= n_assigned
}
}
}
// Determining the amount of tranches required for approval or which assignments are pending
// involves moving through a series of states while looping over the tranches
//
// that we are aware of. First, we perform an initial count of the number of assignments
// until we reach the number of needed assignments for approval. As we progress, we count the
// number of no-shows in each tranche.
//
// Then, if there are any no-shows, we proceed into a series of subsequent states for covering
// no-shows.
//
// We cover each no-show by a non-empty tranche, keeping track of the amount of further
// no-shows encountered along the way. Once all of the no-shows we were previously aware
// of are covered, we then progress to cover the no-shows we encountered while covering those,
// and so on.
#[derive(Debug)]
struct State {
/// The total number of assignments obtained.
assignments: usize,
/// The depth of no-shows we are currently covering.
depth: usize,
/// The amount of no-shows that have been covered at the previous or current depths.
covered: usize,
/// The amount of assignments that we are attempting to cover at this depth.
///
/// At depth 0, these are the initial needed approvals, and at other depths these
/// are no-shows.
covering: usize,
/// The number of uncovered no-shows encountered at this depth. These will be the
/// `covering` of the next depth.
uncovered: usize,
/// The next tick at which a no-show would occur, if any.
next_no_show: Option<Tick>,
}
impl State {
fn output(
&self,
tranche: DelayTranche,
needed_approvals: usize,
n_validators: usize,
no_show_duration: Tick,
) -> RequiredTranches {
let covering = if self.depth == 0 { 0 } else { self.covering };
if self.depth != 0 && self.assignments + covering + self.uncovered >= n_validators {
return RequiredTranches::All;
}
// If we have enough assignments and all no-shows are covered, we have reached the number
// of tranches that we need to have.
if self.assignments >= needed_approvals && (covering + self.uncovered) == 0 {
return RequiredTranches::Exact {
needed: tranche,
tolerated_missing: self.covered,
next_no_show: self.next_no_show,
};
}
// We're pending more assignments and should look at more tranches.
let clock_drift = self.clock_drift(no_show_duration);
if self.depth == 0 {
RequiredTranches::Pending {
considered: tranche,
next_no_show: self.next_no_show,
// during the initial assignment-gathering phase, we want to accept assignments
// from any tranche. Note that honest validators will still not broadcast their
// assignment until it is time to do so, regardless of this value.
maximum_broadcast: DelayTranche::max_value(),
clock_drift,
}
} else {
RequiredTranches::Pending {
considered: tranche,
next_no_show: self.next_no_show,
maximum_broadcast: tranche + (covering + self.uncovered) as DelayTranche,
clock_drift,
}
}
}
fn clock_drift(&self, no_show_duration: Tick) -> Tick {
self.depth as Tick * no_show_duration
}
fn advance(
&self,
new_assignments: usize,
new_no_shows: usize,
next_no_show: Option<Tick>,
) -> State {
let new_covered = if self.depth == 0 {
new_assignments
} else {
// When covering no-shows, we treat each non-empty tranche as covering 1 assignment,
// regardless of how many assignments are within the tranche.
new_assignments.min(1)
};
let assignments = self.assignments + new_assignments;
let covering = self.covering.saturating_sub(new_covered);
let covered = if self.depth == 0 {
// If we're at depth 0, we're not actually covering no-shows,
// so we don't need to count them as such.
0
} else {
self.covered + new_covered
};
let uncovered = self.uncovered + new_no_shows;
let next_no_show = super::min_prefer_some(
self.next_no_show,
next_no_show,
);
let (depth, covering, uncovered) = if covering == 0 {
if uncovered == 0 {
(self.depth, 0, uncovered)
} else {
(self.depth + 1, uncovered, 0)
}
} else {
(self.depth, covering, uncovered)
};
State { assignments, depth, covered, covering, uncovered, next_no_show }
}
}
/// Determine the amount of tranches of assignments needed to determine approval of a candidate.
pub fn tranches_to_approve(
approval_entry: &ApprovalEntry,
approvals: &BitSlice<BitOrderLsb0, u8>,
tranche_now: DelayTranche,
block_tick: Tick,
no_show_duration: Tick,
needed_approvals: usize,
) -> RequiredTranches {
let tick_now = tranche_now as Tick + block_tick;
let n_validators = approval_entry.n_validators();
let initial_state = State {
assignments: 0,
depth: 0,
covered: 0,
covering: needed_approvals,
uncovered: 0,
next_no_show: None,
};
// The `ApprovalEntry` doesn't have any data for empty tranches. We still want to iterate over
// these empty tranches, so we create an iterator to fill the gaps.
//
// This iterator has an infinitely long amount of non-empty tranches appended to the end.
let tranches_with_gaps_filled = {
let mut gap_end = 0;
let approval_entries_filled = approval_entry.tranches()
.iter()
.flat_map(move |tranche_entry| {
let tranche = tranche_entry.tranche();
let assignments = tranche_entry.assignments();
let gap_start = gap_end + 1;
gap_end = tranche;
(gap_start..tranche).map(|i| (i, &[] as &[_]))
.chain(std::iter::once((tranche, assignments)))
});
let pre_end = approval_entry.tranches().first().map(|t| t.tranche());
let post_start = approval_entry.tranches().last().map_or(0, |t| t.tranche() + 1);
let pre = pre_end.into_iter()
.flat_map(|pre_end| (0..pre_end).map(|i| (i, &[] as &[_])));
let post = (post_start..).map(|i| (i, &[] as &[_]));
pre.chain(approval_entries_filled).chain(post)
};
tranches_with_gaps_filled
.scan(Some(initial_state), |state, (tranche, assignments)| {
// The `Option` here is used for early exit.
let s = match state.take() {
None => return None,
Some(s) => s,
};
let clock_drift = s.clock_drift(no_show_duration);
let drifted_tick_now = tick_now.saturating_sub(clock_drift);
let drifted_tranche_now = drifted_tick_now.saturating_sub(block_tick) as DelayTranche;
// Break the loop once we've taken enough tranches.
// Note that we always take tranche 0 as `drifted_tranche_now` cannot be less than 0.
if tranche > drifted_tranche_now {
return None;
}
let n_assignments = assignments.len();
// count no-shows. An assignment is a no-show if there is no corresponding approval vote
// after a fixed duration.
//
// While we count the no-shows, we also determine the next possible no-show we might
// see within this tranche.
let mut next_no_show = None;
let no_shows = {
let next_no_show = &mut next_no_show;
assignments.iter()
.map(|(v_index, tick)| (v_index, tick.saturating_sub(clock_drift) + no_show_duration))
.filter(|&(v_index, no_show_at)| {
let has_approved = approvals.get(*v_index as usize).map(|b| *b).unwrap_or(false);
let is_no_show = !has_approved && no_show_at <= drifted_tick_now;
if !is_no_show && !has_approved {
*next_no_show = super::min_prefer_some(
*next_no_show,
Some(no_show_at + clock_drift),
);
}
is_no_show
}).count()
};
let s = s.advance(n_assignments, no_shows, next_no_show);
let output = s.output(tranche, needed_approvals, n_validators, no_show_duration);
*state = match output {
RequiredTranches::Exact { .. } | RequiredTranches::All => {
// Wipe the state clean so the next iteration of this closure will terminate
// the iterator. This guarantees that we can call `last` further down to see
// either a `Finished` or `Pending` result
None
}
RequiredTranches::Pending { .. } => {
// Pending results are only interesting when they are the last result of the iterator
// i.e. we never achieve a satisfactory level of assignment.
Some(s)
}
};
Some(output)
})
.last()
.expect("the underlying iterator is infinite, starts at 0, and never exits early before tranche 1; qed")
}
#[cfg(test)]
mod tests {
use super::*;
use polkadot_primitives::v1::GroupIndex;
use bitvec::bitvec;
use bitvec::order::Lsb0 as BitOrderLsb0;
use crate::approval_db;
#[test]
fn pending_is_not_approved() {
let candidate = approval_db::v1::CandidateEntry {
candidate: Default::default(),
session: 0,
block_assignments: Default::default(),
approvals: Default::default(),
}.into();
let approval_entry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: Default::default(),
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
assert!(!check_approval(
&candidate,
&approval_entry,
RequiredTranches::Pending {
considered: 0,
next_no_show: None,
maximum_broadcast: 0,
clock_drift: 0,
},
));
}
#[test]
fn all_requires_supermajority() {
let mut candidate: CandidateEntry = approval_db::v1::CandidateEntry {
candidate: Default::default(),
session: 0,
block_assignments: Default::default(),
approvals: bitvec![BitOrderLsb0, u8; 0; 10],
}.into();
for i in 0..6 {
candidate.mark_approval(i);
}
let approval_entry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 1; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
assert!(!check_approval(&candidate, &approval_entry, RequiredTranches::All));
candidate.mark_approval(6);
assert!(check_approval(&candidate, &approval_entry, RequiredTranches::All));
}
#[test]
fn exact_takes_only_assignments_up_to() {
let mut candidate: CandidateEntry = approval_db::v1::CandidateEntry {
candidate: Default::default(),
session: 0,
block_assignments: Default::default(),
approvals: bitvec![BitOrderLsb0, u8; 0; 10],
}.into();
for i in 0..6 {
candidate.mark_approval(i);
}
let approval_entry = approval_db::v1::ApprovalEntry {
tranches: vec![
approval_db::v1::TrancheEntry {
tranche: 0,
assignments: (0..4).map(|i| (i, 0.into())).collect(),
},
approval_db::v1::TrancheEntry {
tranche: 1,
assignments: (4..6).map(|i| (i, 1.into())).collect(),
},
approval_db::v1::TrancheEntry {
tranche: 2,
assignments: (6..10).map(|i| (i, 0.into())).collect(),
},
],
assignments: bitvec![BitOrderLsb0, u8; 1; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
assert!(check_approval(
&candidate,
&approval_entry,
RequiredTranches::Exact {
needed: 1,
tolerated_missing: 0,
next_no_show: None,
},
));
assert!(!check_approval(
&candidate,
&approval_entry,
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 0,
next_no_show: None,
},
));
assert!(check_approval(
&candidate,
&approval_entry,
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 4,
next_no_show: None,
},
));
}
#[test]
fn tranches_to_approve_everyone_present() {
let block_tick = 0;
let no_show_duration = 10;
let needed_approvals = 4;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; 5],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick + 1);
approval_entry.import_assignment(1, 3, block_tick + 1);
approval_entry.import_assignment(2, 4, block_tick + 2);
let approvals = bitvec![BitOrderLsb0, u8; 1; 5];
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
2,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact { needed: 1, tolerated_missing: 0, next_no_show: None },
);
}
#[test]
fn tranches_to_approve_not_enough_initial_count() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(1, 2, block_tick);
let approvals = bitvec![BitOrderLsb0, u8; 0; 10];
let tranche_now = 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 2,
next_no_show: Some(block_tick + no_show_duration),
maximum_broadcast: DelayTranche::max_value(),
clock_drift: 0,
},
);
}
#[test]
fn tranches_to_approve_no_shows_before_initial_count_treated_same_as_not_initial() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; 10];
approvals.set(0, true);
approvals.set(1, true);
let tranche_now = no_show_duration as DelayTranche + 1;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 11,
next_no_show: None,
maximum_broadcast: DelayTranche::max_value(),
clock_drift: 0,
},
);
}
#[test]
fn tranches_to_approve_cover_no_show_not_enough() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let n_validators = 8;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; n_validators],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick);
approval_entry.import_assignment(1, 3, block_tick);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; n_validators];
approvals.set(0, true);
approvals.set(1, true);
// skip 2
approvals.set(3, true);
let tranche_now = no_show_duration as DelayTranche + 1;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 1,
next_no_show: None,
maximum_broadcast: 2, // tranche 1 + 1 no-show
clock_drift: 1 * no_show_duration,
}
);
approvals.set(0, false);
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 1,
next_no_show: None,
maximum_broadcast: 3, // tranche 1 + 2 no-shows
clock_drift: 1 * no_show_duration,
}
);
}
#[test]
fn tranches_to_approve_multi_cover_not_enough() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let n_validators = 8;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; n_validators],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick + 1);
approval_entry.import_assignment(1, 3, block_tick + 1);
approval_entry.import_assignment(2, 4, block_tick + no_show_duration + 2);
approval_entry.import_assignment(2, 5, block_tick + no_show_duration + 2);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; n_validators];
approvals.set(0, true);
approvals.set(1, true);
// skip 2
approvals.set(3, true);
// skip 4
approvals.set(5, true);
let tranche_now = 1;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 1,
tolerated_missing: 0,
next_no_show: Some(block_tick + no_show_duration + 1),
},
);
// first no-show covered.
let tranche_now = no_show_duration as DelayTranche + 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 1,
next_no_show: Some(block_tick + 2*no_show_duration + 2),
},
);
// another no-show in tranche 2.
let tranche_now = (no_show_duration * 2) as DelayTranche + 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 2,
next_no_show: None,
maximum_broadcast: 3, // tranche 2 + 1 uncovered no-show.
clock_drift: 2 * no_show_duration,
},
);
}
#[test]
fn tranches_to_approve_cover_no_show() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let n_validators = 8;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; n_validators],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick + 1);
approval_entry.import_assignment(1, 3, block_tick + 1);
approval_entry.import_assignment(2, 4, block_tick + no_show_duration + 2);
approval_entry.import_assignment(2, 5, block_tick + no_show_duration + 2);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; n_validators];
approvals.set(0, true);
approvals.set(1, true);
// skip 2
approvals.set(3, true);
approvals.set(4, true);
approvals.set(5, true);
let tranche_now = no_show_duration as DelayTranche + 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 1,
next_no_show: None,
},
);
// Even though tranche 2 has 2 validators, it only covers 1 no-show.
// to cover a second no-show, we need to take another non-empty tranche.
approvals.set(0, false);
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 2,
next_no_show: None,
maximum_broadcast: 3,
clock_drift: no_show_duration,
},
);
approval_entry.import_assignment(3, 6, block_tick);
approvals.set(6, true);
let tranche_now = no_show_duration as DelayTranche + 3;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 3,
tolerated_missing: 2,
next_no_show: None,
},
);
}
}
#[test]
fn depth_0_covering_not_treated_as_such() {
let state = State {
assignments: 0,
depth: 0,
covered: 0,
covering: 10,
uncovered: 0,
next_no_show: None,
};
assert_eq!(
state.output(0, 10, 10, 20),
RequiredTranches::Pending {
considered: 0,
next_no_show: None,
maximum_broadcast: DelayTranche::max_value(),
clock_drift: 0,
},
);
}
#[test]
fn depth_0_issued_as_exact_even_when_all() {
let state = State {
assignments: 10,
depth: 0,
covered: 0,
covering: 0,
uncovered: 0,
next_no_show: None,
};
assert_eq!(
state.output(0, 10, 10, 20),
RequiredTranches::Exact {
needed: 0,
tolerated_missing: 0,
next_no_show: None,
},
);
}
@@ -0,0 +1,33 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Approval DB accessors and writers for on-disk persisted approval storage
//! data.
//!
//! We persist data to disk although it is not intended to be used across runs of the
//! program. This is because under medium to long periods of finality stalling, for whatever
//! reason that may be, the amount of data we'd need to keep would be potentially too large
//! for memory.
//!
//! With tens or hundreds of parachains, hundreds of validators, and parablocks
//! in every relay chain block, there can be a humongous amount of information to reference
//! at any given time.
//!
//! As such, we provide a function from this module to clear the database on start-up.
//! In the future, we may use a temporary DB which doesn't need to be wiped, but for the
//! time being we share the same DB with the rest of Substrate.
pub mod v1;
@@ -14,27 +14,10 @@
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Auxiliary DB schema, accessors, and writers for on-disk persisted approval storage
//! data.
//!
//! We persist data to disk although it is not intended to be used across runs of the
//! program. This is because under medium to long periods of finality stalling, for whatever
//! reason that may be, the amount of data we'd need to keep would be potentially too large
//! for memory.
//!
//! With tens or hundreds of parachains, hundreds of validators, and parablocks
//! in every relay chain block, there can be a humongous amount of information to reference
//! at any given time.
//!
//! As such, we provide a function from this module to clear the database on start-up.
//! In the future, we may use a temporary DB which doesn't need to be wiped, but for the
//! time being we share the same DB with the rest of Substrate.
// TODO https://github.com/paritytech/polkadot/issues/1975: remove this
#![allow(unused)]
//! Version 1 of the DB schema.
use sc_client_api::backend::AuxStore;
use polkadot_node_primitives::approval::{DelayTranche, RelayVRF};
use polkadot_node_primitives::approval::{DelayTranche, AssignmentCert};
use polkadot_primitives::v1::{
ValidatorIndex, GroupIndex, CandidateReceipt, SessionIndex, CoreIndex,
BlockNumber, Hash, CandidateHash,
@@ -46,73 +29,95 @@ use std::collections::{BTreeMap, HashMap};
use std::collections::hash_map::Entry;
use bitvec::{vec::BitVec, order::Lsb0 as BitOrderLsb0};
use super::Tick;
#[cfg(test)]
mod tests;
// slot_duration * 2 + DelayTranche gives the number of delay tranches since the
// unix epoch.
#[derive(Encode, Decode, Clone, Copy, Debug, PartialEq)]
pub struct Tick(u64);
pub type Bitfield = BitVec<BitOrderLsb0, u8>;
const STORED_BLOCKS_KEY: &[u8] = b"Approvals_StoredBlocks";
/// Details pertaining to our assignment on a block.
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct OurAssignment {
pub cert: AssignmentCert,
pub tranche: DelayTranche,
pub validator_index: ValidatorIndex,
// Whether the assignment has been triggered already.
pub triggered: bool,
}
/// Metadata regarding a specific tranche of assignments for a specific candidate.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct TrancheEntry {
tranche: DelayTranche,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct TrancheEntry {
pub tranche: DelayTranche,
// Assigned validators, and the instant we received their assignment, rounded
// to the nearest tick.
assignments: Vec<(ValidatorIndex, Tick)>,
pub assignments: Vec<(ValidatorIndex, Tick)>,
}
/// Metadata regarding approval of a particular candidate within the context of some
/// particular block.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct ApprovalEntry {
tranches: Vec<TrancheEntry>,
backing_group: GroupIndex,
// When the next wakeup for this entry should occur. This is either to
// check a no-show or to check if we need to broadcast an assignment.
next_wakeup: Tick,
our_assignment: Option<OurAssignment>,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct ApprovalEntry {
pub tranches: Vec<TrancheEntry>,
pub backing_group: GroupIndex,
pub our_assignment: Option<OurAssignment>,
// `n_validators` bits.
assignments: BitVec<BitOrderLsb0, u8>,
approved: bool,
pub assignments: Bitfield,
pub approved: bool,
}
/// Metadata regarding approval of a particular candidate.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct CandidateEntry {
candidate: CandidateReceipt,
session: SessionIndex,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct CandidateEntry {
pub candidate: CandidateReceipt,
pub session: SessionIndex,
// Assignments are based on blocks, so we need to track assignments separately
// based on the block we are looking at.
block_assignments: BTreeMap<Hash, ApprovalEntry>,
approvals: BitVec<BitOrderLsb0, u8>,
pub block_assignments: BTreeMap<Hash, ApprovalEntry>,
pub approvals: Bitfield,
}
/// Metadata regarding approval of a particular block, by way of approval of the
/// candidates contained within it.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct BlockEntry {
block_hash: Hash,
session: SessionIndex,
slot: Slot,
relay_vrf_story: RelayVRF,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct BlockEntry {
pub block_hash: Hash,
pub session: SessionIndex,
pub slot: Slot,
/// Random bytes derived from the VRF submitted within the block by the block
/// author as a credential and used as input to approval assignment criteria.
pub relay_vrf_story: [u8; 32],
// The candidates included as-of this block and the index of the core they are
// leaving. Sorted ascending by core index.
candidates: Vec<(CoreIndex, CandidateHash)>,
pub candidates: Vec<(CoreIndex, CandidateHash)>,
// A bitfield where the i'th bit corresponds to the i'th candidate in `candidates`.
// The i'th bit is `true` iff the candidate has been approved in the context of this
// block. The block can be considered approved if the bitfield has all bits set to `true`.
approved_bitfield: BitVec<BitOrderLsb0, u8>,
children: Vec<Hash>,
pub approved_bitfield: Bitfield,
pub children: Vec<Hash>,
}
/// A range from earliest..last block number stored within the DB.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct StoredBlockRange(BlockNumber, BlockNumber);
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct StoredBlockRange(BlockNumber, BlockNumber);
// TODO https://github.com/paritytech/polkadot/issues/1975: probably in lib.rs
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct OurAssignment { }
impl From<crate::Tick> for Tick {
fn from(tick: crate::Tick) -> Tick {
Tick(tick)
}
}
impl From<Tick> for crate::Tick {
fn from(tick: Tick) -> crate::Tick {
tick.0
}
}
/// Canonicalize some particular block, pruning everything before it and
/// pruning any competing branches at the same height.
@@ -351,9 +356,9 @@ fn load_decode<D: Decode>(store: &impl AuxStore, key: &[u8])
/// candidate and approval entries.
#[derive(Clone)]
pub(crate) struct NewCandidateInfo {
candidate: CandidateReceipt,
backing_group: GroupIndex,
our_assignment: Option<OurAssignment>,
pub candidate: CandidateReceipt,
pub backing_group: GroupIndex,
pub our_assignment: Option<OurAssignment>,
}
/// Record a new block entry.
@@ -364,7 +369,8 @@ pub(crate) struct NewCandidateInfo {
/// parent hash.
///
/// Has no effect if there is already an entry for the block or `candidate_info` returns
/// `None` for any of the candidates referenced by the block entry.
/// `None` for any of the candidates referenced by the block entry. In these cases,
/// no information about new candidates will be referred to by this function.
pub(crate) fn add_block_entry(
store: &impl AuxStore,
parent_hash: Hash,
@@ -372,7 +378,7 @@ pub(crate) fn add_block_entry(
entry: BlockEntry,
n_validators: usize,
candidate_info: impl Fn(&CandidateHash) -> Option<NewCandidateInfo>,
) -> sp_blockchain::Result<()> {
) -> sp_blockchain::Result<Vec<(CandidateHash, CandidateEntry)>> {
let session = entry.session;
let new_block_range = {
@@ -392,13 +398,15 @@ pub(crate) fn add_block_entry(
let mut blocks_at_height = load_blocks_at_height(store, number)?;
if blocks_at_height.contains(&entry.block_hash) {
// seems we already have a block entry for this block. nothing to do here.
return Ok(())
return Ok(Vec::new())
}
blocks_at_height.push(entry.block_hash);
(blocks_at_height_key(number), blocks_at_height.encode())
};
let mut candidate_entries = Vec::with_capacity(entry.candidates.len());
let candidate_entry_updates = {
let mut updated_entries = Vec::with_capacity(entry.candidates.len());
for &(_, ref candidate_hash) in &entry.candidates {
@@ -407,7 +415,7 @@ pub(crate) fn add_block_entry(
backing_group,
our_assignment,
} = match candidate_info(candidate_hash) {
None => return Ok(()),
None => return Ok(Vec::new()),
Some(info) => info,
};
@@ -424,7 +432,6 @@ pub(crate) fn add_block_entry(
ApprovalEntry {
tranches: Vec::new(),
backing_group,
next_wakeup: 0,
our_assignment,
assignments: bitvec::bitvec![BitOrderLsb0, u8; 0; n_validators],
approved: false,
@@ -434,6 +441,8 @@ pub(crate) fn add_block_entry(
updated_entries.push(
(candidate_entry_key(&candidate_hash), candidate_entry.encode())
);
candidate_entries.push((*candidate_hash, candidate_entry));
}
updated_entries
@@ -466,11 +475,61 @@ pub(crate) fn add_block_entry(
store.insert_aux(&all_keys_and_values, &[])?;
Ok(())
Ok(candidate_entries)
}
// An atomic transaction of multiple candidate or block entries.
#[derive(Default)]
#[must_use = "Transactions do nothing unless written to a DB"]
pub struct Transaction {
block_entries: HashMap<Hash, BlockEntry>,
candidate_entries: HashMap<CandidateHash, CandidateEntry>,
}
impl Transaction {
/// Put a block entry in the transaction, overwriting any other with the
/// same hash.
pub(crate) fn put_block_entry(&mut self, entry: BlockEntry) {
let hash = entry.block_hash;
let _ = self.block_entries.insert(hash, entry);
}
/// Put a candidate entry in the transaction, overwriting any other with the
/// same hash.
pub(crate) fn put_candidate_entry(&mut self, hash: CandidateHash, entry: CandidateEntry) {
let _ = self.candidate_entries.insert(hash, entry);
}
/// Write the contents of the transaction, atomically, to the DB.
pub(crate) fn write(self, db: &impl AuxStore) -> sp_blockchain::Result<()> {
if self.block_entries.is_empty() && self.candidate_entries.is_empty() {
return Ok(())
}
let blocks: Vec<_> = self.block_entries.into_iter().map(|(hash, entry)| {
let k = block_entry_key(&hash);
let v = entry.encode();
(k, v)
}).collect();
let candidates: Vec<_> = self.candidate_entries.into_iter().map(|(hash, entry)| {
let k = candidate_entry_key(&hash);
let v = entry.encode();
(k, v)
}).collect();
let kv = blocks.iter().map(|(k, v)| (&k[..], &v[..]))
.chain(candidates.iter().map(|(k, v)| (&k[..], &v[..])))
.collect::<Vec<_>>();
db.insert_aux(&kv, &[])
}
}
/// Load the stored-blocks key from the state.
pub(crate) fn load_stored_blocks(store: &impl AuxStore)
fn load_stored_blocks(store: &impl AuxStore)
-> sp_blockchain::Result<Option<StoredBlockRange>>
{
load_decode(store, STORED_BLOCKS_KEY)
@@ -17,8 +17,8 @@
//! Tests for the aux-schema of approval voting.
use super::*;
use std::cell::RefCell;
use polkadot_primitives::v1::Id as ParaId;
use std::cell::RefCell;
#[derive(Default)]
struct TestStore {
@@ -49,28 +49,28 @@ impl AuxStore for TestStore {
}
impl TestStore {
fn write_stored_blocks(&self, range: StoredBlockRange) {
pub(crate) fn write_stored_blocks(&self, range: StoredBlockRange) {
self.inner.borrow_mut().insert(
STORED_BLOCKS_KEY.to_vec(),
range.encode(),
);
}
fn write_blocks_at_height(&self, height: BlockNumber, blocks: &[Hash]) {
pub(crate) fn write_blocks_at_height(&self, height: BlockNumber, blocks: &[Hash]) {
self.inner.borrow_mut().insert(
blocks_at_height_key(height).to_vec(),
blocks.encode(),
);
}
fn write_block_entry(&self, block_hash: &Hash, entry: &BlockEntry) {
pub(crate) fn write_block_entry(&self, block_hash: &Hash, entry: &BlockEntry) {
self.inner.borrow_mut().insert(
block_entry_key(block_hash).to_vec(),
entry.encode(),
);
}
fn write_candidate_entry(&self, candidate_hash: &CandidateHash, entry: &CandidateEntry) {
pub(crate) fn write_candidate_entry(&self, candidate_hash: &CandidateHash, entry: &CandidateEntry) {
self.inner.borrow_mut().insert(
candidate_entry_key(candidate_hash).to_vec(),
entry.encode(),
@@ -89,8 +89,8 @@ fn make_block_entry(
BlockEntry {
block_hash,
session: 1,
slot: 1.into(),
relay_vrf_story: RelayVRF([0u8; 32]),
slot: Slot::from(1),
relay_vrf_story: [0u8; 32],
approved_bitfield: make_bitvec(candidates.len()),
candidates,
children: Vec::new(),
@@ -129,7 +129,6 @@ fn read_write() {
(hash_a, ApprovalEntry {
tranches: Vec::new(),
backing_group: GroupIndex(1),
next_wakeup: 1000,
our_assignment: None,
assignments: Default::default(),
approved: false,
@@ -156,7 +155,7 @@ fn read_write() {
];
let delete_keys: Vec<_> = delete_keys.iter().map(|k| &k[..]).collect();
store.insert_aux(&[], &delete_keys);
store.insert_aux(&[], &delete_keys).unwrap();
assert!(load_stored_blocks(&store).unwrap().is_none());
assert!(load_blocks_at_height(&store, 1).unwrap().is_empty());
@@ -296,7 +295,6 @@ fn clear_works() {
(hash_a, ApprovalEntry {
tranches: Vec::new(),
backing_group: GroupIndex(1),
next_wakeup: 1000,
our_assignment: None,
assignments: Default::default(),
approved: false,
@@ -331,7 +329,7 @@ fn canonicalize_works() {
// -> B1 -> C1 -> D1
// A -> B2 -> C2 -> D2
//
// We'll canonicalize C1. Everything except D1 should disappear.
// We'll canonicalize C1. Everytning except D1 should disappear.
//
// Candidates:
// Cand1 in B2
@@ -0,0 +1,782 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Assignment criteria VRF generation and checking.
use polkadot_node_primitives::approval::{
self as approval_types, AssignmentCert, AssignmentCertKind, DelayTranche, RelayVRFStory,
};
use polkadot_primitives::v1::{
CoreIndex, ValidatorIndex, SessionInfo, AssignmentPair, AssignmentId, GroupIndex,
};
use sc_keystore::LocalKeystore;
use parity_scale_codec::{Encode, Decode};
use sp_application_crypto::Public;
use merlin::Transcript;
use schnorrkel::vrf::VRFInOut;
use std::collections::HashMap;
use std::collections::hash_map::Entry;
use super::LOG_TARGET;
/// Details pertaining to our assignment on a block.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub struct OurAssignment {
cert: AssignmentCert,
tranche: DelayTranche,
validator_index: ValidatorIndex,
// Whether the assignment has been triggered already.
triggered: bool,
}
impl OurAssignment {
pub(crate) fn cert(&self) -> &AssignmentCert {
&self.cert
}
pub(crate) fn tranche(&self) -> DelayTranche {
self.tranche
}
pub(crate) fn validator_index(&self) -> ValidatorIndex {
self.validator_index
}
pub(crate) fn triggered(&self) -> bool {
self.triggered
}
pub(crate) fn mark_triggered(&mut self) {
self.triggered = true;
}
}
impl From<crate::approval_db::v1::OurAssignment> for OurAssignment {
fn from(entry: crate::approval_db::v1::OurAssignment) -> Self {
OurAssignment {
cert: entry.cert,
tranche: entry.tranche,
validator_index: entry.validator_index,
triggered: entry.triggered,
}
}
}
impl From<OurAssignment> for crate::approval_db::v1::OurAssignment {
fn from(entry: OurAssignment) -> Self {
Self {
cert: entry.cert,
tranche: entry.tranche,
validator_index: entry.validator_index,
triggered: entry.triggered,
}
}
}
fn relay_vrf_modulo_transcript(
relay_vrf_story: RelayVRFStory,
sample: u32,
) -> Transcript {
// combine the relay VRF story with a sample number.
let mut t = Transcript::new(approval_types::RELAY_VRF_MODULO_CONTEXT);
t.append_message(b"RC-VRF", &relay_vrf_story.0);
sample.using_encoded(|s| t.append_message(b"sample", s));
t
}
fn relay_vrf_modulo_core(
vrf_in_out: &VRFInOut,
n_cores: u32,
) -> CoreIndex {
let bytes: [u8; 4] = vrf_in_out.make_bytes(approval_types::CORE_RANDOMNESS_CONTEXT);
// interpret as little-endian u32.
let random_core = u32::from_le_bytes(bytes) % n_cores;
CoreIndex(random_core)
}
fn relay_vrf_delay_transcript(
relay_vrf_story: RelayVRFStory,
core_index: CoreIndex,
) -> Transcript {
let mut t = Transcript::new(approval_types::RELAY_VRF_DELAY_CONTEXT);
t.append_message(b"RC-VRF", &relay_vrf_story.0);
core_index.0.using_encoded(|s| t.append_message(b"core", s));
t
}
fn relay_vrf_delay_tranche(
vrf_in_out: &VRFInOut,
num_delay_tranches: u32,
zeroth_delay_tranche_width: u32,
) -> DelayTranche {
let bytes: [u8; 4] = vrf_in_out.make_bytes(approval_types::TRANCHE_RANDOMNESS_CONTEXT);
// interpret as little-endian u32 and reduce by the number of tranches.
let wide_tranche = u32::from_le_bytes(bytes) % (num_delay_tranches + zeroth_delay_tranche_width);
// Consolidate early results to tranche zero so tranche zero is extra wide.
wide_tranche.saturating_sub(zeroth_delay_tranche_width)
}
fn assigned_core_transcript(core_index: CoreIndex) -> Transcript {
let mut t = Transcript::new(approval_types::ASSIGNED_CORE_CONTEXT);
core_index.0.using_encoded(|s| t.append_message(b"core", s));
t
}
/// Information about the world assignments are being produced in.
#[derive(Clone)]
pub(crate) struct Config {
/// The assignment public keys for validators.
assignment_keys: Vec<AssignmentId>,
/// The groups of validators assigned to each core.
validator_groups: Vec<Vec<ValidatorIndex>>,
/// The number of availability cores used by the protocol during this session.
n_cores: u32,
/// The zeroth delay tranche width.
zeroth_delay_tranche_width: u32,
/// The number of samples we do of relay_vrf_modulo.
relay_vrf_modulo_samples: u32,
/// The number of delay tranches in total.
n_delay_tranches: u32,
}
impl<'a> From<&'a SessionInfo> for Config {
fn from(s: &'a SessionInfo) -> Self {
Config {
assignment_keys: s.assignment_keys.clone(),
validator_groups: s.validator_groups.clone(),
n_cores: s.n_cores.clone(),
zeroth_delay_tranche_width: s.zeroth_delay_tranche_width.clone(),
relay_vrf_modulo_samples: s.relay_vrf_modulo_samples.clone(),
n_delay_tranches: s.n_delay_tranches.clone(),
}
}
}
/// A trait for producing and checking assignments. Used to mock.
pub(crate) trait AssignmentCriteria {
fn compute_assignments(
&self,
keystore: &LocalKeystore,
relay_vrf_story: RelayVRFStory,
config: &Config,
leaving_cores: Vec<(CoreIndex, GroupIndex)>,
) -> HashMap<CoreIndex, OurAssignment>;
fn check_assignment_cert(
&self,
claimed_core_index: CoreIndex,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
assignment: &AssignmentCert,
backing_group: GroupIndex,
) -> Result<DelayTranche, InvalidAssignment>;
}
pub(crate) struct RealAssignmentCriteria;
impl AssignmentCriteria for RealAssignmentCriteria {
fn compute_assignments(
&self,
keystore: &LocalKeystore,
relay_vrf_story: RelayVRFStory,
config: &Config,
leaving_cores: Vec<(CoreIndex, GroupIndex)>,
) -> HashMap<CoreIndex, OurAssignment> {
compute_assignments(
keystore,
relay_vrf_story,
config,
leaving_cores,
)
}
fn check_assignment_cert(
&self,
claimed_core_index: CoreIndex,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
assignment: &AssignmentCert,
backing_group: GroupIndex,
) -> Result<DelayTranche, InvalidAssignment> {
check_assignment_cert(
claimed_core_index,
validator_index,
config,
relay_vrf_story,
assignment,
backing_group,
)
}
}
/// Compute the assignments for a given block. Returns a map containing all assignments to cores in
/// the block. If more than one assignment targets the given core, only the earliest assignment is kept.
///
/// The `leaving_cores` parameter indicates all cores within the block where a candidate was included,
/// as well as the group index backing those.
///
/// The current description of the protocol assigns every validator to check every core. But at different times.
/// The idea is that most assignments are never triggered and fall by the wayside.
///
/// This will not assign to anything the local validator was part of the backing group for.
pub(crate) fn compute_assignments(
keystore: &LocalKeystore,
relay_vrf_story: RelayVRFStory,
config: &Config,
leaving_cores: impl IntoIterator<Item = (CoreIndex, GroupIndex)> + Clone,
) -> HashMap<CoreIndex, OurAssignment> {
let (index, assignments_key): (ValidatorIndex, AssignmentPair) = {
let key = config.assignment_keys.iter().enumerate()
.filter_map(|(i, p)| match keystore.key_pair(p) {
Ok(pair) => Some((i as ValidatorIndex, pair)),
Err(sc_keystore::Error::PairNotFound(_)) => None,
Err(e) => {
tracing::warn!(target: LOG_TARGET, "Encountered keystore error: {:?}", e);
None
}
})
.next();
match key {
None => return Default::default(),
Some(k) => k,
}
};
// Ignore any cores where the assigned group is our own.
let leaving_cores = leaving_cores.into_iter()
.filter(|&(_, ref g)| !is_in_backing_group(&config.validator_groups, index, *g))
.map(|(c, _)| c)
.collect::<Vec<_>>();
let assignments_key: &sp_application_crypto::sr25519::Pair = assignments_key.as_ref();
let assignments_key: &schnorrkel::Keypair = assignments_key.as_ref();
let mut assignments = HashMap::new();
// First run `RelayVRFModulo` for each sample.
compute_relay_vrf_modulo_assignments(
&assignments_key,
index,
config,
relay_vrf_story.clone(),
leaving_cores.iter().cloned(),
&mut assignments,
);
// Then run `RelayVRFDelay` once for the whole block.
compute_relay_vrf_delay_assignments(
&assignments_key,
index,
config,
relay_vrf_story,
leaving_cores,
&mut assignments,
);
assignments
}
fn compute_relay_vrf_modulo_assignments(
assignments_key: &schnorrkel::Keypair,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
leaving_cores: impl IntoIterator<Item = CoreIndex> + Clone,
assignments: &mut HashMap<CoreIndex, OurAssignment>,
) {
for rvm_sample in 0..config.relay_vrf_modulo_samples {
let mut core = Default::default();
let maybe_assignment = {
// Extra scope to ensure borrowing instead of moving core
// into closure.
let core = &mut core;
assignments_key.vrf_sign_extra_after_check(
relay_vrf_modulo_transcript(relay_vrf_story.clone(), rvm_sample),
|vrf_in_out| {
*core = relay_vrf_modulo_core(&vrf_in_out, config.n_cores);
if leaving_cores.clone().into_iter().any(|c| c == *core) {
Some(assigned_core_transcript(*core))
} else {
None
}
}
)
};
if let Some((vrf_in_out, vrf_proof, _)) = maybe_assignment {
// Sanity: `core` is always initialized to non-default here, as the closure above
// has been executed.
let cert = AssignmentCert {
kind: AssignmentCertKind::RelayVRFModulo { sample: rvm_sample },
vrf: (approval_types::VRFOutput(vrf_in_out.to_output()), approval_types::VRFProof(vrf_proof)),
};
// All assignments of type RelayVRFModulo have tranche 0.
assignments.entry(core).or_insert(OurAssignment {
cert,
tranche: 0,
validator_index,
triggered: false,
});
}
}
}
fn compute_relay_vrf_delay_assignments(
assignments_key: &schnorrkel::Keypair,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
leaving_cores: impl IntoIterator<Item = CoreIndex>,
assignments: &mut HashMap<CoreIndex, OurAssignment>,
) {
for core in leaving_cores {
let (vrf_in_out, vrf_proof, _) = assignments_key.vrf_sign(
relay_vrf_delay_transcript(relay_vrf_story.clone(), core),
);
let tranche = relay_vrf_delay_tranche(
&vrf_in_out,
config.n_delay_tranches,
config.zeroth_delay_tranche_width,
);
let cert = AssignmentCert {
kind: AssignmentCertKind::RelayVRFDelay { core_index: core },
vrf: (approval_types::VRFOutput(vrf_in_out.to_output()), approval_types::VRFProof(vrf_proof)),
};
let our_assignment = OurAssignment {
cert,
tranche,
validator_index,
triggered: false,
};
match assignments.entry(core) {
Entry::Vacant(e) => { let _ = e.insert(our_assignment); }
Entry::Occupied(mut e) => if e.get().tranche > our_assignment.tranche {
e.insert(our_assignment);
},
}
}
}
/// Assignment invalid.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct InvalidAssignment;
impl std::fmt::Display for InvalidAssignment {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "Invalid Assignment")
}
}
impl std::error::Error for InvalidAssignment { }
/// Checks the crypto of an assignment cert. Failure conditions:
/// * Validator index out of bounds
/// * VRF signature check fails
/// * VRF output doesn't match assigned core
/// * Core is not covered by extra data in signature
/// * Core index out of bounds
/// * Sample is out of bounds
/// * Validator is present in backing group.
///
/// This function does not check whether the core is actually a valid assignment or not. That should be done
/// outside of the scope of this function.
pub(crate) fn check_assignment_cert(
claimed_core_index: CoreIndex,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
assignment: &AssignmentCert,
backing_group: GroupIndex,
) -> Result<DelayTranche, InvalidAssignment> {
let validator_public = config.assignment_keys
.get(validator_index as usize)
.ok_or(InvalidAssignment)?;
let public = schnorrkel::PublicKey::from_bytes(validator_public.as_slice())
.map_err(|_| InvalidAssignment)?;
if claimed_core_index.0 >= config.n_cores {
return Err(InvalidAssignment);
}
// Check that the validator was not part of the backing group
// and not already assigned.
let is_in_backing = is_in_backing_group(
&config.validator_groups,
validator_index,
backing_group,
);
if is_in_backing {
return Err(InvalidAssignment);
}
let &(ref vrf_output, ref vrf_proof) = &assignment.vrf;
match assignment.kind {
AssignmentCertKind::RelayVRFModulo { sample } => {
if sample >= config.relay_vrf_modulo_samples {
return Err(InvalidAssignment);
}
let (vrf_in_out, _) = public.vrf_verify_extra(
relay_vrf_modulo_transcript(relay_vrf_story, sample),
&vrf_output.0,
&vrf_proof.0,
assigned_core_transcript(claimed_core_index),
).map_err(|_| InvalidAssignment)?;
// ensure that the `vrf_in_out` actually gives us the claimed core.
if relay_vrf_modulo_core(&vrf_in_out, config.n_cores) == claimed_core_index {
Ok(0)
} else {
Err(InvalidAssignment)
}
}
AssignmentCertKind::RelayVRFDelay { core_index } => {
if core_index != claimed_core_index {
return Err(InvalidAssignment);
}
let (vrf_in_out, _) = public.vrf_verify(
relay_vrf_delay_transcript(relay_vrf_story, core_index),
&vrf_output.0,
&vrf_proof.0,
).map_err(|_| InvalidAssignment)?;
Ok(relay_vrf_delay_tranche(
&vrf_in_out,
config.n_delay_tranches,
config.zeroth_delay_tranche_width,
))
}
}
}
fn is_in_backing_group(
validator_groups: &[Vec<ValidatorIndex>],
validator: ValidatorIndex,
group: GroupIndex,
) -> bool {
validator_groups.get(group.0 as usize).map_or(false, |g| g.contains(&validator))
}
#[cfg(test)]
mod tests {
use super::*;
use sp_keystore::CryptoStore;
use sp_keyring::sr25519::Keyring as Sr25519Keyring;
use sp_application_crypto::sr25519;
use sp_core::crypto::Pair as PairT;
use polkadot_primitives::v1::ASSIGNMENT_KEY_TYPE_ID;
use polkadot_node_primitives::approval::{VRFOutput, VRFProof};
// sets up a keystore with the given keyring accounts.
async fn make_keystore(accounts: &[Sr25519Keyring]) -> LocalKeystore {
let store = LocalKeystore::in_memory();
for s in accounts.iter().copied().map(|k| k.to_seed()) {
store.sr25519_generate_new(
ASSIGNMENT_KEY_TYPE_ID,
Some(s.as_str()),
).await.unwrap();
}
store
}
fn assignment_keys(accounts: &[Sr25519Keyring]) -> Vec<AssignmentId> {
assignment_keys_plus_random(accounts, 0)
}
fn assignment_keys_plus_random(accounts: &[Sr25519Keyring], random: usize) -> Vec<AssignmentId> {
let gen_random = (0..random).map(|_|
AssignmentId::from(sr25519::Pair::generate().0.public())
);
accounts.iter()
.map(|k| AssignmentId::from(k.public()))
.chain(gen_random)
.collect()
}
fn basic_groups(n_validators: usize, n_groups: usize) -> Vec<Vec<ValidatorIndex>> {
let size = n_validators / n_groups;
let big_groups = n_validators % n_groups;
let scraps = n_groups * size;
(0..n_groups).map(|i| {
(i * size .. (i + 1) *size)
.chain(if i < big_groups { Some(scraps + i) } else { None })
.map(|j| j as ValidatorIndex)
.collect::<Vec<_>>()
}).collect()
}
// used for generating assignments where the validity of the VRF doesn't matter.
fn garbage_vrf() -> (VRFOutput, VRFProof) {
let key = Sr25519Keyring::Alice.pair();
let key: &schnorrkel::Keypair = key.as_ref();
let (o, p, _) = key.vrf_sign(Transcript::new(b"test-garbage"));
(VRFOutput(o.to_output()), VRFProof(p))
}
#[test]
fn assignments_produced_for_non_backing() {
let keystore = futures::executor::block_on(
make_keystore(&[Sr25519Keyring::Alice])
);
let relay_vrf_story = RelayVRFStory([42u8; 32]);
let assignments = compute_assignments(
&keystore,
relay_vrf_story,
&Config {
assignment_keys: assignment_keys(&[
Sr25519Keyring::Alice,
Sr25519Keyring::Bob,
Sr25519Keyring::Charlie,
]),
validator_groups: vec![vec![0], vec![1, 2]],
n_cores: 2,
zeroth_delay_tranche_width: 10,
relay_vrf_modulo_samples: 3,
n_delay_tranches: 40,
},
vec![(CoreIndex(0), GroupIndex(1)), (CoreIndex(1), GroupIndex(0))],
);
// Note that alice is in group 0, which was the backing group for core 1.
// Alice should have self-assigned to check core 0 but not 1.
assert_eq!(assignments.len(), 1);
assert!(assignments.get(&CoreIndex(0)).is_some());
}
#[test]
fn assign_to_nonzero_core() {
let keystore = futures::executor::block_on(
make_keystore(&[Sr25519Keyring::Alice])
);
let relay_vrf_story = RelayVRFStory([42u8; 32]);
let assignments = compute_assignments(
&keystore,
relay_vrf_story,
&Config {
assignment_keys: assignment_keys(&[
Sr25519Keyring::Alice,
Sr25519Keyring::Bob,
Sr25519Keyring::Charlie,
]),
validator_groups: vec![vec![0], vec![1, 2]],
n_cores: 2,
zeroth_delay_tranche_width: 10,
relay_vrf_modulo_samples: 3,
n_delay_tranches: 40,
},
vec![(CoreIndex(0), GroupIndex(0)), (CoreIndex(1), GroupIndex(1))],
);
assert_eq!(assignments.len(), 1);
assert!(assignments.get(&CoreIndex(1)).is_some());
}
struct MutatedAssignment {
core: CoreIndex,
cert: AssignmentCert,
group: GroupIndex,
own_group: GroupIndex,
val_index: ValidatorIndex,
config: Config,
}
// This fails if the closure requests to skip everything.
fn check_mutated_assignments(
n_validators: usize,
n_cores: usize,
rotation_offset: usize,
f: impl Fn(&mut MutatedAssignment) -> Option<bool>, // None = skip
) {
let keystore = futures::executor::block_on(
make_keystore(&[Sr25519Keyring::Alice])
);
let group_for_core = |i| GroupIndex(((i + rotation_offset) % n_cores) as _);
let config = Config {
assignment_keys: assignment_keys_plus_random(&[Sr25519Keyring::Alice], n_validators - 1),
validator_groups: basic_groups(n_validators, n_cores),
n_cores: n_cores as u32,
zeroth_delay_tranche_width: 10,
relay_vrf_modulo_samples: 3,
n_delay_tranches: 40,
};
let relay_vrf_story = RelayVRFStory([42u8; 32]);
let assignments = compute_assignments(
&keystore,
relay_vrf_story.clone(),
&config,
(0..n_cores)
.map(|i| (
CoreIndex(i as u32),
group_for_core(i),
))
.collect::<Vec<_>>(),
);
let mut counted = 0;
for (core, assignment) in assignments {
let mut mutated = MutatedAssignment {
core,
group: group_for_core(core.0 as _),
cert: assignment.cert,
own_group: GroupIndex(0),
val_index: 0,
config: config.clone(),
};
let expected = match f(&mut mutated) {
None => continue,
Some(e) => e,
};
counted += 1;
let is_good = check_assignment_cert(
mutated.core,
mutated.val_index,
&mutated.config,
relay_vrf_story.clone(),
&mutated.cert,
mutated.group,
).is_ok();
assert_eq!(expected, is_good)
}
assert!(counted > 0);
}
#[test]
fn computed_assignments_pass_checks() {
check_mutated_assignments(200, 100, 25, |_| Some(true));
}
#[test]
fn check_rejects_claimed_core_out_of_bounds() {
check_mutated_assignments(200, 100, 25, |m| {
m.core.0 += 100;
Some(false)
});
}
#[test]
fn check_rejects_in_backing_group() {
check_mutated_assignments(200, 100, 25, |m| {
m.group = m.own_group;
Some(false)
});
}
#[test]
fn check_rejects_nonexistent_key() {
check_mutated_assignments(200, 100, 25, |m| {
m.val_index += 200;
Some(false)
});
}
#[test]
fn check_rejects_delay_bad_vrf() {
check_mutated_assignments(40, 10, 8, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFDelay { .. } => {
m.cert.vrf = garbage_vrf();
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_modulo_bad_vrf() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFModulo { .. } => {
m.cert.vrf = garbage_vrf();
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_modulo_sample_out_of_bounds() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFModulo { sample } => {
m.config.relay_vrf_modulo_samples = sample;
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_delay_claimed_core_wrong() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFDelay { .. } => {
m.core = CoreIndex((m.core.0 + 1) % 100);
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_modulo_core_wrong() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFModulo { .. } => {
m.core = CoreIndex((m.core.0 + 1) % 100);
Some(false)
}
_ => None, // skip everything else.
}
});
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,412 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Entries pertaining to approval which need to be persisted.
//!
//! The actual persisting of data is handled by the `approval_db` module.
//! Within that context, things are plain-old-data. Within this module,
//! data and logic are intertwined.
use polkadot_node_primitives::approval::{DelayTranche, RelayVRFStory, AssignmentCert};
use polkadot_primitives::v1::{
ValidatorIndex, CandidateReceipt, SessionIndex, GroupIndex, CoreIndex,
Hash, CandidateHash,
};
use sp_consensus_slots::Slot;
use std::collections::BTreeMap;
use bitvec::{slice::BitSlice, vec::BitVec, order::Lsb0 as BitOrderLsb0};
use super::time::Tick;
use super::criteria::OurAssignment;
/// Metadata regarding a specific tranche of assignments for a specific candidate.
#[derive(Debug, Clone, PartialEq)]
pub struct TrancheEntry {
tranche: DelayTranche,
// Assigned validators, and the instant we received their assignment, rounded
// to the nearest tick.
assignments: Vec<(ValidatorIndex, Tick)>,
}
impl TrancheEntry {
/// Get the tranche of this entry.
pub fn tranche(&self) -> DelayTranche {
self.tranche
}
/// Get the assignments for this entry.
pub fn assignments(&self) -> &[(ValidatorIndex, Tick)] {
&self.assignments
}
}
impl From<crate::approval_db::v1::TrancheEntry> for TrancheEntry {
fn from(entry: crate::approval_db::v1::TrancheEntry) -> Self {
TrancheEntry {
tranche: entry.tranche,
assignments: entry.assignments.into_iter().map(|(v, t)| (v, t.into())).collect(),
}
}
}
impl From<TrancheEntry> for crate::approval_db::v1::TrancheEntry {
fn from(entry: TrancheEntry) -> Self {
Self {
tranche: entry.tranche,
assignments: entry.assignments.into_iter().map(|(v, t)| (v, t.into())).collect(),
}
}
}
/// Metadata regarding approval of a particular candidate within the context of some
/// particular block.
#[derive(Debug, Clone, PartialEq)]
pub struct ApprovalEntry {
tranches: Vec<TrancheEntry>,
backing_group: GroupIndex,
our_assignment: Option<OurAssignment>,
// `n_validators` bits.
assignments: BitVec<BitOrderLsb0, u8>,
approved: bool,
}
impl ApprovalEntry {
// Access our assignment for this approval entry.
pub fn our_assignment(&self) -> Option<&OurAssignment> {
self.our_assignment.as_ref()
}
// Note that our assignment is triggered. No-op if already triggered.
pub fn trigger_our_assignment(&mut self, tick_now: Tick)
-> Option<(AssignmentCert, ValidatorIndex)>
{
let our = self.our_assignment.as_mut().and_then(|a| {
if a.triggered() { return None }
a.mark_triggered();
Some(a.clone())
});
our.map(|a| {
self.import_assignment(a.tranche(), a.validator_index(), tick_now);
(a.cert().clone(), a.validator_index())
})
}
/// Whether a validator is already assigned.
pub fn is_assigned(&self, validator_index: ValidatorIndex) -> bool {
self.assignments.get(validator_index as usize).map(|b| *b).unwrap_or(false)
}
/// Import an assignment. No-op if already assigned on the same tranche.
pub fn import_assignment(
&mut self,
tranche: DelayTranche,
validator_index: ValidatorIndex,
tick_now: Tick,
) {
// linear search probably faster than binary. not many tranches typically.
let idx = match self.tranches.iter().position(|t| t.tranche >= tranche) {
Some(pos) => {
if self.tranches[pos].tranche > tranche {
self.tranches.insert(pos, TrancheEntry {
tranche: tranche,
assignments: Vec::new(),
});
}
pos
}
None => {
self.tranches.push(TrancheEntry {
tranche: tranche,
assignments: Vec::new(),
});
self.tranches.len() - 1
}
};
self.tranches[idx].assignments.push((validator_index, tick_now));
self.assignments.set(validator_index as _, true);
}
// Produce a bitvec indicating the assignments of all validators up to and
// including `tranche`.
pub fn assignments_up_to(&self, tranche: DelayTranche) -> BitVec<BitOrderLsb0, u8> {
self.tranches.iter()
.take_while(|e| e.tranche <= tranche)
.fold(bitvec::bitvec![BitOrderLsb0, u8; 0; self.assignments.len()], |mut a, e| {
for &(v, _) in &e.assignments {
a.set(v as _, true);
}
a
})
}
/// Whether the approval entry is approved
pub fn is_approved(&self) -> bool {
self.approved
}
/// Mark the approval entry as approved.
pub fn mark_approved(&mut self) {
self.approved = true;
}
/// Access the tranches.
pub fn tranches(&self) -> &[TrancheEntry] {
&self.tranches
}
/// Get the number of validators in this approval entry.
pub fn n_validators(&self) -> usize {
self.assignments.len()
}
/// Get the backing group index of the approval entry.
pub fn backing_group(&self) -> GroupIndex {
self.backing_group
}
/// For tests: set our assignment.
#[cfg(test)]
pub fn set_our_assignment(&mut self, our_assignment: OurAssignment) {
self.our_assignment = Some(our_assignment);
}
}
impl From<crate::approval_db::v1::ApprovalEntry> for ApprovalEntry {
fn from(entry: crate::approval_db::v1::ApprovalEntry) -> Self {
ApprovalEntry {
tranches: entry.tranches.into_iter().map(Into::into).collect(),
backing_group: entry.backing_group,
our_assignment: entry.our_assignment.map(Into::into),
assignments: entry.assignments,
approved: entry.approved,
}
}
}
impl From<ApprovalEntry> for crate::approval_db::v1::ApprovalEntry {
fn from(entry: ApprovalEntry) -> Self {
Self {
tranches: entry.tranches.into_iter().map(Into::into).collect(),
backing_group: entry.backing_group,
our_assignment: entry.our_assignment.map(Into::into),
assignments: entry.assignments,
approved: entry.approved,
}
}
}
/// Metadata regarding approval of a particular candidate.
#[derive(Debug, Clone, PartialEq)]
pub struct CandidateEntry {
candidate: CandidateReceipt,
session: SessionIndex,
// Assignments are based on blocks, so we need to track assignments separately
// based on the block we are looking at.
block_assignments: BTreeMap<Hash, ApprovalEntry>,
approvals: BitVec<BitOrderLsb0, u8>,
}
impl CandidateEntry {
/// Access the bit-vec of approvals.
pub fn approvals(&self) -> &BitSlice<BitOrderLsb0, u8> {
&self.approvals
}
/// Note that a given validator has approved. Return the previous approval state.
pub fn mark_approval(&mut self, validator: ValidatorIndex) -> bool {
let prev = self.approvals.get(validator as usize).map(|b| *b).unwrap_or(false);
self.approvals.set(validator as usize, true);
prev
}
/// Get the candidate receipt.
pub fn candidate_receipt(&self) -> &CandidateReceipt {
&self.candidate
}
/// Get the approval entry, mutably, for this candidate under a specific block.
pub fn approval_entry_mut(&mut self, block_hash: &Hash) -> Option<&mut ApprovalEntry> {
self.block_assignments.get_mut(block_hash)
}
/// Get the approval entry for this candidate under a specific block.
pub fn approval_entry(&self, block_hash: &Hash) -> Option<&ApprovalEntry> {
self.block_assignments.get(block_hash)
}
/// Iterate over approval entries.
pub fn iter_approval_entries(&self) -> impl IntoIterator<Item = (&Hash, &ApprovalEntry)> {
self.block_assignments.iter()
}
#[cfg(test)]
pub fn add_approval_entry(
&mut self,
block_hash: Hash,
approval_entry: ApprovalEntry,
) {
self.block_assignments.insert(block_hash, approval_entry);
}
}
impl From<crate::approval_db::v1::CandidateEntry> for CandidateEntry {
fn from(entry: crate::approval_db::v1::CandidateEntry) -> Self {
CandidateEntry {
candidate: entry.candidate,
session: entry.session,
block_assignments: entry.block_assignments.into_iter().map(|(h, ae)| (h, ae.into())).collect(),
approvals: entry.approvals,
}
}
}
impl From<CandidateEntry> for crate::approval_db::v1::CandidateEntry {
fn from(entry: CandidateEntry) -> Self {
Self {
candidate: entry.candidate,
session: entry.session,
block_assignments: entry.block_assignments.into_iter().map(|(h, ae)| (h, ae.into())).collect(),
approvals: entry.approvals,
}
}
}
/// Metadata regarding approval of a particular block, by way of approval of the
/// candidates contained within it.
#[derive(Debug, Clone, PartialEq)]
pub struct BlockEntry {
block_hash: Hash,
session: SessionIndex,
slot: Slot,
relay_vrf_story: RelayVRFStory,
// The candidates included as-of this block and the index of the core they are
// leaving. Sorted ascending by core index.
candidates: Vec<(CoreIndex, CandidateHash)>,
// A bitfield where the i'th bit corresponds to the i'th candidate in `candidates`.
// The i'th bit is `true` iff the candidate has been approved in the context of this
// block. The block can be considered approved if the bitfield has all bits set to `true`.
approved_bitfield: BitVec<BitOrderLsb0, u8>,
children: Vec<Hash>,
}
impl BlockEntry {
/// Mark a candidate as fully approved in the bitfield.
pub fn mark_approved_by_hash(&mut self, candidate_hash: &CandidateHash) {
if let Some(p) = self.candidates.iter().position(|(_, h)| h == candidate_hash) {
self.approved_bitfield.set(p, true);
}
}
/// Whether the block entry is fully approved.
pub fn is_fully_approved(&self) -> bool {
self.approved_bitfield.all()
}
#[cfg(test)]
pub fn block_hash(&self) -> Hash {
self.block_hash
}
#[cfg(test)]
pub fn is_candidate_approved(&self, candidate_hash: &CandidateHash) -> bool {
self.candidates.iter().position(|(_, h)| h == candidate_hash)
.and_then(|p| self.approved_bitfield.get(p).map(|b| *b))
.unwrap_or(false)
}
/// For tests: Add a candidate to the block entry. Returns the
/// index where the candidate was added.
///
/// Panics if the core is already used.
#[cfg(test)]
pub fn add_candidate(&mut self, core: CoreIndex, candidate_hash: CandidateHash) -> usize {
let pos = self.candidates
.binary_search_by_key(&core, |(c, _)| *c)
.unwrap_err();
self.candidates.insert(pos, (core, candidate_hash));
// bug in bitvec?
if pos < self.approved_bitfield.len() {
self.approved_bitfield.insert(pos, false);
} else {
self.approved_bitfield.push(false);
}
pos
}
/// Get the slot of the block.
pub fn slot(&self) -> Slot {
self.slot
}
/// Get the relay-vrf-story of the block.
pub fn relay_vrf_story(&self) -> RelayVRFStory {
self.relay_vrf_story.clone()
}
/// Get the session index of the block.
pub fn session(&self) -> SessionIndex {
self.session
}
/// Get the i'th candidate.
pub fn candidate(&self, i: usize) -> Option<&(CoreIndex, CandidateHash)> {
self.candidates.get(i)
}
/// Access the underlying candidates as a slice.
pub fn candidates(&self) -> &[(CoreIndex, CandidateHash)] {
&self.candidates
}
}
impl From<crate::approval_db::v1::BlockEntry> for BlockEntry {
fn from(entry: crate::approval_db::v1::BlockEntry) -> Self {
BlockEntry {
block_hash: entry.block_hash,
session: entry.session,
slot: entry.slot,
relay_vrf_story: RelayVRFStory(entry.relay_vrf_story),
candidates: entry.candidates,
approved_bitfield: entry.approved_bitfield,
children: entry.children,
}
}
}
impl From<BlockEntry> for crate::approval_db::v1::BlockEntry {
fn from(entry: BlockEntry) -> Self {
Self {
block_hash: entry.block_hash,
session: entry.session,
slot: entry.slot,
relay_vrf_story: entry.relay_vrf_story.0,
candidates: entry.candidates,
approved_bitfield: entry.approved_bitfield,
children: entry.children,
}
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,88 @@
// Copyright 2021 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Time utilities for approval voting.
use polkadot_node_primitives::approval::DelayTranche;
use sp_consensus_slots::Slot;
use futures::prelude::*;
use std::time::{Duration, SystemTime};
use std::pin::Pin;
const TICK_DURATION_MILLIS: u64 = 500;
/// A base unit of time, starting from the unix epoch, split into half-second intervals.
pub(crate) type Tick = u64;
/// A clock which allows querying of the current tick as well as
/// waiting for a tick to be reached.
pub(crate) trait Clock {
/// Yields the current tick.
fn tick_now(&self) -> Tick;
/// Yields a future which concludes when the given tick is reached.
fn wait(&self, tick: Tick) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>>;
}
/// Extension methods for clocks.
pub(crate) trait ClockExt {
fn tranche_now(&self, slot_duration_millis: u64, base_slot: Slot) -> DelayTranche;
}
impl<C: Clock + ?Sized> ClockExt for C {
fn tranche_now(&self, slot_duration_millis: u64, base_slot: Slot) -> DelayTranche {
self.tick_now()
.saturating_sub(slot_number_to_tick(slot_duration_millis, base_slot)) as u32
}
}
/// A clock which uses the actual underlying system clock.
pub(crate) struct SystemClock;
impl Clock for SystemClock {
/// Yields the current tick.
fn tick_now(&self) -> Tick {
match SystemTime::now().duration_since(SystemTime::UNIX_EPOCH) {
Err(_) => 0,
Ok(d) => d.as_millis() as u64 / TICK_DURATION_MILLIS,
}
}
/// Yields a future which concludes when the given tick is reached.
fn wait(&self, tick: Tick) -> Pin<Box<dyn Future<Output = ()> + Send>> {
let fut = async move {
let now = SystemTime::now();
let tick_onset = tick_to_time(tick);
if now < tick_onset {
if let Some(until) = tick_onset.duration_since(now).ok() {
futures_timer::Delay::new(until).await;
}
}
};
Box::pin(fut)
}
}
fn tick_to_time(tick: Tick) -> SystemTime {
SystemTime::UNIX_EPOCH + Duration::from_millis(TICK_DURATION_MILLIS * tick)
}
/// assumes `slot_duration_millis` evenly divided by tick duration.
pub(crate) fn slot_number_to_tick(slot_duration_millis: u64, slot: Slot) -> Tick {
let ticks_per_slot = slot_duration_millis / TICK_DURATION_MILLIS;
u64::from(slot) * ticks_per_slot
}
+2 -2
View File
@@ -641,7 +641,7 @@ async fn process_block_activated(
for event in candidate_events {
match event {
CandidateEvent::CandidateBacked(receipt, _head) => {
CandidateEvent::CandidateBacked(receipt, _head, _core_index, _group_index) => {
note_block_backed(
&subsystem.db,
&mut tx,
@@ -651,7 +651,7 @@ async fn process_block_activated(
receipt,
)?;
}
CandidateEvent::CandidateIncluded(receipt, _head) => {
CandidateEvent::CandidateIncluded(receipt, _head, _core_index, _group_index) => {
note_block_included(
&subsystem.db,
&mut tx,
+13 -3
View File
@@ -27,6 +27,7 @@ use futures::{
use polkadot_primitives::v1::{
AvailableData, BlockData, CandidateDescriptor, CandidateReceipt, HeadData,
PersistedValidationData, PoV, Id as ParaId, CandidateHash, Header, ValidatorId,
CoreIndex, GroupIndex,
};
use polkadot_node_subsystem_util::TimeoutExt;
use polkadot_subsystem::{
@@ -219,6 +220,15 @@ fn with_tx(db: &Arc<impl KeyValueDB>, f: impl FnOnce(&mut DBTransaction)) {
db.write(tx).unwrap();
}
fn candidate_included(receipt: CandidateReceipt) -> CandidateEvent {
CandidateEvent::CandidateIncluded(
receipt,
HeadData::default(),
CoreIndex::default(),
GroupIndex::default(),
)
}
#[test]
fn runtime_api_error_does_not_stop_the_subsystem() {
let store = Arc::new(kvdb_memorydb::create(columns::NUM_COLUMNS));
@@ -595,7 +605,7 @@ fn stored_data_kept_until_finalized() {
&mut virtual_overseer,
parent,
block_number,
vec![CandidateEvent::CandidateIncluded(candidate, HeadData::default())],
vec![candidate_included(candidate)],
(0..n_validators).map(|_| Sr25519Keyring::Alice.public().into()).collect(),
).await;
@@ -737,7 +747,7 @@ fn forkfullness_works() {
&mut virtual_overseer,
parent_1,
block_number_1,
vec![CandidateEvent::CandidateIncluded(candidate_1, HeadData::default())],
vec![candidate_included(candidate_1)],
validators.clone(),
).await;
@@ -745,7 +755,7 @@ fn forkfullness_works() {
&mut virtual_overseer,
parent_2,
block_number_2,
vec![CandidateEvent::CandidateIncluded(candidate_2, HeadData::default())],
vec![candidate_included(candidate_2)],
validators.clone(),
).await;
@@ -13,6 +13,7 @@ parity-util-mem = { version = "0.9.0", default-features = false }
sp-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-consensus-babe = { git = "https://github.com/paritytech/substrate", branch = "master" }
polkadot-primitives = { path = "../../../primitives" }
polkadot-subsystem = { package = "polkadot-node-subsystem", path = "../../subsystem" }
@@ -22,3 +23,4 @@ polkadot-node-subsystem-util = { path = "../../subsystem-util" }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
futures = { version = "0.3.12", features = ["thread-pool"] }
polkadot-node-subsystem-test-helpers = { path = "../../subsystem-test-helpers" }
polkadot-node-primitives = { path = "../../primitives" }
@@ -20,6 +20,7 @@ use polkadot_primitives::v1::{
PersistedValidationData, Id as ParaId, OccupiedCoreAssumption,
SessionIndex, SessionInfo, ValidationCode, ValidatorId, ValidatorIndex,
};
use sp_consensus_babe::Epoch;
use parity_util_mem::{MallocSizeOf, MallocSizeOfExt};
@@ -40,6 +41,7 @@ const CANDIDATE_EVENTS_CACHE_SIZE: usize = 64 * 1024;
const SESSION_INFO_CACHE_SIZE: usize = 64 * 1024;
const DMQ_CONTENTS_CACHE_SIZE: usize = 64 * 1024;
const INBOUND_HRMP_CHANNELS_CACHE_SIZE: usize = 64 * 1024;
const CURRENT_BABE_EPOCH_CACHE_SIZE: usize = 64 * 1024;
struct ResidentSizeOf<T>(T);
@@ -49,6 +51,14 @@ impl<T: MallocSizeOf> ResidentSize for ResidentSizeOf<T> {
}
}
struct DoesNotAllocate<T>(T);
impl<T> ResidentSize for DoesNotAllocate<T> {
fn resident_size(&self) -> usize {
std::mem::size_of::<Self>()
}
}
pub(crate) struct RequestResultCache {
validators: MemoryLruCache<Hash, ResidentSizeOf<Vec<ValidatorId>>>,
validator_groups: MemoryLruCache<Hash, ResidentSizeOf<(Vec<Vec<ValidatorIndex>>, GroupRotationInfo)>>,
@@ -63,6 +73,7 @@ pub(crate) struct RequestResultCache {
session_info: MemoryLruCache<(Hash, SessionIndex), ResidentSizeOf<Option<SessionInfo>>>,
dmq_contents: MemoryLruCache<(Hash, ParaId), ResidentSizeOf<Vec<InboundDownwardMessage<BlockNumber>>>>,
inbound_hrmp_channels_contents: MemoryLruCache<(Hash, ParaId), ResidentSizeOf<BTreeMap<ParaId, Vec<InboundHrmpMessage<BlockNumber>>>>>,
current_babe_epoch: MemoryLruCache<Hash, DoesNotAllocate<Epoch>>,
}
impl Default for RequestResultCache {
@@ -81,6 +92,7 @@ impl Default for RequestResultCache {
session_info: MemoryLruCache::new(SESSION_INFO_CACHE_SIZE),
dmq_contents: MemoryLruCache::new(DMQ_CONTENTS_CACHE_SIZE),
inbound_hrmp_channels_contents: MemoryLruCache::new(INBOUND_HRMP_CHANNELS_CACHE_SIZE),
current_babe_epoch: MemoryLruCache::new(CURRENT_BABE_EPOCH_CACHE_SIZE),
}
}
}
@@ -189,6 +201,14 @@ impl RequestResultCache {
pub(crate) fn cache_inbound_hrmp_channel_contents(&mut self, key: (Hash, ParaId), value: BTreeMap<ParaId, Vec<InboundHrmpMessage<BlockNumber>>>) {
self.inbound_hrmp_channels_contents.insert(key, ResidentSizeOf(value));
}
pub(crate) fn current_babe_epoch(&mut self, relay_parent: &Hash) -> Option<&Epoch> {
self.current_babe_epoch.get(relay_parent).map(|v| &v.0)
}
pub(crate) fn cache_current_babe_epoch(&mut self, relay_parent: Hash, epoch: Epoch) {
self.current_babe_epoch.insert(relay_parent, DoesNotAllocate(epoch));
}
}
pub(crate) enum RequestResult {
@@ -205,4 +225,5 @@ pub(crate) enum RequestResult {
SessionInfo(Hash, SessionIndex, Option<SessionInfo>),
DmqContents(Hash, ParaId, Vec<InboundDownwardMessage<BlockNumber>>),
InboundHrmpChannelsContents(Hash, ParaId, BTreeMap<ParaId, Vec<InboundHrmpMessage<BlockNumber>>>),
CurrentBabeEpoch(Hash, Epoch),
}
+78 -5
View File
@@ -35,6 +35,7 @@ use polkadot_primitives::v1::{Block, BlockId, Hash, ParachainHost};
use sp_api::ProvideRuntimeApi;
use sp_core::traits::SpawnNamed;
use sp_consensus_babe::BabeApi;
use futures::{prelude::*, stream::FuturesUnordered, channel::oneshot, select};
use std::{sync::Arc, collections::VecDeque, pin::Pin};
@@ -82,7 +83,7 @@ impl<Client> RuntimeApiSubsystem<Client> {
impl<Client, Context> Subsystem<Context> for RuntimeApiSubsystem<Client> where
Client: ProvideRuntimeApi<Block> + Send + 'static + Sync,
Client::Api: ParachainHost<Block>,
Client::Api: ParachainHost<Block> + BabeApi<Block>,
Context: SubsystemContext<Message = RuntimeApiMessage>
{
fn start(self, ctx: Context) -> SpawnedSubsystem {
@@ -95,7 +96,7 @@ impl<Client, Context> Subsystem<Context> for RuntimeApiSubsystem<Client> where
impl<Client> RuntimeApiSubsystem<Client> where
Client: ProvideRuntimeApi<Block> + Send + 'static + Sync,
Client::Api: ParachainHost<Block>,
Client::Api: ParachainHost<Block> + BabeApi<Block>,
{
fn store_cache(&mut self, result: RequestResult) {
use RequestResult::*;
@@ -127,6 +128,8 @@ impl<Client> RuntimeApiSubsystem<Client> where
self.requests_cache.cache_dmq_contents((relay_parent, para_id), messages),
InboundHrmpChannelsContents(relay_parent, para_id, contents) =>
self.requests_cache.cache_inbound_hrmp_channel_contents((relay_parent, para_id), contents),
CurrentBabeEpoch(relay_parent, epoch) =>
self.requests_cache.cache_current_babe_epoch(relay_parent, epoch),
}
}
@@ -189,7 +192,10 @@ impl<Client> RuntimeApiSubsystem<Client> where
.map(|sender| Request::DmqContents(id, sender)),
Request::InboundHrmpChannelsContents(id, sender) =>
query!(inbound_hrmp_channels_contents(id), sender)
.map(|sender| Request::InboundHrmpChannelsContents(id, sender))
.map(|sender| Request::InboundHrmpChannelsContents(id, sender)),
Request::CurrentBabeEpoch(sender) =>
query!(current_babe_epoch(), sender)
.map(|sender| Request::CurrentBabeEpoch(sender)),
}
}
@@ -257,7 +263,7 @@ async fn run<Client>(
mut subsystem: RuntimeApiSubsystem<Client>,
) -> SubsystemResult<()> where
Client: ProvideRuntimeApi<Block> + Send + Sync + 'static,
Client::Api: ParachainHost<Block>,
Client::Api: ParachainHost<Block> + BabeApi<Block>,
{
loop {
select! {
@@ -285,7 +291,7 @@ fn make_runtime_api_request<Client>(
) -> Option<RequestResult>
where
Client: ProvideRuntimeApi<Block>,
Client::Api: ParachainHost<Block>,
Client::Api: ParachainHost<Block> + BabeApi<Block>,
{
let _timer = metrics.time_make_runtime_api_request();
@@ -339,6 +345,7 @@ where
Request::SessionInfo(index, sender) => query!(SessionInfo, session_info(index), sender),
Request::DmqContents(id, sender) => query!(DmqContents, dmq_contents(id), sender),
Request::InboundHrmpChannelsContents(id, sender) => query!(InboundHrmpChannelsContents, inbound_hrmp_channels_contents(id), sender),
Request::CurrentBabeEpoch(sender) => query!(CurrentBabeEpoch, current_epoch(), sender),
}
}
@@ -415,6 +422,7 @@ mod tests {
use sp_core::testing::TaskExecutor;
use std::{collections::{HashMap, BTreeMap}, sync::{Arc, Mutex}};
use futures::channel::oneshot;
use polkadot_node_primitives::BabeEpoch;
#[derive(Default, Clone)]
struct MockRuntimeApi {
@@ -432,6 +440,7 @@ mod tests {
candidate_events: Vec<CandidateEvent>,
dmq_contents: HashMap<ParaId, Vec<InboundDownwardMessage>>,
hrmp_channels: HashMap<ParaId, BTreeMap<ParaId, Vec<InboundHrmpMessage>>>,
babe_epoch: Option<BabeEpoch>,
}
impl ProvideRuntimeApi<Block> for MockRuntimeApi {
@@ -541,6 +550,38 @@ mod tests {
self.hrmp_channels.get(&recipient).map(|q| q.clone()).unwrap_or_default()
}
}
impl BabeApi<Block> for MockRuntimeApi {
fn configuration(&self) -> sp_consensus_babe::BabeGenesisConfiguration {
unimplemented!()
}
fn current_epoch_start(&self) -> sp_consensus_babe::Slot {
self.babe_epoch.as_ref().unwrap().start_slot
}
fn current_epoch(&self) -> BabeEpoch {
self.babe_epoch.as_ref().unwrap().clone()
}
fn next_epoch(&self) -> BabeEpoch {
unimplemented!()
}
fn generate_key_ownership_proof(
_slot: sp_consensus_babe::Slot,
_authority_id: sp_consensus_babe::AuthorityId,
) -> Option<sp_consensus_babe::OpaqueKeyOwnershipProof> {
None
}
fn submit_report_equivocation_unsigned_extrinsic(
_equivocation_proof: sp_consensus_babe::EquivocationProof<polkadot_primitives::v1::Header>,
_key_owner_proof: sp_consensus_babe::OpaqueKeyOwnershipProof,
) -> Option<()> {
None
}
}
}
#[test]
@@ -1108,4 +1149,36 @@ mod tests {
futures::executor::block_on(future::join(subsystem_task, test_task));
}
#[test]
fn request_babe_epoch() {
let (ctx, mut ctx_handle) = test_helpers::make_subsystem_context(TaskExecutor::new());
let mut runtime_api = MockRuntimeApi::default();
let epoch = BabeEpoch {
epoch_index: 100,
start_slot: sp_consensus_babe::Slot::from(1000),
duration: 10,
authorities: Vec::new(),
randomness: [1u8; 32],
};
runtime_api.babe_epoch = Some(epoch.clone());
let runtime_api = Arc::new(runtime_api);
let relay_parent = [1; 32].into();
let spawner = sp_core::testing::TaskExecutor::new();
let subsystem = RuntimeApiSubsystem::new(runtime_api.clone(), Metrics(None), spawner);
let subsystem_task = run(ctx, subsystem).map(|x| x.unwrap());
let test_task = async move {
let (tx, rx) = oneshot::channel();
ctx_handle.send(FromOverseer::Communication {
msg: RuntimeApiMessage::Request(relay_parent, Request::CurrentBabeEpoch(tx))
}).await;
assert_eq!(rx.await.unwrap().unwrap(), epoch);
ctx_handle.send(FromOverseer::Signal(OverseerSignal::Conclude)).await;
};
futures::executor::block_on(future::join(subsystem_task, test_task));
}
}
@@ -350,6 +350,7 @@ impl State {
ctx.send_message(AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
assignment.clone(),
claimed_candidate_index,
tx,
))).await;
@@ -222,6 +222,7 @@ fn try_import_the_same_assignment() {
overseer_recv(overseer).await,
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
assignment,
0u32,
tx,
)) => {
assert_eq!(assignment, cert);
@@ -313,9 +314,11 @@ fn spam_attack_results_in_negative_reputation_change() {
overseer_recv(overseer).await,
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
assignment,
claimed_candidate_index,
tx,
)) => {
assert_eq!(assignment, assignments[i].0);
assert_eq!(claimed_candidate_index, assignments[i].1);
tx.send(AssignmentCheckResult::Accepted).unwrap();
}
);
@@ -477,9 +480,11 @@ fn import_approval_bad() {
overseer_recv(overseer).await,
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
assignment,
i,
tx,
)) => {
assert_eq!(assignment, cert);
assert_eq!(i, candidate_index);
tx.send(AssignmentCheckResult::Accepted).unwrap();
}
);
@@ -760,9 +765,11 @@ fn import_remotely_then_locally() {
overseer_recv(overseer).await,
AllMessages::ApprovalVoting(ApprovalVotingMessage::CheckAndImportAssignment(
assignment,
i,
tx,
)) => {
assert_eq!(assignment, cert);
assert_eq!(i, candidate_index);
tx.send(AssignmentCheckResult::Accepted).unwrap();
}
);
+1 -1
View File
@@ -1840,7 +1840,7 @@ where
let _ = self.approval_distribution_subsystem.send_message(msg).await;
},
AllMessages::ApprovalVoting(_msg) => {
// FIXME: https://github.com/paritytech/polkadot/issues/1975
// FIXME: https://github.com/paritytech/polkadot/issues/2321
},
}
+4 -1
View File
@@ -12,5 +12,8 @@ polkadot-statement-table = { path = "../../statement-table" }
parity-scale-codec = { version = "2.0.0", default-features = false, features = ["derive"] }
runtime_primitives = { package = "sp-runtime", git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-application-crypto = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-consensus-vrf = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-consensus-slots = { git = "https://github.com/paritytech/substrate", branch = "master", default-features = false }
sp-consensus-babe = { git = "https://github.com/paritytech/substrate", branch = "master" }
schnorrkel = "0.9.1"
thiserror = "1.0.22"
+103 -10
View File
@@ -16,29 +16,44 @@
//! Types relevant for approval.
pub use sp_consensus_vrf::schnorrkel::{VRFOutput, VRFProof};
pub use sp_consensus_slots::Slot;
pub use sp_consensus_vrf::schnorrkel::{VRFOutput, VRFProof, Randomness};
pub use sp_consensus_babe::Slot;
use polkadot_primitives::v1::{
CandidateHash, Hash, ValidatorIndex, Signed, ValidatorSignature, CoreIndex,
BlockNumber, CandidateIndex,
CandidateHash, Hash, ValidatorIndex, ValidatorSignature, CoreIndex,
Header, BlockNumber, CandidateIndex,
};
use parity_scale_codec::{Encode, Decode};
use sp_consensus_babe as babe_primitives;
use sp_application_crypto::Public;
/// Validators assigning to check a particular candidate are split up into tranches.
/// Earlier tranches of validators check first, with later tranches serving as backup.
pub type DelayTranche = u32;
/// A static context used to compute the Relay VRF story based on the
/// VRF output included in the header-chain.
pub const RELAY_VRF_STORY_CONTEXT: &[u8] = b"A&V RC-VRF";
/// A static context used for all relay-vrf-modulo VRFs.
pub const RELAY_VRF_MODULO_CONTEXT: &[u8] = b"A&V MOD";
/// A static context used for all relay-vrf-delay VRFs.
pub const RELAY_VRF_DELAY_CONTEXT: &[u8] = b"A&V TRANCHE";
/// A static context used for all relay-vrf-modulo VRFs.
pub const RELAY_VRF_DELAY_CONTEXT: &[u8] = b"A&V DELAY";
/// A static context used for transcripts indicating assigned availability core.
pub const ASSIGNED_CORE_CONTEXT: &[u8] = b"A&V ASSIGNED";
/// A static context associated with producing randomness for a core.
pub const CORE_RANDOMNESS_CONTEXT: &[u8] = b"A&V CORE";
/// A static context associated with producing randomness for a tranche.
pub const TRANCHE_RANDOMNESS_CONTEXT: &[u8] = b"A&V TRANCHE";
/// random bytes derived from the VRF submitted within the block by the
/// block author as a credential and used as input to approval assignment criteria.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub struct RelayVRF(pub [u8; 32]);
pub struct RelayVRFStory(pub [u8; 32]);
/// Different kinds of input data or criteria that can prove a validator's assignment
/// to check a particular parachain.
@@ -87,9 +102,6 @@ pub struct IndirectAssignmentCert {
#[derive(Debug, Clone, Encode, Decode)]
pub struct ApprovalVote(pub CandidateHash);
/// An approval vote signed by some validator.
pub type SignedApprovalVote = Signed<ApprovalVote>;
/// A signed approval vote which references the candidate indirectly via the block.
///
/// In practice, we have a look-up from block hash and candidate index to candidate hash,
@@ -121,3 +133,84 @@ pub struct BlockApprovalMeta {
/// The consensus slot of the block.
pub slot: Slot,
}
/// Errors that can occur during the approvals protocol.
#[derive(Debug, thiserror::Error)]
#[allow(missing_docs)]
pub enum ApprovalError {
#[error("Schnorrkel signature error")]
SchnorrkelSignature(schnorrkel::errors::SignatureError),
#[error("Authority index {0} out of bounds")]
AuthorityOutOfBounds(usize),
}
/// An unsafe VRF output. Provide BABE Epoch info to create a `RelayVRFStory`.
pub struct UnsafeVRFOutput {
vrf_output: VRFOutput,
slot: Slot,
authority_index: u32,
}
impl UnsafeVRFOutput {
/// Get the slot.
pub fn slot(&self) -> Slot {
self.slot
}
/// Compute the randomness associated with this VRF output.
pub fn compute_randomness(
self,
authorities: &[(babe_primitives::AuthorityId, babe_primitives::BabeAuthorityWeight)],
randomness: &babe_primitives::Randomness,
epoch_index: u64,
) -> Result<RelayVRFStory, ApprovalError> {
let author = match authorities.get(self.authority_index as usize) {
None => return Err(ApprovalError::AuthorityOutOfBounds(self.authority_index as _)),
Some(x) => &x.0,
};
let pubkey = schnorrkel::PublicKey::from_bytes(author.as_slice())
.map_err(ApprovalError::SchnorrkelSignature)?;
let transcript = babe_primitives::make_transcript(
randomness,
self.slot,
epoch_index,
);
let inout = self.vrf_output.0.attach_input_hash(&pubkey, transcript)
.map_err(ApprovalError::SchnorrkelSignature)?;
Ok(RelayVRFStory(inout.make_bytes(RELAY_VRF_STORY_CONTEXT)))
}
}
/// Extract the slot number and relay VRF from a header.
///
/// This fails if either there is no BABE `PreRuntime` digest or
/// the digest has type `SecondaryPlain`, which Substrate nodes do
/// not produce or accept anymore.
pub fn babe_unsafe_vrf_info(header: &Header) -> Option<UnsafeVRFOutput> {
use babe_primitives::digests::{CompatibleDigestItem, PreDigest};
for digest in &header.digest.logs {
if let Some(pre) = digest.as_babe_pre_digest() {
let slot = pre.slot();
let authority_index = pre.authority_index();
// exhaustive match to defend against upstream variant changes.
let vrf_output = match pre {
PreDigest::Primary(primary) => primary.vrf_output,
PreDigest::SecondaryVRF(secondary) => secondary.vrf_output,
PreDigest::SecondaryPlain(_) => return None,
};
return Some(UnsafeVRFOutput {
vrf_output,
slot,
authority_index,
});
}
}
None
}
+1
View File
@@ -32,6 +32,7 @@ use polkadot_primitives::v1::{
use std::pin::Pin;
pub use sp_core::traits::SpawnNamed;
pub use sp_consensus_babe::Epoch as BabeEpoch;
pub mod approval;
+11 -5
View File
@@ -38,6 +38,7 @@ use {
sp_keystore::SyncCryptoStorePtr,
sp_trie::PrefixedMemoryDB,
sc_client_api::ExecutorProvider,
babe_primitives::BabeApi,
};
#[cfg(feature = "real-overseer")]
use polkadot_network_bridge::RequestMultiplexer;
@@ -218,6 +219,7 @@ fn new_partial<RuntimeApi, Executor>(config: &mut Configuration, jaeger_agent: O
babe::BabeLink<Block>
),
grandpa::SharedVoterState,
u64, // slot-duration
)
>,
Error
@@ -265,8 +267,9 @@ fn new_partial<RuntimeApi, Executor>(config: &mut Configuration, jaeger_agent: O
let justification_import = grandpa_block_import.clone();
let babe_config = babe::Config::get_or_compute(&*client)?;
let (block_import, babe_link) = babe::block_import(
babe::Config::get_or_compute(&*client)?,
babe_config.clone(),
grandpa_block_import,
client.clone(),
)?;
@@ -294,8 +297,8 @@ fn new_partial<RuntimeApi, Executor>(config: &mut Configuration, jaeger_agent: O
let import_setup = (block_import.clone(), grandpa_link, babe_link.clone());
let rpc_setup = shared_voter_state.clone();
let babe_config = babe_link.config().clone();
let shared_epoch_changes = babe_link.epoch_changes().clone();
let slot_duration = babe_config.slot_duration();
let rpc_extensions_builder = {
let client = client.clone();
@@ -338,7 +341,7 @@ fn new_partial<RuntimeApi, Executor>(config: &mut Configuration, jaeger_agent: O
import_queue,
transaction_pool,
inherent_data_providers,
other: (rpc_extensions_builder, import_setup, rpc_setup)
other: (rpc_extensions_builder, import_setup, rpc_setup, slot_duration)
})
}
@@ -355,6 +358,7 @@ fn real_overseer<Spawner, RuntimeClient>(
spawner: Spawner,
_: IsCollator,
_: IsolationStrategy,
_: u64,
) -> Result<(Overseer<Spawner>, OverseerHandler), Error>
where
RuntimeClient: 'static + ProvideRuntimeApi<Block> + HeaderBackend<Block>,
@@ -382,10 +386,11 @@ fn real_overseer<Spawner, RuntimeClient>(
spawner: Spawner,
is_collator: IsCollator,
isolation_strategy: IsolationStrategy,
_slot_duration: u64, // TODO [now]: instantiate approval voting.
) -> Result<(Overseer<Spawner>, OverseerHandler), Error>
where
RuntimeClient: 'static + ProvideRuntimeApi<Block> + HeaderBackend<Block>,
RuntimeClient::Api: ParachainHost<Block>,
RuntimeClient::Api: ParachainHost<Block> + BabeApi<Block>,
Spawner: 'static + SpawnNamed + Clone + Unpin,
{
use polkadot_node_subsystem_util::metrics::Metrics;
@@ -571,7 +576,7 @@ pub fn new_full<RuntimeApi, Executor>(
import_queue,
transaction_pool,
inherent_data_providers,
other: (rpc_extensions_builder, import_setup, rpc_setup)
other: (rpc_extensions_builder, import_setup, rpc_setup, slot_duration)
} = new_partial::<RuntimeApi, Executor>(&mut config, jaeger_agent)?;
let prometheus_registry = config.prometheus_registry().cloned();
@@ -713,6 +718,7 @@ pub fn new_full<RuntimeApi, Executor>(
spawner,
is_collator,
isolation_strategy,
slot_duration,
)?;
let overseer_handler_clone = overseer_handler.clone();
+4
View File
@@ -31,6 +31,7 @@ use polkadot_node_network_protocol::{
use polkadot_node_primitives::{
CollationGenerationConfig, SignedFullStatement, ValidationResult,
approval::{BlockApprovalMeta, IndirectAssignmentCert, IndirectSignedApprovalVote},
BabeEpoch,
};
use polkadot_primitives::v1::{
AuthorityDiscoveryId, AvailableData, BackedCandidate, BlockNumber, SessionInfo,
@@ -474,6 +475,8 @@ pub enum RuntimeApiRequest {
ParaId,
RuntimeApiSender<BTreeMap<ParaId, Vec<InboundHrmpMessage<BlockNumber>>>>,
),
/// Get information about the BABE epoch the block was included in.
CurrentBabeEpoch(RuntimeApiSender<BabeEpoch>),
}
/// A message to the Runtime API subsystem.
@@ -631,6 +634,7 @@ pub enum ApprovalVotingMessage {
/// Should not be sent unless the block hash is known.
CheckAndImportAssignment(
IndirectAssignmentCert,
CandidateIndex,
oneshot::Sender<AssignmentCheckResult>,
),
/// Check if the approval vote is valid and can be accepted by our view of the
+19 -9
View File
@@ -177,14 +177,20 @@ pub const ASSIGNMENT_KEY_TYPE_ID: KeyTypeId = KeyTypeId(*b"asgn");
// The public key of a keypair used by a validator for determining assignments
/// to approve included parachain candidates.
mod assigment_app {
mod assignment_app {
use application_crypto::{app_crypto, sr25519};
app_crypto!(sr25519, super::ASSIGNMENT_KEY_TYPE_ID);
}
/// The public key of a keypair used by a validator for determining assignments
/// to approve included parachain candidates.
pub type AssignmentId = assigment_app::Public;
pub type AssignmentId = assignment_app::Public;
application_crypto::with_pair! {
/// The full keypair used by a validator for determining assignments to approve included
/// parachain candidates.
pub type AssignmentPair = assignment_app::Pair;
}
#[cfg(feature = "std")]
impl MallocSizeOf for AssignmentId {
@@ -544,8 +550,8 @@ pub fn check_candidate_backing<H: AsRef<[u8]> + Clone + Encode>(
}
/// The unique (during session) index of a core.
#[derive(Encode, Decode, Default, PartialOrd, Ord, Eq, PartialEq, Clone, Copy, Hash)]
#[cfg_attr(feature = "std", derive(Debug))]
#[derive(Encode, Decode, Default, PartialOrd, Ord, Eq, PartialEq, Clone, Copy)]
#[cfg_attr(feature = "std", derive(Debug, Hash, MallocSizeOf))]
pub struct CoreIndex(pub u32);
impl From<u32> for CoreIndex {
@@ -555,8 +561,8 @@ impl From<u32> for CoreIndex {
}
/// The unique (during session) index of a validator group.
#[derive(Encode, Decode, Default, Clone, Copy, Debug)]
#[cfg_attr(feature = "std", derive(Eq, Hash, PartialEq, MallocSizeOf))]
#[derive(Encode, Decode, Default, Clone, Copy, Debug, PartialEq, Eq)]
#[cfg_attr(feature = "std", derive(Hash, MallocSizeOf))]
pub struct GroupIndex(pub u32);
impl From<u32> for GroupIndex {
@@ -752,14 +758,18 @@ pub enum OccupiedCoreAssumption {
#[cfg_attr(feature = "std", derive(PartialEq, Debug, MallocSizeOf))]
pub enum CandidateEvent<H = Hash> {
/// This candidate receipt was backed in the most recent block.
/// This includes the core index the candidate is now occupying.
#[codec(index = 0)]
CandidateBacked(CandidateReceipt<H>, HeadData),
CandidateBacked(CandidateReceipt<H>, HeadData, CoreIndex, GroupIndex),
/// This candidate receipt was included and became a parablock at the most recent block.
/// This includes the core index the candidate was occupying as well as the group responsible
/// for backing the candidate.
#[codec(index = 1)]
CandidateIncluded(CandidateReceipt<H>, HeadData),
CandidateIncluded(CandidateReceipt<H>, HeadData, CoreIndex, GroupIndex),
/// This candidate receipt was not made available in time and timed out.
/// This includes the core index the candidate was occupying.
#[codec(index = 2)]
CandidateTimedOut(CandidateReceipt<H>, HeadData),
CandidateTimedOut(CandidateReceipt<H>, HeadData, CoreIndex),
}
/// Information about validator sets of a session.
@@ -238,7 +238,7 @@ On receiving an `ApprovedAncestor(Hash, BlockNumber, response_channel)`:
* Checks every `ApprovalEntry` that is not yet `approved` for whether it is now approved.
* For each `ApprovalEntry` in the `CandidateEntry` that is not `approved` and passes the `filter`
* Load the block entry for the `ApprovalEntry`.
* If so, [determine the tranches to inspect](#determine-required-tranches) of the candidate,
* If so, [determine the tranches to inspect](#determine-required-tranches) of the candidate,
* If [the candidate is approved under the block](#check-approval), set the corresponding bit in the `block_entry.approved_bitfield`.
* Otherwise, [schedule a wakeup of the candidate](#schedule-wakeup)
@@ -255,7 +255,7 @@ On receiving an `ApprovedAncestor(Hash, BlockNumber, response_channel)`:
* If we should trigger our assignment
* Import the assignment to the `ApprovalEntry`
* Broadcast on network with an `ApprovalDistributionMessage::DistributeAssignment`.
* [Launch approval work](#launch-approval-work) for the candidate.
* [Launch approval work](#launch-approval-work) for the candidate.
* [Schedule a new wakeup](#schedule-wakeup) of the candidate.
#### Schedule Wakeup
+32 -9
View File
@@ -51,8 +51,6 @@ pub struct AvailabilityBitfieldRecord<N> {
}
/// A backed candidate pending availability.
// TODO: split this type and change this to hold a plain `CandidateReceipt`.
// https://github.com/paritytech/polkadot/issues/1357
#[derive(Encode, Decode, PartialEq)]
#[cfg_attr(test, derive(Debug))]
pub struct CandidatePendingAvailability<H, N> {
@@ -70,6 +68,8 @@ pub struct CandidatePendingAvailability<H, N> {
relay_parent_number: N,
/// The block number of the relay-chain block this was backed in.
backed_in_number: N,
/// The group index backing this block.
backing_group: GroupIndex,
}
impl<H, N> CandidatePendingAvailability<H, N> {
@@ -197,11 +197,11 @@ decl_error! {
decl_event! {
pub enum Event<T> where <T as frame_system::Config>::Hash {
/// A candidate was backed. [candidate, head_data]
CandidateBacked(CandidateReceipt<Hash>, HeadData),
CandidateBacked(CandidateReceipt<Hash>, HeadData, CoreIndex, GroupIndex),
/// A candidate was included. [candidate, head_data]
CandidateIncluded(CandidateReceipt<Hash>, HeadData),
CandidateIncluded(CandidateReceipt<Hash>, HeadData, CoreIndex, GroupIndex),
/// A candidate timed out. [candidate, head_data]
CandidateTimedOut(CandidateReceipt<Hash>, HeadData),
CandidateTimedOut(CandidateReceipt<Hash>, HeadData, CoreIndex),
}
}
@@ -368,6 +368,8 @@ impl<T: Config> Module<T> {
receipt,
pending_availability.backers,
pending_availability.availability_votes,
pending_availability.core,
pending_availability.backing_group,
);
freed_cores.push(pending_availability.core);
@@ -555,7 +557,7 @@ impl<T: Config> Module<T> {
}
}
core_indices_and_backers.push((assignment.core, backers));
core_indices_and_backers.push((assignment.core, backers, assignment.group_idx));
continue 'a;
}
}
@@ -578,8 +580,8 @@ impl<T: Config> Module<T> {
};
// one more sweep for actually writing to storage.
let core_indices = core_indices_and_backers.iter().map(|&(ref c, _)| c.clone()).collect();
for (candidate, (core, backers)) in candidates.into_iter().zip(core_indices_and_backers) {
let core_indices = core_indices_and_backers.iter().map(|&(ref c, _, _)| c.clone()).collect();
for (candidate, (core, backers, group)) in candidates.into_iter().zip(core_indices_and_backers) {
let para_id = candidate.descriptor().para_id;
// initialize all availability votes to 0.
@@ -589,6 +591,8 @@ impl<T: Config> Module<T> {
Self::deposit_event(Event::<T>::CandidateBacked(
candidate.candidate.to_plain(),
candidate.candidate.commitments.head_data.clone(),
core,
group,
));
let candidate_hash = candidate.candidate.hash();
@@ -606,6 +610,7 @@ impl<T: Config> Module<T> {
relay_parent_number,
backers,
backed_in_number: check_cx.now,
backing_group: group,
});
<PendingAvailabilityCommitments>::insert(&para_id, commitments);
}
@@ -651,6 +656,8 @@ impl<T: Config> Module<T> {
receipt: CommittedCandidateReceipt<T::Hash>,
backers: BitVec<BitOrderLsb0, u8>,
availability_votes: BitVec<BitOrderLsb0, u8>,
core_index: CoreIndex,
backing_group: GroupIndex,
) -> Weight {
let plain = receipt.to_plain();
let commitments = receipt.commitments;
@@ -695,7 +702,7 @@ impl<T: Config> Module<T> {
);
Self::deposit_event(
Event::<T>::CandidateIncluded(plain, commitments.head_data.clone())
Event::<T>::CandidateIncluded(plain, commitments.head_data.clone(), core_index, backing_group)
);
weight + <paras::Module<T>>::note_new_head(
@@ -736,6 +743,7 @@ impl<T: Config> Module<T> {
Self::deposit_event(Event::<T>::CandidateTimedOut(
candidate,
commitments.head_data,
pending.core,
));
}
}
@@ -764,6 +772,8 @@ impl<T: Config> Module<T> {
candidate,
pending.backers,
pending.availability_votes,
pending.core,
pending.backing_group,
);
}
}
@@ -1154,6 +1164,7 @@ mod tests {
relay_parent_number: 0,
backed_in_number: 0,
backers: default_backing_bitfield(),
backing_group: GroupIndex::from(0),
});
PendingAvailabilityCommitments::insert(chain_a, default_candidate.commitments.clone());
@@ -1165,6 +1176,7 @@ mod tests {
relay_parent_number: 0,
backed_in_number: 0,
backers: default_backing_bitfield(),
backing_group: GroupIndex::from(1),
});
PendingAvailabilityCommitments::insert(chain_b, default_candidate.commitments);
@@ -1330,6 +1342,7 @@ mod tests {
relay_parent_number: 0,
backed_in_number: 0,
backers: default_backing_bitfield(),
backing_group: GroupIndex::from(0),
});
PendingAvailabilityCommitments::insert(chain_a, default_candidate.commitments);
@@ -1366,6 +1379,7 @@ mod tests {
relay_parent_number: 0,
backed_in_number: 0,
backers: default_backing_bitfield(),
backing_group: GroupIndex::from(0),
});
*bare_bitfield.0.get_mut(0).unwrap() = true;
@@ -1439,6 +1453,7 @@ mod tests {
relay_parent_number: 0,
backed_in_number: 0,
backers: backing_bitfield(&[3, 4]),
backing_group: GroupIndex::from(0),
});
PendingAvailabilityCommitments::insert(chain_a, candidate_a.commitments);
@@ -1456,6 +1471,7 @@ mod tests {
relay_parent_number: 0,
backed_in_number: 0,
backers: backing_bitfield(&[0, 2]),
backing_group: GroupIndex::from(1),
});
PendingAvailabilityCommitments::insert(chain_b, candidate_b.commitments);
@@ -1893,6 +1909,7 @@ mod tests {
relay_parent_number: 3,
backed_in_number: 4,
backers: default_backing_bitfield(),
backing_group: GroupIndex::from(0),
});
<PendingAvailabilityCommitments>::insert(&chain_a, candidate.commitments);
@@ -2187,6 +2204,7 @@ mod tests {
relay_parent_number: System::block_number() - 1,
backed_in_number: System::block_number(),
backers: backing_bitfield(&[0, 1]),
backing_group: GroupIndex::from(0),
})
);
assert_eq!(
@@ -2204,6 +2222,7 @@ mod tests {
relay_parent_number: System::block_number() - 1,
backed_in_number: System::block_number(),
backers: backing_bitfield(&[2, 3]),
backing_group: GroupIndex::from(1),
})
);
assert_eq!(
@@ -2221,6 +2240,7 @@ mod tests {
relay_parent_number: System::block_number() - 1,
backed_in_number: System::block_number(),
backers: backing_bitfield(&[4]),
backing_group: GroupIndex::from(2),
})
);
assert_eq!(
@@ -2318,6 +2338,7 @@ mod tests {
relay_parent_number: System::block_number() - 1,
backed_in_number: System::block_number(),
backers: backing_bitfield(&[0, 1, 2]),
backing_group: GroupIndex::from(0),
})
);
assert_eq!(
@@ -2394,6 +2415,7 @@ mod tests {
relay_parent_number: 5,
backed_in_number: 6,
backers: default_backing_bitfield(),
backing_group: GroupIndex::from(0),
});
<PendingAvailabilityCommitments>::insert(&chain_a, candidate.commitments.clone());
@@ -2405,6 +2427,7 @@ mod tests {
relay_parent_number: 6,
backed_in_number: 7,
backers: default_backing_bitfield(),
backing_group: GroupIndex::from(1),
});
<PendingAvailabilityCommitments>::insert(&chain_b, candidate.commitments);
@@ -271,9 +271,12 @@ where
<frame_system::Module<T>>::events().into_iter()
.filter_map(|record| extract_event(record.event))
.map(|event| match event {
RawEvent::<T>::CandidateBacked(c, h) => CandidateEvent::CandidateBacked(c, h),
RawEvent::<T>::CandidateIncluded(c, h) => CandidateEvent::CandidateIncluded(c, h),
RawEvent::<T>::CandidateTimedOut(c, h) => CandidateEvent::CandidateTimedOut(c, h),
RawEvent::<T>::CandidateBacked(c, h, core, group)
=> CandidateEvent::CandidateBacked(c, h, core, group),
RawEvent::<T>::CandidateIncluded(c, h, core, group)
=> CandidateEvent::CandidateIncluded(c, h, core, group),
RawEvent::<T>::CandidateTimedOut(c, h, core)
=> CandidateEvent::CandidateTimedOut(c, h, core),
})
.collect()
}