Implement Approval Voting Subsystem (#2112)

* skeleton

* skeleton aux-schema module

* start approval types

* start aux schema with aux store

* doc

* finish basic types

* start approval types

* doc

* finish basic types

* write out schema types

* add debug and codec impls to approval types

* add debug and codec impls to approval types

also add some key computation

* add debug and codec impls to approval types

* getters for block and candidate entries

* grumbles

* remove unused AssignmentId

* load_decode utility

* implement DB clearing

* function for adding new block entry to aux store

* start `canonicalize` implementation

* more skeleton

* finish implementing canonicalize

* tag TODO

* implement a test AuxStore

* add allow(unused)

* basic loading and deleting test

* block_entry test function

* add a test for `add_block_entry`

* ensure range is exclusive at end

* test clear()

* test that add_block sets children

* add a test for canonicalize

* extract Pre-digest from header

* utilities for extracting RelayVRFStory from the header-chain

* add approval voting message types

* approval distribution message type

* subsystem skeleton

* state struct

* add futures-timer

* prepare service for babe slot duration

* more skeleton

* better integrate AuxStore

* RelayVRF -> RelayVRFStory

* canonicalize

* implement some tick functionality

* guide: tweaks

* check_approval

* more tweaks and helpers

* guide: add core index to candidate event

* primitives: add core index to candidate event

* runtime: add core index to candidate events

* head handling (session window)

* implement `determine_new_blocks`

* add TODO

* change error type on functions

* compute RelayVRFModulo assignments

* compute RelayVRFDelay assignments

* fix delay tranche calc

* assignment checking

* pluralize

* some dummy code for fetching assignments

* guide: add babe epoch runtime API

* implement a current_epoch() runtime API

* compute assignments

* candidate events get backing group

* import blocks and assignments into DB

* push block approval meta

* add message types, no overseer integration yet

* notify approval distribution of new blocks

* refactor import into separate functions

* impl tranches_to_approve

* guide: improve function signatures

* guide: remove Tick from ApprovalEntry

* trigger and broadcast assignment

* most of approval launching

* remove byteorder crate

* load blocks back to finality, except on startup

* check unchecked assignments

* add claimed core to approval voting message

* fix checks

* assign only to backing group

* remove import_checked_assignment from guide

* newline

* import assignments

* abstract out a bit

* check and import approvals

* check full approvals from assignment import too

* comment

* create a Transaction utility

* must_use

* use transaction in `check_full_approvals`

* wire up wakeups

* add Ord to CandidateHash

* wakeup refactoring

* return candidate info from add_block_entry

* schedule wakeups

* background task: do candidate validation

* forward candidate validation requests

* issue approval votes when requested

* clean up a couple TODOs

* fix up session caching

* clean up last unimplemented!() items

* fix remaining warnings

* remove TODO

* implement handle_approved_ancestor

* update Cargo.lock

* fix runtime API tests

* guide: cleanup assignment checking

* use claimed candidate index instead of core

* extract time to a trait

* tests module

* write a mock clock for testing

* allow swapping out the clock

* make abstract over assignment criteria

* add some skeleton tests and simplify params

* fix backing group check

* do backing group check inside check_assignment_cert

* write some empty test functions to implement

* add a test for non-backing

* test that produced checks pass

* some empty test ideas

* runtime/inclusion: remove outdated TODO

* fix compilation

* av-store: fix tests

* dummy cert

* criteria tests

* move `TestStore` to main tests file

* fix unused warning

* test harness beginnings

* resolve slots renaming fallout

* more compilation fixes

* wip: extract pure data into a separate module

* wip: extract pure data into a separate module

* move types completely to v1

* add persisted_entries

* add conversion trait impls

* clean up some warnings

* extract import logic to own module

* schedule wakeups

* experiment with Actions

* uncomment approval-checking

* separate module for approval checking utilities

* port more code to use actions

* get approval pipeline using actions

* all logic is uncommented

* main loop processes actions

* all loop logic uncommented

* separate function for handling actions

* remove last unimplemented item

* clean up warnings

* State gives read-only access to underlying DB

* tests for approval checking

* tests for approval criteria

* skeleton test module for import

* list of import tests to do

* some test glue code

* test reject bad assignment

* test slot too far in future

* test reject assignment with unknown candidate

* remove loads_blocks tests

* determine_new_blocks back to finalized & harness

* more coverage for determining new blocks

* make `imported_block_info` have less reliance on State

* candidate_info tests

* tests for session caching

* remove println

* extricate DB and main TestStores

* rewrite approval checking logic to counteract early delays

* move state out of function

* update approval-checking tests

* tweak wakeups & scheduling logic

* rename check_full_approvals

* test that assignment import updates candidate

* some approval import tests

* some tests for check_and_apply_approval

* add 'full' qualifier to avoid confusion

* extract should-trigger logic to separate function

* some tests for all triggering

* tests for when we trigger assignments

* test wakeups

* add block utilities for testing

* some more tests for approval updates

* approved_ancestor tests

* new action type for launch approval

* process-wakeup tests

* clean up some warnings

* fix in_future test

* approval checking tests

* tighten up too-far-in-future

* special-case genesis when caching sessions

* fix bitfield len

Co-authored-by: Andronik Ordian <write@reusable.software>
This commit is contained in:
Robert Habermeier
2021-02-11 10:21:47 -06:00
committed by GitHub
parent 09eadfc979
commit e48c687504
29 changed files with 7505 additions and 132 deletions
@@ -0,0 +1,879 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Utilities for checking whether a candidate has been approved under a given block.
use polkadot_node_primitives::approval::DelayTranche;
use bitvec::slice::BitSlice;
use bitvec::order::Lsb0 as BitOrderLsb0;
use crate::persisted_entries::{ApprovalEntry, CandidateEntry};
use crate::time::Tick;
/// The required tranches of assignments needed to determine whether a candidate is approved.
#[derive(Debug, PartialEq, Clone)]
pub enum RequiredTranches {
/// All validators appear to be required, based on tranches already taken and remaining
/// no-shows.
All,
/// More tranches required - We're awaiting more assignments.
Pending {
/// The highest considered delay tranche when counting assignments.
considered: DelayTranche,
/// The tick at which the next no-show, of the assignments counted, would occur.
next_no_show: Option<Tick>,
/// The highest tranche to consider when looking to broadcast own assignment.
/// This should be considered along with the clock drift to avoid broadcasting
/// assignments that are before the local time.
maximum_broadcast: DelayTranche,
/// The clock drift, in ticks, to apply to the local clock when determining whether
/// to broadcast an assignment or when to schedule a wakeup. The local clock should be treated
/// as though it is `clock_drift` ticks earlier.
clock_drift: Tick,
},
/// An exact number of required tranches and a number of no-shows. This indicates that
/// at least the amount of `needed_approvals` are assigned and additionally all no-shows
/// are covered.
Exact {
/// The tranche to inspect up to.
needed: DelayTranche,
/// The amount of missing votes that should be tolerated.
tolerated_missing: usize,
/// When the next no-show would be, if any. This is used to schedule the next wakeup in the
/// event that there are some assignments that don't have corresponding approval votes. If this
/// is `None`, all assignments have approvals.
next_no_show: Option<Tick>,
}
}
/// Check the approval of a candidate.
pub fn check_approval(
candidate: &CandidateEntry,
approval: &ApprovalEntry,
required: RequiredTranches,
) -> bool {
match required {
RequiredTranches::Pending { .. } => false,
RequiredTranches::All => {
let approvals = candidate.approvals();
3 * approvals.count_ones() > 2 * approvals.len()
}
RequiredTranches::Exact { needed, tolerated_missing, .. } => {
// whether all assigned validators up to `needed` less no_shows have approved.
// e.g. if we had 5 tranches and 1 no-show, we would accept all validators in
// tranches 0..=5 except for 1 approving. In that example, we also accept all
// validators in tranches 0..=5 approving, but that would indicate that the
// RequiredTranches value was incorrectly constructed, so it is not realistic.
// If there are more missing approvals than there are no-shows, that indicates
// that there are some assignments which are not yet no-shows, but may become
// no-shows.
let mut assigned_mask = approval.assignments_up_to(needed);
let approvals = candidate.approvals();
let n_assigned = assigned_mask.count_ones();
// Filter the amount of assigned validators by those which have approved.
assigned_mask &= approvals.iter().by_val();
let n_approved = assigned_mask.count_ones();
// note: the process of computing `required` only chooses `exact` if
// that will surpass a minimum amount of checks.
// shouldn't typically go above, since all no-shows are supposed to be covered.
n_approved + tolerated_missing >= n_assigned
}
}
}
// Determining the amount of tranches required for approval or which assignments are pending
// involves moving through a series of states while looping over the tranches
//
// that we are aware of. First, we perform an initial count of the number of assignments
// until we reach the number of needed assignments for approval. As we progress, we count the
// number of no-shows in each tranche.
//
// Then, if there are any no-shows, we proceed into a series of subsequent states for covering
// no-shows.
//
// We cover each no-show by a non-empty tranche, keeping track of the amount of further
// no-shows encountered along the way. Once all of the no-shows we were previously aware
// of are covered, we then progress to cover the no-shows we encountered while covering those,
// and so on.
#[derive(Debug)]
struct State {
/// The total number of assignments obtained.
assignments: usize,
/// The depth of no-shows we are currently covering.
depth: usize,
/// The amount of no-shows that have been covered at the previous or current depths.
covered: usize,
/// The amount of assignments that we are attempting to cover at this depth.
///
/// At depth 0, these are the initial needed approvals, and at other depths these
/// are no-shows.
covering: usize,
/// The number of uncovered no-shows encountered at this depth. These will be the
/// `covering` of the next depth.
uncovered: usize,
/// The next tick at which a no-show would occur, if any.
next_no_show: Option<Tick>,
}
impl State {
fn output(
&self,
tranche: DelayTranche,
needed_approvals: usize,
n_validators: usize,
no_show_duration: Tick,
) -> RequiredTranches {
let covering = if self.depth == 0 { 0 } else { self.covering };
if self.depth != 0 && self.assignments + covering + self.uncovered >= n_validators {
return RequiredTranches::All;
}
// If we have enough assignments and all no-shows are covered, we have reached the number
// of tranches that we need to have.
if self.assignments >= needed_approvals && (covering + self.uncovered) == 0 {
return RequiredTranches::Exact {
needed: tranche,
tolerated_missing: self.covered,
next_no_show: self.next_no_show,
};
}
// We're pending more assignments and should look at more tranches.
let clock_drift = self.clock_drift(no_show_duration);
if self.depth == 0 {
RequiredTranches::Pending {
considered: tranche,
next_no_show: self.next_no_show,
// during the initial assignment-gathering phase, we want to accept assignments
// from any tranche. Note that honest validators will still not broadcast their
// assignment until it is time to do so, regardless of this value.
maximum_broadcast: DelayTranche::max_value(),
clock_drift,
}
} else {
RequiredTranches::Pending {
considered: tranche,
next_no_show: self.next_no_show,
maximum_broadcast: tranche + (covering + self.uncovered) as DelayTranche,
clock_drift,
}
}
}
fn clock_drift(&self, no_show_duration: Tick) -> Tick {
self.depth as Tick * no_show_duration
}
fn advance(
&self,
new_assignments: usize,
new_no_shows: usize,
next_no_show: Option<Tick>,
) -> State {
let new_covered = if self.depth == 0 {
new_assignments
} else {
// When covering no-shows, we treat each non-empty tranche as covering 1 assignment,
// regardless of how many assignments are within the tranche.
new_assignments.min(1)
};
let assignments = self.assignments + new_assignments;
let covering = self.covering.saturating_sub(new_covered);
let covered = if self.depth == 0 {
// If we're at depth 0, we're not actually covering no-shows,
// so we don't need to count them as such.
0
} else {
self.covered + new_covered
};
let uncovered = self.uncovered + new_no_shows;
let next_no_show = super::min_prefer_some(
self.next_no_show,
next_no_show,
);
let (depth, covering, uncovered) = if covering == 0 {
if uncovered == 0 {
(self.depth, 0, uncovered)
} else {
(self.depth + 1, uncovered, 0)
}
} else {
(self.depth, covering, uncovered)
};
State { assignments, depth, covered, covering, uncovered, next_no_show }
}
}
/// Determine the amount of tranches of assignments needed to determine approval of a candidate.
pub fn tranches_to_approve(
approval_entry: &ApprovalEntry,
approvals: &BitSlice<BitOrderLsb0, u8>,
tranche_now: DelayTranche,
block_tick: Tick,
no_show_duration: Tick,
needed_approvals: usize,
) -> RequiredTranches {
let tick_now = tranche_now as Tick + block_tick;
let n_validators = approval_entry.n_validators();
let initial_state = State {
assignments: 0,
depth: 0,
covered: 0,
covering: needed_approvals,
uncovered: 0,
next_no_show: None,
};
// The `ApprovalEntry` doesn't have any data for empty tranches. We still want to iterate over
// these empty tranches, so we create an iterator to fill the gaps.
//
// This iterator has an infinitely long amount of non-empty tranches appended to the end.
let tranches_with_gaps_filled = {
let mut gap_end = 0;
let approval_entries_filled = approval_entry.tranches()
.iter()
.flat_map(move |tranche_entry| {
let tranche = tranche_entry.tranche();
let assignments = tranche_entry.assignments();
let gap_start = gap_end + 1;
gap_end = tranche;
(gap_start..tranche).map(|i| (i, &[] as &[_]))
.chain(std::iter::once((tranche, assignments)))
});
let pre_end = approval_entry.tranches().first().map(|t| t.tranche());
let post_start = approval_entry.tranches().last().map_or(0, |t| t.tranche() + 1);
let pre = pre_end.into_iter()
.flat_map(|pre_end| (0..pre_end).map(|i| (i, &[] as &[_])));
let post = (post_start..).map(|i| (i, &[] as &[_]));
pre.chain(approval_entries_filled).chain(post)
};
tranches_with_gaps_filled
.scan(Some(initial_state), |state, (tranche, assignments)| {
// The `Option` here is used for early exit.
let s = match state.take() {
None => return None,
Some(s) => s,
};
let clock_drift = s.clock_drift(no_show_duration);
let drifted_tick_now = tick_now.saturating_sub(clock_drift);
let drifted_tranche_now = drifted_tick_now.saturating_sub(block_tick) as DelayTranche;
// Break the loop once we've taken enough tranches.
// Note that we always take tranche 0 as `drifted_tranche_now` cannot be less than 0.
if tranche > drifted_tranche_now {
return None;
}
let n_assignments = assignments.len();
// count no-shows. An assignment is a no-show if there is no corresponding approval vote
// after a fixed duration.
//
// While we count the no-shows, we also determine the next possible no-show we might
// see within this tranche.
let mut next_no_show = None;
let no_shows = {
let next_no_show = &mut next_no_show;
assignments.iter()
.map(|(v_index, tick)| (v_index, tick.saturating_sub(clock_drift) + no_show_duration))
.filter(|&(v_index, no_show_at)| {
let has_approved = approvals.get(*v_index as usize).map(|b| *b).unwrap_or(false);
let is_no_show = !has_approved && no_show_at <= drifted_tick_now;
if !is_no_show && !has_approved {
*next_no_show = super::min_prefer_some(
*next_no_show,
Some(no_show_at + clock_drift),
);
}
is_no_show
}).count()
};
let s = s.advance(n_assignments, no_shows, next_no_show);
let output = s.output(tranche, needed_approvals, n_validators, no_show_duration);
*state = match output {
RequiredTranches::Exact { .. } | RequiredTranches::All => {
// Wipe the state clean so the next iteration of this closure will terminate
// the iterator. This guarantees that we can call `last` further down to see
// either a `Finished` or `Pending` result
None
}
RequiredTranches::Pending { .. } => {
// Pending results are only interesting when they are the last result of the iterator
// i.e. we never achieve a satisfactory level of assignment.
Some(s)
}
};
Some(output)
})
.last()
.expect("the underlying iterator is infinite, starts at 0, and never exits early before tranche 1; qed")
}
#[cfg(test)]
mod tests {
use super::*;
use polkadot_primitives::v1::GroupIndex;
use bitvec::bitvec;
use bitvec::order::Lsb0 as BitOrderLsb0;
use crate::approval_db;
#[test]
fn pending_is_not_approved() {
let candidate = approval_db::v1::CandidateEntry {
candidate: Default::default(),
session: 0,
block_assignments: Default::default(),
approvals: Default::default(),
}.into();
let approval_entry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: Default::default(),
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
assert!(!check_approval(
&candidate,
&approval_entry,
RequiredTranches::Pending {
considered: 0,
next_no_show: None,
maximum_broadcast: 0,
clock_drift: 0,
},
));
}
#[test]
fn all_requires_supermajority() {
let mut candidate: CandidateEntry = approval_db::v1::CandidateEntry {
candidate: Default::default(),
session: 0,
block_assignments: Default::default(),
approvals: bitvec![BitOrderLsb0, u8; 0; 10],
}.into();
for i in 0..6 {
candidate.mark_approval(i);
}
let approval_entry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 1; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
assert!(!check_approval(&candidate, &approval_entry, RequiredTranches::All));
candidate.mark_approval(6);
assert!(check_approval(&candidate, &approval_entry, RequiredTranches::All));
}
#[test]
fn exact_takes_only_assignments_up_to() {
let mut candidate: CandidateEntry = approval_db::v1::CandidateEntry {
candidate: Default::default(),
session: 0,
block_assignments: Default::default(),
approvals: bitvec![BitOrderLsb0, u8; 0; 10],
}.into();
for i in 0..6 {
candidate.mark_approval(i);
}
let approval_entry = approval_db::v1::ApprovalEntry {
tranches: vec![
approval_db::v1::TrancheEntry {
tranche: 0,
assignments: (0..4).map(|i| (i, 0.into())).collect(),
},
approval_db::v1::TrancheEntry {
tranche: 1,
assignments: (4..6).map(|i| (i, 1.into())).collect(),
},
approval_db::v1::TrancheEntry {
tranche: 2,
assignments: (6..10).map(|i| (i, 0.into())).collect(),
},
],
assignments: bitvec![BitOrderLsb0, u8; 1; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
assert!(check_approval(
&candidate,
&approval_entry,
RequiredTranches::Exact {
needed: 1,
tolerated_missing: 0,
next_no_show: None,
},
));
assert!(!check_approval(
&candidate,
&approval_entry,
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 0,
next_no_show: None,
},
));
assert!(check_approval(
&candidate,
&approval_entry,
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 4,
next_no_show: None,
},
));
}
#[test]
fn tranches_to_approve_everyone_present() {
let block_tick = 0;
let no_show_duration = 10;
let needed_approvals = 4;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; 5],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick + 1);
approval_entry.import_assignment(1, 3, block_tick + 1);
approval_entry.import_assignment(2, 4, block_tick + 2);
let approvals = bitvec![BitOrderLsb0, u8; 1; 5];
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
2,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact { needed: 1, tolerated_missing: 0, next_no_show: None },
);
}
#[test]
fn tranches_to_approve_not_enough_initial_count() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(1, 2, block_tick);
let approvals = bitvec![BitOrderLsb0, u8; 0; 10];
let tranche_now = 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 2,
next_no_show: Some(block_tick + no_show_duration),
maximum_broadcast: DelayTranche::max_value(),
clock_drift: 0,
},
);
}
#[test]
fn tranches_to_approve_no_shows_before_initial_count_treated_same_as_not_initial() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; 10],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; 10];
approvals.set(0, true);
approvals.set(1, true);
let tranche_now = no_show_duration as DelayTranche + 1;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 11,
next_no_show: None,
maximum_broadcast: DelayTranche::max_value(),
clock_drift: 0,
},
);
}
#[test]
fn tranches_to_approve_cover_no_show_not_enough() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let n_validators = 8;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; n_validators],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick);
approval_entry.import_assignment(1, 3, block_tick);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; n_validators];
approvals.set(0, true);
approvals.set(1, true);
// skip 2
approvals.set(3, true);
let tranche_now = no_show_duration as DelayTranche + 1;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 1,
next_no_show: None,
maximum_broadcast: 2, // tranche 1 + 1 no-show
clock_drift: 1 * no_show_duration,
}
);
approvals.set(0, false);
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 1,
next_no_show: None,
maximum_broadcast: 3, // tranche 1 + 2 no-shows
clock_drift: 1 * no_show_duration,
}
);
}
#[test]
fn tranches_to_approve_multi_cover_not_enough() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let n_validators = 8;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; n_validators],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick + 1);
approval_entry.import_assignment(1, 3, block_tick + 1);
approval_entry.import_assignment(2, 4, block_tick + no_show_duration + 2);
approval_entry.import_assignment(2, 5, block_tick + no_show_duration + 2);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; n_validators];
approvals.set(0, true);
approvals.set(1, true);
// skip 2
approvals.set(3, true);
// skip 4
approvals.set(5, true);
let tranche_now = 1;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 1,
tolerated_missing: 0,
next_no_show: Some(block_tick + no_show_duration + 1),
},
);
// first no-show covered.
let tranche_now = no_show_duration as DelayTranche + 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 1,
next_no_show: Some(block_tick + 2*no_show_duration + 2),
},
);
// another no-show in tranche 2.
let tranche_now = (no_show_duration * 2) as DelayTranche + 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 2,
next_no_show: None,
maximum_broadcast: 3, // tranche 2 + 1 uncovered no-show.
clock_drift: 2 * no_show_duration,
},
);
}
#[test]
fn tranches_to_approve_cover_no_show() {
let block_tick = 20;
let no_show_duration = 10;
let needed_approvals = 4;
let n_validators = 8;
let mut approval_entry: ApprovalEntry = approval_db::v1::ApprovalEntry {
tranches: Vec::new(),
assignments: bitvec![BitOrderLsb0, u8; 0; n_validators],
our_assignment: None,
backing_group: GroupIndex(0),
approved: false,
}.into();
approval_entry.import_assignment(0, 0, block_tick);
approval_entry.import_assignment(0, 1, block_tick);
approval_entry.import_assignment(1, 2, block_tick + 1);
approval_entry.import_assignment(1, 3, block_tick + 1);
approval_entry.import_assignment(2, 4, block_tick + no_show_duration + 2);
approval_entry.import_assignment(2, 5, block_tick + no_show_duration + 2);
let mut approvals = bitvec![BitOrderLsb0, u8; 0; n_validators];
approvals.set(0, true);
approvals.set(1, true);
// skip 2
approvals.set(3, true);
approvals.set(4, true);
approvals.set(5, true);
let tranche_now = no_show_duration as DelayTranche + 2;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 2,
tolerated_missing: 1,
next_no_show: None,
},
);
// Even though tranche 2 has 2 validators, it only covers 1 no-show.
// to cover a second no-show, we need to take another non-empty tranche.
approvals.set(0, false);
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Pending {
considered: 2,
next_no_show: None,
maximum_broadcast: 3,
clock_drift: no_show_duration,
},
);
approval_entry.import_assignment(3, 6, block_tick);
approvals.set(6, true);
let tranche_now = no_show_duration as DelayTranche + 3;
assert_eq!(
tranches_to_approve(
&approval_entry,
&approvals,
tranche_now,
block_tick,
no_show_duration,
needed_approvals,
),
RequiredTranches::Exact {
needed: 3,
tolerated_missing: 2,
next_no_show: None,
},
);
}
}
#[test]
fn depth_0_covering_not_treated_as_such() {
let state = State {
assignments: 0,
depth: 0,
covered: 0,
covering: 10,
uncovered: 0,
next_no_show: None,
};
assert_eq!(
state.output(0, 10, 10, 20),
RequiredTranches::Pending {
considered: 0,
next_no_show: None,
maximum_broadcast: DelayTranche::max_value(),
clock_drift: 0,
},
);
}
#[test]
fn depth_0_issued_as_exact_even_when_all() {
let state = State {
assignments: 10,
depth: 0,
covered: 0,
covering: 0,
uncovered: 0,
next_no_show: None,
};
assert_eq!(
state.output(0, 10, 10, 20),
RequiredTranches::Exact {
needed: 0,
tolerated_missing: 0,
next_no_show: None,
},
);
}
@@ -0,0 +1,33 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Approval DB accessors and writers for on-disk persisted approval storage
//! data.
//!
//! We persist data to disk although it is not intended to be used across runs of the
//! program. This is because under medium to long periods of finality stalling, for whatever
//! reason that may be, the amount of data we'd need to keep would be potentially too large
//! for memory.
//!
//! With tens or hundreds of parachains, hundreds of validators, and parablocks
//! in every relay chain block, there can be a humongous amount of information to reference
//! at any given time.
//!
//! As such, we provide a function from this module to clear the database on start-up.
//! In the future, we may use a temporary DB which doesn't need to be wiped, but for the
//! time being we share the same DB with the rest of Substrate.
pub mod v1;
@@ -14,27 +14,10 @@
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Auxiliary DB schema, accessors, and writers for on-disk persisted approval storage
//! data.
//!
//! We persist data to disk although it is not intended to be used across runs of the
//! program. This is because under medium to long periods of finality stalling, for whatever
//! reason that may be, the amount of data we'd need to keep would be potentially too large
//! for memory.
//!
//! With tens or hundreds of parachains, hundreds of validators, and parablocks
//! in every relay chain block, there can be a humongous amount of information to reference
//! at any given time.
//!
//! As such, we provide a function from this module to clear the database on start-up.
//! In the future, we may use a temporary DB which doesn't need to be wiped, but for the
//! time being we share the same DB with the rest of Substrate.
// TODO https://github.com/paritytech/polkadot/issues/1975: remove this
#![allow(unused)]
//! Version 1 of the DB schema.
use sc_client_api::backend::AuxStore;
use polkadot_node_primitives::approval::{DelayTranche, RelayVRF};
use polkadot_node_primitives::approval::{DelayTranche, AssignmentCert};
use polkadot_primitives::v1::{
ValidatorIndex, GroupIndex, CandidateReceipt, SessionIndex, CoreIndex,
BlockNumber, Hash, CandidateHash,
@@ -46,73 +29,95 @@ use std::collections::{BTreeMap, HashMap};
use std::collections::hash_map::Entry;
use bitvec::{vec::BitVec, order::Lsb0 as BitOrderLsb0};
use super::Tick;
#[cfg(test)]
mod tests;
// slot_duration * 2 + DelayTranche gives the number of delay tranches since the
// unix epoch.
#[derive(Encode, Decode, Clone, Copy, Debug, PartialEq)]
pub struct Tick(u64);
pub type Bitfield = BitVec<BitOrderLsb0, u8>;
const STORED_BLOCKS_KEY: &[u8] = b"Approvals_StoredBlocks";
/// Details pertaining to our assignment on a block.
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct OurAssignment {
pub cert: AssignmentCert,
pub tranche: DelayTranche,
pub validator_index: ValidatorIndex,
// Whether the assignment has been triggered already.
pub triggered: bool,
}
/// Metadata regarding a specific tranche of assignments for a specific candidate.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct TrancheEntry {
tranche: DelayTranche,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct TrancheEntry {
pub tranche: DelayTranche,
// Assigned validators, and the instant we received their assignment, rounded
// to the nearest tick.
assignments: Vec<(ValidatorIndex, Tick)>,
pub assignments: Vec<(ValidatorIndex, Tick)>,
}
/// Metadata regarding approval of a particular candidate within the context of some
/// particular block.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct ApprovalEntry {
tranches: Vec<TrancheEntry>,
backing_group: GroupIndex,
// When the next wakeup for this entry should occur. This is either to
// check a no-show or to check if we need to broadcast an assignment.
next_wakeup: Tick,
our_assignment: Option<OurAssignment>,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct ApprovalEntry {
pub tranches: Vec<TrancheEntry>,
pub backing_group: GroupIndex,
pub our_assignment: Option<OurAssignment>,
// `n_validators` bits.
assignments: BitVec<BitOrderLsb0, u8>,
approved: bool,
pub assignments: Bitfield,
pub approved: bool,
}
/// Metadata regarding approval of a particular candidate.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct CandidateEntry {
candidate: CandidateReceipt,
session: SessionIndex,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct CandidateEntry {
pub candidate: CandidateReceipt,
pub session: SessionIndex,
// Assignments are based on blocks, so we need to track assignments separately
// based on the block we are looking at.
block_assignments: BTreeMap<Hash, ApprovalEntry>,
approvals: BitVec<BitOrderLsb0, u8>,
pub block_assignments: BTreeMap<Hash, ApprovalEntry>,
pub approvals: Bitfield,
}
/// Metadata regarding approval of a particular block, by way of approval of the
/// candidates contained within it.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct BlockEntry {
block_hash: Hash,
session: SessionIndex,
slot: Slot,
relay_vrf_story: RelayVRF,
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct BlockEntry {
pub block_hash: Hash,
pub session: SessionIndex,
pub slot: Slot,
/// Random bytes derived from the VRF submitted within the block by the block
/// author as a credential and used as input to approval assignment criteria.
pub relay_vrf_story: [u8; 32],
// The candidates included as-of this block and the index of the core they are
// leaving. Sorted ascending by core index.
candidates: Vec<(CoreIndex, CandidateHash)>,
pub candidates: Vec<(CoreIndex, CandidateHash)>,
// A bitfield where the i'th bit corresponds to the i'th candidate in `candidates`.
// The i'th bit is `true` iff the candidate has been approved in the context of this
// block. The block can be considered approved if the bitfield has all bits set to `true`.
approved_bitfield: BitVec<BitOrderLsb0, u8>,
children: Vec<Hash>,
pub approved_bitfield: Bitfield,
pub children: Vec<Hash>,
}
/// A range from earliest..last block number stored within the DB.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct StoredBlockRange(BlockNumber, BlockNumber);
#[derive(Encode, Decode, Debug, Clone, PartialEq)]
pub struct StoredBlockRange(BlockNumber, BlockNumber);
// TODO https://github.com/paritytech/polkadot/issues/1975: probably in lib.rs
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub(crate) struct OurAssignment { }
impl From<crate::Tick> for Tick {
fn from(tick: crate::Tick) -> Tick {
Tick(tick)
}
}
impl From<Tick> for crate::Tick {
fn from(tick: Tick) -> crate::Tick {
tick.0
}
}
/// Canonicalize some particular block, pruning everything before it and
/// pruning any competing branches at the same height.
@@ -351,9 +356,9 @@ fn load_decode<D: Decode>(store: &impl AuxStore, key: &[u8])
/// candidate and approval entries.
#[derive(Clone)]
pub(crate) struct NewCandidateInfo {
candidate: CandidateReceipt,
backing_group: GroupIndex,
our_assignment: Option<OurAssignment>,
pub candidate: CandidateReceipt,
pub backing_group: GroupIndex,
pub our_assignment: Option<OurAssignment>,
}
/// Record a new block entry.
@@ -364,7 +369,8 @@ pub(crate) struct NewCandidateInfo {
/// parent hash.
///
/// Has no effect if there is already an entry for the block or `candidate_info` returns
/// `None` for any of the candidates referenced by the block entry.
/// `None` for any of the candidates referenced by the block entry. In these cases,
/// no information about new candidates will be referred to by this function.
pub(crate) fn add_block_entry(
store: &impl AuxStore,
parent_hash: Hash,
@@ -372,7 +378,7 @@ pub(crate) fn add_block_entry(
entry: BlockEntry,
n_validators: usize,
candidate_info: impl Fn(&CandidateHash) -> Option<NewCandidateInfo>,
) -> sp_blockchain::Result<()> {
) -> sp_blockchain::Result<Vec<(CandidateHash, CandidateEntry)>> {
let session = entry.session;
let new_block_range = {
@@ -392,13 +398,15 @@ pub(crate) fn add_block_entry(
let mut blocks_at_height = load_blocks_at_height(store, number)?;
if blocks_at_height.contains(&entry.block_hash) {
// seems we already have a block entry for this block. nothing to do here.
return Ok(())
return Ok(Vec::new())
}
blocks_at_height.push(entry.block_hash);
(blocks_at_height_key(number), blocks_at_height.encode())
};
let mut candidate_entries = Vec::with_capacity(entry.candidates.len());
let candidate_entry_updates = {
let mut updated_entries = Vec::with_capacity(entry.candidates.len());
for &(_, ref candidate_hash) in &entry.candidates {
@@ -407,7 +415,7 @@ pub(crate) fn add_block_entry(
backing_group,
our_assignment,
} = match candidate_info(candidate_hash) {
None => return Ok(()),
None => return Ok(Vec::new()),
Some(info) => info,
};
@@ -424,7 +432,6 @@ pub(crate) fn add_block_entry(
ApprovalEntry {
tranches: Vec::new(),
backing_group,
next_wakeup: 0,
our_assignment,
assignments: bitvec::bitvec![BitOrderLsb0, u8; 0; n_validators],
approved: false,
@@ -434,6 +441,8 @@ pub(crate) fn add_block_entry(
updated_entries.push(
(candidate_entry_key(&candidate_hash), candidate_entry.encode())
);
candidate_entries.push((*candidate_hash, candidate_entry));
}
updated_entries
@@ -466,11 +475,61 @@ pub(crate) fn add_block_entry(
store.insert_aux(&all_keys_and_values, &[])?;
Ok(())
Ok(candidate_entries)
}
// An atomic transaction of multiple candidate or block entries.
#[derive(Default)]
#[must_use = "Transactions do nothing unless written to a DB"]
pub struct Transaction {
block_entries: HashMap<Hash, BlockEntry>,
candidate_entries: HashMap<CandidateHash, CandidateEntry>,
}
impl Transaction {
/// Put a block entry in the transaction, overwriting any other with the
/// same hash.
pub(crate) fn put_block_entry(&mut self, entry: BlockEntry) {
let hash = entry.block_hash;
let _ = self.block_entries.insert(hash, entry);
}
/// Put a candidate entry in the transaction, overwriting any other with the
/// same hash.
pub(crate) fn put_candidate_entry(&mut self, hash: CandidateHash, entry: CandidateEntry) {
let _ = self.candidate_entries.insert(hash, entry);
}
/// Write the contents of the transaction, atomically, to the DB.
pub(crate) fn write(self, db: &impl AuxStore) -> sp_blockchain::Result<()> {
if self.block_entries.is_empty() && self.candidate_entries.is_empty() {
return Ok(())
}
let blocks: Vec<_> = self.block_entries.into_iter().map(|(hash, entry)| {
let k = block_entry_key(&hash);
let v = entry.encode();
(k, v)
}).collect();
let candidates: Vec<_> = self.candidate_entries.into_iter().map(|(hash, entry)| {
let k = candidate_entry_key(&hash);
let v = entry.encode();
(k, v)
}).collect();
let kv = blocks.iter().map(|(k, v)| (&k[..], &v[..]))
.chain(candidates.iter().map(|(k, v)| (&k[..], &v[..])))
.collect::<Vec<_>>();
db.insert_aux(&kv, &[])
}
}
/// Load the stored-blocks key from the state.
pub(crate) fn load_stored_blocks(store: &impl AuxStore)
fn load_stored_blocks(store: &impl AuxStore)
-> sp_blockchain::Result<Option<StoredBlockRange>>
{
load_decode(store, STORED_BLOCKS_KEY)
@@ -17,8 +17,8 @@
//! Tests for the aux-schema of approval voting.
use super::*;
use std::cell::RefCell;
use polkadot_primitives::v1::Id as ParaId;
use std::cell::RefCell;
#[derive(Default)]
struct TestStore {
@@ -49,28 +49,28 @@ impl AuxStore for TestStore {
}
impl TestStore {
fn write_stored_blocks(&self, range: StoredBlockRange) {
pub(crate) fn write_stored_blocks(&self, range: StoredBlockRange) {
self.inner.borrow_mut().insert(
STORED_BLOCKS_KEY.to_vec(),
range.encode(),
);
}
fn write_blocks_at_height(&self, height: BlockNumber, blocks: &[Hash]) {
pub(crate) fn write_blocks_at_height(&self, height: BlockNumber, blocks: &[Hash]) {
self.inner.borrow_mut().insert(
blocks_at_height_key(height).to_vec(),
blocks.encode(),
);
}
fn write_block_entry(&self, block_hash: &Hash, entry: &BlockEntry) {
pub(crate) fn write_block_entry(&self, block_hash: &Hash, entry: &BlockEntry) {
self.inner.borrow_mut().insert(
block_entry_key(block_hash).to_vec(),
entry.encode(),
);
}
fn write_candidate_entry(&self, candidate_hash: &CandidateHash, entry: &CandidateEntry) {
pub(crate) fn write_candidate_entry(&self, candidate_hash: &CandidateHash, entry: &CandidateEntry) {
self.inner.borrow_mut().insert(
candidate_entry_key(candidate_hash).to_vec(),
entry.encode(),
@@ -89,8 +89,8 @@ fn make_block_entry(
BlockEntry {
block_hash,
session: 1,
slot: 1.into(),
relay_vrf_story: RelayVRF([0u8; 32]),
slot: Slot::from(1),
relay_vrf_story: [0u8; 32],
approved_bitfield: make_bitvec(candidates.len()),
candidates,
children: Vec::new(),
@@ -129,7 +129,6 @@ fn read_write() {
(hash_a, ApprovalEntry {
tranches: Vec::new(),
backing_group: GroupIndex(1),
next_wakeup: 1000,
our_assignment: None,
assignments: Default::default(),
approved: false,
@@ -156,7 +155,7 @@ fn read_write() {
];
let delete_keys: Vec<_> = delete_keys.iter().map(|k| &k[..]).collect();
store.insert_aux(&[], &delete_keys);
store.insert_aux(&[], &delete_keys).unwrap();
assert!(load_stored_blocks(&store).unwrap().is_none());
assert!(load_blocks_at_height(&store, 1).unwrap().is_empty());
@@ -296,7 +295,6 @@ fn clear_works() {
(hash_a, ApprovalEntry {
tranches: Vec::new(),
backing_group: GroupIndex(1),
next_wakeup: 1000,
our_assignment: None,
assignments: Default::default(),
approved: false,
@@ -331,7 +329,7 @@ fn canonicalize_works() {
// -> B1 -> C1 -> D1
// A -> B2 -> C2 -> D2
//
// We'll canonicalize C1. Everything except D1 should disappear.
// We'll canonicalize C1. Everytning except D1 should disappear.
//
// Candidates:
// Cand1 in B2
@@ -0,0 +1,782 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Assignment criteria VRF generation and checking.
use polkadot_node_primitives::approval::{
self as approval_types, AssignmentCert, AssignmentCertKind, DelayTranche, RelayVRFStory,
};
use polkadot_primitives::v1::{
CoreIndex, ValidatorIndex, SessionInfo, AssignmentPair, AssignmentId, GroupIndex,
};
use sc_keystore::LocalKeystore;
use parity_scale_codec::{Encode, Decode};
use sp_application_crypto::Public;
use merlin::Transcript;
use schnorrkel::vrf::VRFInOut;
use std::collections::HashMap;
use std::collections::hash_map::Entry;
use super::LOG_TARGET;
/// Details pertaining to our assignment on a block.
#[derive(Debug, Clone, Encode, Decode, PartialEq)]
pub struct OurAssignment {
cert: AssignmentCert,
tranche: DelayTranche,
validator_index: ValidatorIndex,
// Whether the assignment has been triggered already.
triggered: bool,
}
impl OurAssignment {
pub(crate) fn cert(&self) -> &AssignmentCert {
&self.cert
}
pub(crate) fn tranche(&self) -> DelayTranche {
self.tranche
}
pub(crate) fn validator_index(&self) -> ValidatorIndex {
self.validator_index
}
pub(crate) fn triggered(&self) -> bool {
self.triggered
}
pub(crate) fn mark_triggered(&mut self) {
self.triggered = true;
}
}
impl From<crate::approval_db::v1::OurAssignment> for OurAssignment {
fn from(entry: crate::approval_db::v1::OurAssignment) -> Self {
OurAssignment {
cert: entry.cert,
tranche: entry.tranche,
validator_index: entry.validator_index,
triggered: entry.triggered,
}
}
}
impl From<OurAssignment> for crate::approval_db::v1::OurAssignment {
fn from(entry: OurAssignment) -> Self {
Self {
cert: entry.cert,
tranche: entry.tranche,
validator_index: entry.validator_index,
triggered: entry.triggered,
}
}
}
fn relay_vrf_modulo_transcript(
relay_vrf_story: RelayVRFStory,
sample: u32,
) -> Transcript {
// combine the relay VRF story with a sample number.
let mut t = Transcript::new(approval_types::RELAY_VRF_MODULO_CONTEXT);
t.append_message(b"RC-VRF", &relay_vrf_story.0);
sample.using_encoded(|s| t.append_message(b"sample", s));
t
}
fn relay_vrf_modulo_core(
vrf_in_out: &VRFInOut,
n_cores: u32,
) -> CoreIndex {
let bytes: [u8; 4] = vrf_in_out.make_bytes(approval_types::CORE_RANDOMNESS_CONTEXT);
// interpret as little-endian u32.
let random_core = u32::from_le_bytes(bytes) % n_cores;
CoreIndex(random_core)
}
fn relay_vrf_delay_transcript(
relay_vrf_story: RelayVRFStory,
core_index: CoreIndex,
) -> Transcript {
let mut t = Transcript::new(approval_types::RELAY_VRF_DELAY_CONTEXT);
t.append_message(b"RC-VRF", &relay_vrf_story.0);
core_index.0.using_encoded(|s| t.append_message(b"core", s));
t
}
fn relay_vrf_delay_tranche(
vrf_in_out: &VRFInOut,
num_delay_tranches: u32,
zeroth_delay_tranche_width: u32,
) -> DelayTranche {
let bytes: [u8; 4] = vrf_in_out.make_bytes(approval_types::TRANCHE_RANDOMNESS_CONTEXT);
// interpret as little-endian u32 and reduce by the number of tranches.
let wide_tranche = u32::from_le_bytes(bytes) % (num_delay_tranches + zeroth_delay_tranche_width);
// Consolidate early results to tranche zero so tranche zero is extra wide.
wide_tranche.saturating_sub(zeroth_delay_tranche_width)
}
fn assigned_core_transcript(core_index: CoreIndex) -> Transcript {
let mut t = Transcript::new(approval_types::ASSIGNED_CORE_CONTEXT);
core_index.0.using_encoded(|s| t.append_message(b"core", s));
t
}
/// Information about the world assignments are being produced in.
#[derive(Clone)]
pub(crate) struct Config {
/// The assignment public keys for validators.
assignment_keys: Vec<AssignmentId>,
/// The groups of validators assigned to each core.
validator_groups: Vec<Vec<ValidatorIndex>>,
/// The number of availability cores used by the protocol during this session.
n_cores: u32,
/// The zeroth delay tranche width.
zeroth_delay_tranche_width: u32,
/// The number of samples we do of relay_vrf_modulo.
relay_vrf_modulo_samples: u32,
/// The number of delay tranches in total.
n_delay_tranches: u32,
}
impl<'a> From<&'a SessionInfo> for Config {
fn from(s: &'a SessionInfo) -> Self {
Config {
assignment_keys: s.assignment_keys.clone(),
validator_groups: s.validator_groups.clone(),
n_cores: s.n_cores.clone(),
zeroth_delay_tranche_width: s.zeroth_delay_tranche_width.clone(),
relay_vrf_modulo_samples: s.relay_vrf_modulo_samples.clone(),
n_delay_tranches: s.n_delay_tranches.clone(),
}
}
}
/// A trait for producing and checking assignments. Used to mock.
pub(crate) trait AssignmentCriteria {
fn compute_assignments(
&self,
keystore: &LocalKeystore,
relay_vrf_story: RelayVRFStory,
config: &Config,
leaving_cores: Vec<(CoreIndex, GroupIndex)>,
) -> HashMap<CoreIndex, OurAssignment>;
fn check_assignment_cert(
&self,
claimed_core_index: CoreIndex,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
assignment: &AssignmentCert,
backing_group: GroupIndex,
) -> Result<DelayTranche, InvalidAssignment>;
}
pub(crate) struct RealAssignmentCriteria;
impl AssignmentCriteria for RealAssignmentCriteria {
fn compute_assignments(
&self,
keystore: &LocalKeystore,
relay_vrf_story: RelayVRFStory,
config: &Config,
leaving_cores: Vec<(CoreIndex, GroupIndex)>,
) -> HashMap<CoreIndex, OurAssignment> {
compute_assignments(
keystore,
relay_vrf_story,
config,
leaving_cores,
)
}
fn check_assignment_cert(
&self,
claimed_core_index: CoreIndex,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
assignment: &AssignmentCert,
backing_group: GroupIndex,
) -> Result<DelayTranche, InvalidAssignment> {
check_assignment_cert(
claimed_core_index,
validator_index,
config,
relay_vrf_story,
assignment,
backing_group,
)
}
}
/// Compute the assignments for a given block. Returns a map containing all assignments to cores in
/// the block. If more than one assignment targets the given core, only the earliest assignment is kept.
///
/// The `leaving_cores` parameter indicates all cores within the block where a candidate was included,
/// as well as the group index backing those.
///
/// The current description of the protocol assigns every validator to check every core. But at different times.
/// The idea is that most assignments are never triggered and fall by the wayside.
///
/// This will not assign to anything the local validator was part of the backing group for.
pub(crate) fn compute_assignments(
keystore: &LocalKeystore,
relay_vrf_story: RelayVRFStory,
config: &Config,
leaving_cores: impl IntoIterator<Item = (CoreIndex, GroupIndex)> + Clone,
) -> HashMap<CoreIndex, OurAssignment> {
let (index, assignments_key): (ValidatorIndex, AssignmentPair) = {
let key = config.assignment_keys.iter().enumerate()
.filter_map(|(i, p)| match keystore.key_pair(p) {
Ok(pair) => Some((i as ValidatorIndex, pair)),
Err(sc_keystore::Error::PairNotFound(_)) => None,
Err(e) => {
tracing::warn!(target: LOG_TARGET, "Encountered keystore error: {:?}", e);
None
}
})
.next();
match key {
None => return Default::default(),
Some(k) => k,
}
};
// Ignore any cores where the assigned group is our own.
let leaving_cores = leaving_cores.into_iter()
.filter(|&(_, ref g)| !is_in_backing_group(&config.validator_groups, index, *g))
.map(|(c, _)| c)
.collect::<Vec<_>>();
let assignments_key: &sp_application_crypto::sr25519::Pair = assignments_key.as_ref();
let assignments_key: &schnorrkel::Keypair = assignments_key.as_ref();
let mut assignments = HashMap::new();
// First run `RelayVRFModulo` for each sample.
compute_relay_vrf_modulo_assignments(
&assignments_key,
index,
config,
relay_vrf_story.clone(),
leaving_cores.iter().cloned(),
&mut assignments,
);
// Then run `RelayVRFDelay` once for the whole block.
compute_relay_vrf_delay_assignments(
&assignments_key,
index,
config,
relay_vrf_story,
leaving_cores,
&mut assignments,
);
assignments
}
fn compute_relay_vrf_modulo_assignments(
assignments_key: &schnorrkel::Keypair,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
leaving_cores: impl IntoIterator<Item = CoreIndex> + Clone,
assignments: &mut HashMap<CoreIndex, OurAssignment>,
) {
for rvm_sample in 0..config.relay_vrf_modulo_samples {
let mut core = Default::default();
let maybe_assignment = {
// Extra scope to ensure borrowing instead of moving core
// into closure.
let core = &mut core;
assignments_key.vrf_sign_extra_after_check(
relay_vrf_modulo_transcript(relay_vrf_story.clone(), rvm_sample),
|vrf_in_out| {
*core = relay_vrf_modulo_core(&vrf_in_out, config.n_cores);
if leaving_cores.clone().into_iter().any(|c| c == *core) {
Some(assigned_core_transcript(*core))
} else {
None
}
}
)
};
if let Some((vrf_in_out, vrf_proof, _)) = maybe_assignment {
// Sanity: `core` is always initialized to non-default here, as the closure above
// has been executed.
let cert = AssignmentCert {
kind: AssignmentCertKind::RelayVRFModulo { sample: rvm_sample },
vrf: (approval_types::VRFOutput(vrf_in_out.to_output()), approval_types::VRFProof(vrf_proof)),
};
// All assignments of type RelayVRFModulo have tranche 0.
assignments.entry(core).or_insert(OurAssignment {
cert,
tranche: 0,
validator_index,
triggered: false,
});
}
}
}
fn compute_relay_vrf_delay_assignments(
assignments_key: &schnorrkel::Keypair,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
leaving_cores: impl IntoIterator<Item = CoreIndex>,
assignments: &mut HashMap<CoreIndex, OurAssignment>,
) {
for core in leaving_cores {
let (vrf_in_out, vrf_proof, _) = assignments_key.vrf_sign(
relay_vrf_delay_transcript(relay_vrf_story.clone(), core),
);
let tranche = relay_vrf_delay_tranche(
&vrf_in_out,
config.n_delay_tranches,
config.zeroth_delay_tranche_width,
);
let cert = AssignmentCert {
kind: AssignmentCertKind::RelayVRFDelay { core_index: core },
vrf: (approval_types::VRFOutput(vrf_in_out.to_output()), approval_types::VRFProof(vrf_proof)),
};
let our_assignment = OurAssignment {
cert,
tranche,
validator_index,
triggered: false,
};
match assignments.entry(core) {
Entry::Vacant(e) => { let _ = e.insert(our_assignment); }
Entry::Occupied(mut e) => if e.get().tranche > our_assignment.tranche {
e.insert(our_assignment);
},
}
}
}
/// Assignment invalid.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct InvalidAssignment;
impl std::fmt::Display for InvalidAssignment {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "Invalid Assignment")
}
}
impl std::error::Error for InvalidAssignment { }
/// Checks the crypto of an assignment cert. Failure conditions:
/// * Validator index out of bounds
/// * VRF signature check fails
/// * VRF output doesn't match assigned core
/// * Core is not covered by extra data in signature
/// * Core index out of bounds
/// * Sample is out of bounds
/// * Validator is present in backing group.
///
/// This function does not check whether the core is actually a valid assignment or not. That should be done
/// outside of the scope of this function.
pub(crate) fn check_assignment_cert(
claimed_core_index: CoreIndex,
validator_index: ValidatorIndex,
config: &Config,
relay_vrf_story: RelayVRFStory,
assignment: &AssignmentCert,
backing_group: GroupIndex,
) -> Result<DelayTranche, InvalidAssignment> {
let validator_public = config.assignment_keys
.get(validator_index as usize)
.ok_or(InvalidAssignment)?;
let public = schnorrkel::PublicKey::from_bytes(validator_public.as_slice())
.map_err(|_| InvalidAssignment)?;
if claimed_core_index.0 >= config.n_cores {
return Err(InvalidAssignment);
}
// Check that the validator was not part of the backing group
// and not already assigned.
let is_in_backing = is_in_backing_group(
&config.validator_groups,
validator_index,
backing_group,
);
if is_in_backing {
return Err(InvalidAssignment);
}
let &(ref vrf_output, ref vrf_proof) = &assignment.vrf;
match assignment.kind {
AssignmentCertKind::RelayVRFModulo { sample } => {
if sample >= config.relay_vrf_modulo_samples {
return Err(InvalidAssignment);
}
let (vrf_in_out, _) = public.vrf_verify_extra(
relay_vrf_modulo_transcript(relay_vrf_story, sample),
&vrf_output.0,
&vrf_proof.0,
assigned_core_transcript(claimed_core_index),
).map_err(|_| InvalidAssignment)?;
// ensure that the `vrf_in_out` actually gives us the claimed core.
if relay_vrf_modulo_core(&vrf_in_out, config.n_cores) == claimed_core_index {
Ok(0)
} else {
Err(InvalidAssignment)
}
}
AssignmentCertKind::RelayVRFDelay { core_index } => {
if core_index != claimed_core_index {
return Err(InvalidAssignment);
}
let (vrf_in_out, _) = public.vrf_verify(
relay_vrf_delay_transcript(relay_vrf_story, core_index),
&vrf_output.0,
&vrf_proof.0,
).map_err(|_| InvalidAssignment)?;
Ok(relay_vrf_delay_tranche(
&vrf_in_out,
config.n_delay_tranches,
config.zeroth_delay_tranche_width,
))
}
}
}
fn is_in_backing_group(
validator_groups: &[Vec<ValidatorIndex>],
validator: ValidatorIndex,
group: GroupIndex,
) -> bool {
validator_groups.get(group.0 as usize).map_or(false, |g| g.contains(&validator))
}
#[cfg(test)]
mod tests {
use super::*;
use sp_keystore::CryptoStore;
use sp_keyring::sr25519::Keyring as Sr25519Keyring;
use sp_application_crypto::sr25519;
use sp_core::crypto::Pair as PairT;
use polkadot_primitives::v1::ASSIGNMENT_KEY_TYPE_ID;
use polkadot_node_primitives::approval::{VRFOutput, VRFProof};
// sets up a keystore with the given keyring accounts.
async fn make_keystore(accounts: &[Sr25519Keyring]) -> LocalKeystore {
let store = LocalKeystore::in_memory();
for s in accounts.iter().copied().map(|k| k.to_seed()) {
store.sr25519_generate_new(
ASSIGNMENT_KEY_TYPE_ID,
Some(s.as_str()),
).await.unwrap();
}
store
}
fn assignment_keys(accounts: &[Sr25519Keyring]) -> Vec<AssignmentId> {
assignment_keys_plus_random(accounts, 0)
}
fn assignment_keys_plus_random(accounts: &[Sr25519Keyring], random: usize) -> Vec<AssignmentId> {
let gen_random = (0..random).map(|_|
AssignmentId::from(sr25519::Pair::generate().0.public())
);
accounts.iter()
.map(|k| AssignmentId::from(k.public()))
.chain(gen_random)
.collect()
}
fn basic_groups(n_validators: usize, n_groups: usize) -> Vec<Vec<ValidatorIndex>> {
let size = n_validators / n_groups;
let big_groups = n_validators % n_groups;
let scraps = n_groups * size;
(0..n_groups).map(|i| {
(i * size .. (i + 1) *size)
.chain(if i < big_groups { Some(scraps + i) } else { None })
.map(|j| j as ValidatorIndex)
.collect::<Vec<_>>()
}).collect()
}
// used for generating assignments where the validity of the VRF doesn't matter.
fn garbage_vrf() -> (VRFOutput, VRFProof) {
let key = Sr25519Keyring::Alice.pair();
let key: &schnorrkel::Keypair = key.as_ref();
let (o, p, _) = key.vrf_sign(Transcript::new(b"test-garbage"));
(VRFOutput(o.to_output()), VRFProof(p))
}
#[test]
fn assignments_produced_for_non_backing() {
let keystore = futures::executor::block_on(
make_keystore(&[Sr25519Keyring::Alice])
);
let relay_vrf_story = RelayVRFStory([42u8; 32]);
let assignments = compute_assignments(
&keystore,
relay_vrf_story,
&Config {
assignment_keys: assignment_keys(&[
Sr25519Keyring::Alice,
Sr25519Keyring::Bob,
Sr25519Keyring::Charlie,
]),
validator_groups: vec![vec![0], vec![1, 2]],
n_cores: 2,
zeroth_delay_tranche_width: 10,
relay_vrf_modulo_samples: 3,
n_delay_tranches: 40,
},
vec![(CoreIndex(0), GroupIndex(1)), (CoreIndex(1), GroupIndex(0))],
);
// Note that alice is in group 0, which was the backing group for core 1.
// Alice should have self-assigned to check core 0 but not 1.
assert_eq!(assignments.len(), 1);
assert!(assignments.get(&CoreIndex(0)).is_some());
}
#[test]
fn assign_to_nonzero_core() {
let keystore = futures::executor::block_on(
make_keystore(&[Sr25519Keyring::Alice])
);
let relay_vrf_story = RelayVRFStory([42u8; 32]);
let assignments = compute_assignments(
&keystore,
relay_vrf_story,
&Config {
assignment_keys: assignment_keys(&[
Sr25519Keyring::Alice,
Sr25519Keyring::Bob,
Sr25519Keyring::Charlie,
]),
validator_groups: vec![vec![0], vec![1, 2]],
n_cores: 2,
zeroth_delay_tranche_width: 10,
relay_vrf_modulo_samples: 3,
n_delay_tranches: 40,
},
vec![(CoreIndex(0), GroupIndex(0)), (CoreIndex(1), GroupIndex(1))],
);
assert_eq!(assignments.len(), 1);
assert!(assignments.get(&CoreIndex(1)).is_some());
}
struct MutatedAssignment {
core: CoreIndex,
cert: AssignmentCert,
group: GroupIndex,
own_group: GroupIndex,
val_index: ValidatorIndex,
config: Config,
}
// This fails if the closure requests to skip everything.
fn check_mutated_assignments(
n_validators: usize,
n_cores: usize,
rotation_offset: usize,
f: impl Fn(&mut MutatedAssignment) -> Option<bool>, // None = skip
) {
let keystore = futures::executor::block_on(
make_keystore(&[Sr25519Keyring::Alice])
);
let group_for_core = |i| GroupIndex(((i + rotation_offset) % n_cores) as _);
let config = Config {
assignment_keys: assignment_keys_plus_random(&[Sr25519Keyring::Alice], n_validators - 1),
validator_groups: basic_groups(n_validators, n_cores),
n_cores: n_cores as u32,
zeroth_delay_tranche_width: 10,
relay_vrf_modulo_samples: 3,
n_delay_tranches: 40,
};
let relay_vrf_story = RelayVRFStory([42u8; 32]);
let assignments = compute_assignments(
&keystore,
relay_vrf_story.clone(),
&config,
(0..n_cores)
.map(|i| (
CoreIndex(i as u32),
group_for_core(i),
))
.collect::<Vec<_>>(),
);
let mut counted = 0;
for (core, assignment) in assignments {
let mut mutated = MutatedAssignment {
core,
group: group_for_core(core.0 as _),
cert: assignment.cert,
own_group: GroupIndex(0),
val_index: 0,
config: config.clone(),
};
let expected = match f(&mut mutated) {
None => continue,
Some(e) => e,
};
counted += 1;
let is_good = check_assignment_cert(
mutated.core,
mutated.val_index,
&mutated.config,
relay_vrf_story.clone(),
&mutated.cert,
mutated.group,
).is_ok();
assert_eq!(expected, is_good)
}
assert!(counted > 0);
}
#[test]
fn computed_assignments_pass_checks() {
check_mutated_assignments(200, 100, 25, |_| Some(true));
}
#[test]
fn check_rejects_claimed_core_out_of_bounds() {
check_mutated_assignments(200, 100, 25, |m| {
m.core.0 += 100;
Some(false)
});
}
#[test]
fn check_rejects_in_backing_group() {
check_mutated_assignments(200, 100, 25, |m| {
m.group = m.own_group;
Some(false)
});
}
#[test]
fn check_rejects_nonexistent_key() {
check_mutated_assignments(200, 100, 25, |m| {
m.val_index += 200;
Some(false)
});
}
#[test]
fn check_rejects_delay_bad_vrf() {
check_mutated_assignments(40, 10, 8, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFDelay { .. } => {
m.cert.vrf = garbage_vrf();
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_modulo_bad_vrf() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFModulo { .. } => {
m.cert.vrf = garbage_vrf();
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_modulo_sample_out_of_bounds() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFModulo { sample } => {
m.config.relay_vrf_modulo_samples = sample;
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_delay_claimed_core_wrong() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFDelay { .. } => {
m.core = CoreIndex((m.core.0 + 1) % 100);
Some(false)
}
_ => None, // skip everything else.
}
});
}
#[test]
fn check_rejects_modulo_core_wrong() {
check_mutated_assignments(200, 100, 25, |m| {
match m.cert.kind.clone() {
AssignmentCertKind::RelayVRFModulo { .. } => {
m.core = CoreIndex((m.core.0 + 1) % 100);
Some(false)
}
_ => None, // skip everything else.
}
});
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,412 @@
// Copyright 2020 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Entries pertaining to approval which need to be persisted.
//!
//! The actual persisting of data is handled by the `approval_db` module.
//! Within that context, things are plain-old-data. Within this module,
//! data and logic are intertwined.
use polkadot_node_primitives::approval::{DelayTranche, RelayVRFStory, AssignmentCert};
use polkadot_primitives::v1::{
ValidatorIndex, CandidateReceipt, SessionIndex, GroupIndex, CoreIndex,
Hash, CandidateHash,
};
use sp_consensus_slots::Slot;
use std::collections::BTreeMap;
use bitvec::{slice::BitSlice, vec::BitVec, order::Lsb0 as BitOrderLsb0};
use super::time::Tick;
use super::criteria::OurAssignment;
/// Metadata regarding a specific tranche of assignments for a specific candidate.
#[derive(Debug, Clone, PartialEq)]
pub struct TrancheEntry {
tranche: DelayTranche,
// Assigned validators, and the instant we received their assignment, rounded
// to the nearest tick.
assignments: Vec<(ValidatorIndex, Tick)>,
}
impl TrancheEntry {
/// Get the tranche of this entry.
pub fn tranche(&self) -> DelayTranche {
self.tranche
}
/// Get the assignments for this entry.
pub fn assignments(&self) -> &[(ValidatorIndex, Tick)] {
&self.assignments
}
}
impl From<crate::approval_db::v1::TrancheEntry> for TrancheEntry {
fn from(entry: crate::approval_db::v1::TrancheEntry) -> Self {
TrancheEntry {
tranche: entry.tranche,
assignments: entry.assignments.into_iter().map(|(v, t)| (v, t.into())).collect(),
}
}
}
impl From<TrancheEntry> for crate::approval_db::v1::TrancheEntry {
fn from(entry: TrancheEntry) -> Self {
Self {
tranche: entry.tranche,
assignments: entry.assignments.into_iter().map(|(v, t)| (v, t.into())).collect(),
}
}
}
/// Metadata regarding approval of a particular candidate within the context of some
/// particular block.
#[derive(Debug, Clone, PartialEq)]
pub struct ApprovalEntry {
tranches: Vec<TrancheEntry>,
backing_group: GroupIndex,
our_assignment: Option<OurAssignment>,
// `n_validators` bits.
assignments: BitVec<BitOrderLsb0, u8>,
approved: bool,
}
impl ApprovalEntry {
// Access our assignment for this approval entry.
pub fn our_assignment(&self) -> Option<&OurAssignment> {
self.our_assignment.as_ref()
}
// Note that our assignment is triggered. No-op if already triggered.
pub fn trigger_our_assignment(&mut self, tick_now: Tick)
-> Option<(AssignmentCert, ValidatorIndex)>
{
let our = self.our_assignment.as_mut().and_then(|a| {
if a.triggered() { return None }
a.mark_triggered();
Some(a.clone())
});
our.map(|a| {
self.import_assignment(a.tranche(), a.validator_index(), tick_now);
(a.cert().clone(), a.validator_index())
})
}
/// Whether a validator is already assigned.
pub fn is_assigned(&self, validator_index: ValidatorIndex) -> bool {
self.assignments.get(validator_index as usize).map(|b| *b).unwrap_or(false)
}
/// Import an assignment. No-op if already assigned on the same tranche.
pub fn import_assignment(
&mut self,
tranche: DelayTranche,
validator_index: ValidatorIndex,
tick_now: Tick,
) {
// linear search probably faster than binary. not many tranches typically.
let idx = match self.tranches.iter().position(|t| t.tranche >= tranche) {
Some(pos) => {
if self.tranches[pos].tranche > tranche {
self.tranches.insert(pos, TrancheEntry {
tranche: tranche,
assignments: Vec::new(),
});
}
pos
}
None => {
self.tranches.push(TrancheEntry {
tranche: tranche,
assignments: Vec::new(),
});
self.tranches.len() - 1
}
};
self.tranches[idx].assignments.push((validator_index, tick_now));
self.assignments.set(validator_index as _, true);
}
// Produce a bitvec indicating the assignments of all validators up to and
// including `tranche`.
pub fn assignments_up_to(&self, tranche: DelayTranche) -> BitVec<BitOrderLsb0, u8> {
self.tranches.iter()
.take_while(|e| e.tranche <= tranche)
.fold(bitvec::bitvec![BitOrderLsb0, u8; 0; self.assignments.len()], |mut a, e| {
for &(v, _) in &e.assignments {
a.set(v as _, true);
}
a
})
}
/// Whether the approval entry is approved
pub fn is_approved(&self) -> bool {
self.approved
}
/// Mark the approval entry as approved.
pub fn mark_approved(&mut self) {
self.approved = true;
}
/// Access the tranches.
pub fn tranches(&self) -> &[TrancheEntry] {
&self.tranches
}
/// Get the number of validators in this approval entry.
pub fn n_validators(&self) -> usize {
self.assignments.len()
}
/// Get the backing group index of the approval entry.
pub fn backing_group(&self) -> GroupIndex {
self.backing_group
}
/// For tests: set our assignment.
#[cfg(test)]
pub fn set_our_assignment(&mut self, our_assignment: OurAssignment) {
self.our_assignment = Some(our_assignment);
}
}
impl From<crate::approval_db::v1::ApprovalEntry> for ApprovalEntry {
fn from(entry: crate::approval_db::v1::ApprovalEntry) -> Self {
ApprovalEntry {
tranches: entry.tranches.into_iter().map(Into::into).collect(),
backing_group: entry.backing_group,
our_assignment: entry.our_assignment.map(Into::into),
assignments: entry.assignments,
approved: entry.approved,
}
}
}
impl From<ApprovalEntry> for crate::approval_db::v1::ApprovalEntry {
fn from(entry: ApprovalEntry) -> Self {
Self {
tranches: entry.tranches.into_iter().map(Into::into).collect(),
backing_group: entry.backing_group,
our_assignment: entry.our_assignment.map(Into::into),
assignments: entry.assignments,
approved: entry.approved,
}
}
}
/// Metadata regarding approval of a particular candidate.
#[derive(Debug, Clone, PartialEq)]
pub struct CandidateEntry {
candidate: CandidateReceipt,
session: SessionIndex,
// Assignments are based on blocks, so we need to track assignments separately
// based on the block we are looking at.
block_assignments: BTreeMap<Hash, ApprovalEntry>,
approvals: BitVec<BitOrderLsb0, u8>,
}
impl CandidateEntry {
/// Access the bit-vec of approvals.
pub fn approvals(&self) -> &BitSlice<BitOrderLsb0, u8> {
&self.approvals
}
/// Note that a given validator has approved. Return the previous approval state.
pub fn mark_approval(&mut self, validator: ValidatorIndex) -> bool {
let prev = self.approvals.get(validator as usize).map(|b| *b).unwrap_or(false);
self.approvals.set(validator as usize, true);
prev
}
/// Get the candidate receipt.
pub fn candidate_receipt(&self) -> &CandidateReceipt {
&self.candidate
}
/// Get the approval entry, mutably, for this candidate under a specific block.
pub fn approval_entry_mut(&mut self, block_hash: &Hash) -> Option<&mut ApprovalEntry> {
self.block_assignments.get_mut(block_hash)
}
/// Get the approval entry for this candidate under a specific block.
pub fn approval_entry(&self, block_hash: &Hash) -> Option<&ApprovalEntry> {
self.block_assignments.get(block_hash)
}
/// Iterate over approval entries.
pub fn iter_approval_entries(&self) -> impl IntoIterator<Item = (&Hash, &ApprovalEntry)> {
self.block_assignments.iter()
}
#[cfg(test)]
pub fn add_approval_entry(
&mut self,
block_hash: Hash,
approval_entry: ApprovalEntry,
) {
self.block_assignments.insert(block_hash, approval_entry);
}
}
impl From<crate::approval_db::v1::CandidateEntry> for CandidateEntry {
fn from(entry: crate::approval_db::v1::CandidateEntry) -> Self {
CandidateEntry {
candidate: entry.candidate,
session: entry.session,
block_assignments: entry.block_assignments.into_iter().map(|(h, ae)| (h, ae.into())).collect(),
approvals: entry.approvals,
}
}
}
impl From<CandidateEntry> for crate::approval_db::v1::CandidateEntry {
fn from(entry: CandidateEntry) -> Self {
Self {
candidate: entry.candidate,
session: entry.session,
block_assignments: entry.block_assignments.into_iter().map(|(h, ae)| (h, ae.into())).collect(),
approvals: entry.approvals,
}
}
}
/// Metadata regarding approval of a particular block, by way of approval of the
/// candidates contained within it.
#[derive(Debug, Clone, PartialEq)]
pub struct BlockEntry {
block_hash: Hash,
session: SessionIndex,
slot: Slot,
relay_vrf_story: RelayVRFStory,
// The candidates included as-of this block and the index of the core they are
// leaving. Sorted ascending by core index.
candidates: Vec<(CoreIndex, CandidateHash)>,
// A bitfield where the i'th bit corresponds to the i'th candidate in `candidates`.
// The i'th bit is `true` iff the candidate has been approved in the context of this
// block. The block can be considered approved if the bitfield has all bits set to `true`.
approved_bitfield: BitVec<BitOrderLsb0, u8>,
children: Vec<Hash>,
}
impl BlockEntry {
/// Mark a candidate as fully approved in the bitfield.
pub fn mark_approved_by_hash(&mut self, candidate_hash: &CandidateHash) {
if let Some(p) = self.candidates.iter().position(|(_, h)| h == candidate_hash) {
self.approved_bitfield.set(p, true);
}
}
/// Whether the block entry is fully approved.
pub fn is_fully_approved(&self) -> bool {
self.approved_bitfield.all()
}
#[cfg(test)]
pub fn block_hash(&self) -> Hash {
self.block_hash
}
#[cfg(test)]
pub fn is_candidate_approved(&self, candidate_hash: &CandidateHash) -> bool {
self.candidates.iter().position(|(_, h)| h == candidate_hash)
.and_then(|p| self.approved_bitfield.get(p).map(|b| *b))
.unwrap_or(false)
}
/// For tests: Add a candidate to the block entry. Returns the
/// index where the candidate was added.
///
/// Panics if the core is already used.
#[cfg(test)]
pub fn add_candidate(&mut self, core: CoreIndex, candidate_hash: CandidateHash) -> usize {
let pos = self.candidates
.binary_search_by_key(&core, |(c, _)| *c)
.unwrap_err();
self.candidates.insert(pos, (core, candidate_hash));
// bug in bitvec?
if pos < self.approved_bitfield.len() {
self.approved_bitfield.insert(pos, false);
} else {
self.approved_bitfield.push(false);
}
pos
}
/// Get the slot of the block.
pub fn slot(&self) -> Slot {
self.slot
}
/// Get the relay-vrf-story of the block.
pub fn relay_vrf_story(&self) -> RelayVRFStory {
self.relay_vrf_story.clone()
}
/// Get the session index of the block.
pub fn session(&self) -> SessionIndex {
self.session
}
/// Get the i'th candidate.
pub fn candidate(&self, i: usize) -> Option<&(CoreIndex, CandidateHash)> {
self.candidates.get(i)
}
/// Access the underlying candidates as a slice.
pub fn candidates(&self) -> &[(CoreIndex, CandidateHash)] {
&self.candidates
}
}
impl From<crate::approval_db::v1::BlockEntry> for BlockEntry {
fn from(entry: crate::approval_db::v1::BlockEntry) -> Self {
BlockEntry {
block_hash: entry.block_hash,
session: entry.session,
slot: entry.slot,
relay_vrf_story: RelayVRFStory(entry.relay_vrf_story),
candidates: entry.candidates,
approved_bitfield: entry.approved_bitfield,
children: entry.children,
}
}
}
impl From<BlockEntry> for crate::approval_db::v1::BlockEntry {
fn from(entry: BlockEntry) -> Self {
Self {
block_hash: entry.block_hash,
session: entry.session,
slot: entry.slot,
relay_vrf_story: entry.relay_vrf_story.0,
candidates: entry.candidates,
approved_bitfield: entry.approved_bitfield,
children: entry.children,
}
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,88 @@
// Copyright 2021 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
//! Time utilities for approval voting.
use polkadot_node_primitives::approval::DelayTranche;
use sp_consensus_slots::Slot;
use futures::prelude::*;
use std::time::{Duration, SystemTime};
use std::pin::Pin;
const TICK_DURATION_MILLIS: u64 = 500;
/// A base unit of time, starting from the unix epoch, split into half-second intervals.
pub(crate) type Tick = u64;
/// A clock which allows querying of the current tick as well as
/// waiting for a tick to be reached.
pub(crate) trait Clock {
/// Yields the current tick.
fn tick_now(&self) -> Tick;
/// Yields a future which concludes when the given tick is reached.
fn wait(&self, tick: Tick) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>>;
}
/// Extension methods for clocks.
pub(crate) trait ClockExt {
fn tranche_now(&self, slot_duration_millis: u64, base_slot: Slot) -> DelayTranche;
}
impl<C: Clock + ?Sized> ClockExt for C {
fn tranche_now(&self, slot_duration_millis: u64, base_slot: Slot) -> DelayTranche {
self.tick_now()
.saturating_sub(slot_number_to_tick(slot_duration_millis, base_slot)) as u32
}
}
/// A clock which uses the actual underlying system clock.
pub(crate) struct SystemClock;
impl Clock for SystemClock {
/// Yields the current tick.
fn tick_now(&self) -> Tick {
match SystemTime::now().duration_since(SystemTime::UNIX_EPOCH) {
Err(_) => 0,
Ok(d) => d.as_millis() as u64 / TICK_DURATION_MILLIS,
}
}
/// Yields a future which concludes when the given tick is reached.
fn wait(&self, tick: Tick) -> Pin<Box<dyn Future<Output = ()> + Send>> {
let fut = async move {
let now = SystemTime::now();
let tick_onset = tick_to_time(tick);
if now < tick_onset {
if let Some(until) = tick_onset.duration_since(now).ok() {
futures_timer::Delay::new(until).await;
}
}
};
Box::pin(fut)
}
}
fn tick_to_time(tick: Tick) -> SystemTime {
SystemTime::UNIX_EPOCH + Duration::from_millis(TICK_DURATION_MILLIS * tick)
}
/// assumes `slot_duration_millis` evenly divided by tick duration.
pub(crate) fn slot_number_to_tick(slot_duration_millis: u64, slot: Slot) -> Tick {
let ticks_per_slot = slot_duration_millis / TICK_DURATION_MILLIS;
u64::from(slot) * ticks_per_slot
}