mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-13 12:51:05 +00:00
Minimal parachains part 2: Parachain statement and data routing (#173)
* dynamic inclusion threshold calculator * collators interface * collation helpers * initial proposal-creation future * create proposer when asked to propose * remove local_availability duty * statement table tracks includable parachain count * beginnings of timing future * finish proposal logic * remove stray println * extract shared table to separate module * change ordering * includability tracking * fix doc * initial changes to parachains module * initialise dummy block before API calls * give polkadot control over round proposer based on random seed * propose only after enough candidates * flesh out parachains module a bit more * set_heads * actually introduce set_heads to runtime * update block_builder to accept parachains * split block validity errors from real errors in evaluation * update WASM runtimes * polkadot-api methods for parachains additions * delay evaluation until candidates are ready * comments * fix dynamic inclusion with zero initial * test for includability tracker * wasm validation of parachain candidates * move primitives to primitives crate * remove runtime-std dependency from codec * adjust doc * polkadot-parachain-primitives * kill legacy polkadot-validator crate * basic-add test chain * test for basic_add parachain * move to test-chains dir * use wasm-build * new wasm directory layout * reorganize a bit more * Fix for rh-minimal-parachain (#141) * Remove extern "C" We already encountered such behavior (bug?) in pwasm-std, I believe. * Fix `panic_fmt` signature by adding `_col` Wrong `panic_fmt` signature can inhibit some optimizations in LTO mode. * Add linker flags and use wasm-gc in build script Pass --import-memory to LLD to emit wasm binary with imported memory. Also use wasm-gc instead of wasm-build. * Fix effective_max. I'm not sure why it was the way it was actually. * Recompile wasm. * Fix indent * more basic_add tests * validate parachain WASM * produce statements on receiving statements * tests for reactive statement production * fix build * add OOM lang item to runtime-io * use dynamic_inclusion when evaluating as well * fix update_includable_count * remove dead code * grumbles * actually defer round_proposer logic * update wasm * address a few more grumbles * schedule collation work as soon as BFT is started * impl future in collator * fix comment * governance proposals for adding and removing parachains * bump protocol version * tear out polkadot-specific pieces of substrate-network * extract out polkadot-specific stuff from substrate-network * begin polkadot network subsystem * grumbles * update WASM checkins * parse status from polkadot peer * allow invoke of network specialization * begin statement router implementation * remove dependency on tokio-timer * fix sanity check and have proposer factory create communication streams * pull out statement routing from consensus library * fix comments * adjust typedefs * extract consensus_gossip out of main network protocol handler * port substrate-bft to new tokio * port polkadot-consensus to new tokio * fix typo * start message processing task * initial consensus network implementation * remove known tracking from statement-table crate * extract router into separate module * defer statements until later * double signature is invalid * propagating statements * grumbles * request block data * fix compilation * embed new consensus network into service * port demo CLI to tokio * all test crates compile * some tests for fetching block data * whitespace * adjusting some tokio stuff * update exit-future * remove overly noisy warning * clean up collation work a bit * address review grumbles * fix lock order in protocol handler * rebuild wasm artifacts * tag AuthorityId::from_slice for std only * address formatting grumbles * rename event_loop to executor * some more docs for polkadot-network crate
This commit is contained in:
Robert Habermeier
committed by
GitHub
GitHub
parent
5355956314
commit
be5ff4e62f
@@ -0,0 +1,313 @@
|
||||
// Copyright 2017 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! The "consensus" networking code built on top of the base network service.
|
||||
//! This fulfills the `polkadot_consensus::Network` trait, providing a hook to be called
|
||||
//! each time consensus begins on a new chain head.
|
||||
|
||||
use bft;
|
||||
use ed25519;
|
||||
use substrate_network::{self as net, generic_message as msg};
|
||||
use substrate_network::consensus_gossip::ConsensusMessage;
|
||||
use polkadot_api::{PolkadotApi, LocalPolkadotApi};
|
||||
use polkadot_consensus::{Network, SharedTable, Collators, Collation};
|
||||
use polkadot_primitives::{AccountId, Block, Hash, SessionKey};
|
||||
use polkadot_primitives::parachain::Id as ParaId;
|
||||
|
||||
use futures::{future, prelude::*};
|
||||
use futures::sync::mpsc;
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use tokio::runtime::TaskExecutor;
|
||||
use parking_lot::Mutex;
|
||||
|
||||
use super::{Message, NetworkService, Knowledge, CurrentConsensus};
|
||||
use router::Router;
|
||||
|
||||
/// Sink for output BFT messages.
|
||||
pub struct BftSink<E> {
|
||||
network: Arc<NetworkService>,
|
||||
parent_hash: Hash,
|
||||
_marker: ::std::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<E> Sink for BftSink<E> {
|
||||
type SinkItem = bft::Communication<Block>;
|
||||
// TODO: replace this with the ! type when that's stabilized
|
||||
type SinkError = E;
|
||||
|
||||
fn start_send(&mut self, message: bft::Communication<Block>) -> ::futures::StartSend<bft::Communication<Block>, E> {
|
||||
let network_message = net::LocalizedBftMessage {
|
||||
message: match message {
|
||||
bft::generic::Communication::Consensus(c) => msg::BftMessage::Consensus(match c {
|
||||
bft::generic::LocalizedMessage::Propose(proposal) => msg::SignedConsensusMessage::Propose(msg::SignedConsensusProposal {
|
||||
round_number: proposal.round_number as u32,
|
||||
proposal: proposal.proposal,
|
||||
digest: proposal.digest,
|
||||
sender: proposal.sender,
|
||||
digest_signature: proposal.digest_signature.signature,
|
||||
full_signature: proposal.full_signature.signature,
|
||||
}),
|
||||
bft::generic::LocalizedMessage::Vote(vote) => msg::SignedConsensusMessage::Vote(msg::SignedConsensusVote {
|
||||
sender: vote.sender,
|
||||
signature: vote.signature.signature,
|
||||
vote: match vote.vote {
|
||||
bft::generic::Vote::Prepare(r, h) => msg::ConsensusVote::Prepare(r as u32, h),
|
||||
bft::generic::Vote::Commit(r, h) => msg::ConsensusVote::Commit(r as u32, h),
|
||||
bft::generic::Vote::AdvanceRound(r) => msg::ConsensusVote::AdvanceRound(r as u32),
|
||||
}
|
||||
}),
|
||||
}),
|
||||
bft::generic::Communication::Auxiliary(justification) => msg::BftMessage::Auxiliary(justification.uncheck().into()),
|
||||
},
|
||||
parent_hash: self.parent_hash,
|
||||
};
|
||||
self.network.with_spec(
|
||||
move |spec, ctx| spec.consensus_gossip.multicast_bft_message(ctx, network_message)
|
||||
);
|
||||
Ok(::futures::AsyncSink::Ready)
|
||||
}
|
||||
|
||||
fn poll_complete(&mut self) -> ::futures::Poll<(), E> {
|
||||
Ok(Async::Ready(()))
|
||||
}
|
||||
}
|
||||
|
||||
// check signature and authority validity of message.
|
||||
fn process_bft_message(msg: msg::LocalizedBftMessage<Block, Hash>, local_id: &SessionKey, authorities: &[SessionKey]) -> Result<Option<bft::Communication<Block>>, bft::Error> {
|
||||
Ok(Some(match msg.message {
|
||||
msg::BftMessage::Consensus(c) => bft::generic::Communication::Consensus(match c {
|
||||
msg::SignedConsensusMessage::Propose(proposal) => bft::generic::LocalizedMessage::Propose({
|
||||
if &proposal.sender == local_id { return Ok(None) }
|
||||
let proposal = bft::generic::LocalizedProposal {
|
||||
round_number: proposal.round_number as usize,
|
||||
proposal: proposal.proposal,
|
||||
digest: proposal.digest,
|
||||
sender: proposal.sender,
|
||||
digest_signature: ed25519::LocalizedSignature {
|
||||
signature: proposal.digest_signature,
|
||||
signer: ed25519::Public(proposal.sender.into()),
|
||||
},
|
||||
full_signature: ed25519::LocalizedSignature {
|
||||
signature: proposal.full_signature,
|
||||
signer: ed25519::Public(proposal.sender.into()),
|
||||
}
|
||||
};
|
||||
bft::check_proposal(authorities, &msg.parent_hash, &proposal)?;
|
||||
|
||||
trace!(target: "bft", "importing proposal message for round {} from {}", proposal.round_number, Hash::from(proposal.sender.0));
|
||||
proposal
|
||||
}),
|
||||
msg::SignedConsensusMessage::Vote(vote) => bft::generic::LocalizedMessage::Vote({
|
||||
if &vote.sender == local_id { return Ok(None) }
|
||||
let vote = bft::generic::LocalizedVote {
|
||||
sender: vote.sender,
|
||||
signature: ed25519::LocalizedSignature {
|
||||
signature: vote.signature,
|
||||
signer: ed25519::Public(vote.sender.0),
|
||||
},
|
||||
vote: match vote.vote {
|
||||
msg::ConsensusVote::Prepare(r, h) => bft::generic::Vote::Prepare(r as usize, h),
|
||||
msg::ConsensusVote::Commit(r, h) => bft::generic::Vote::Commit(r as usize, h),
|
||||
msg::ConsensusVote::AdvanceRound(r) => bft::generic::Vote::AdvanceRound(r as usize),
|
||||
}
|
||||
};
|
||||
bft::check_vote::<Block>(authorities, &msg.parent_hash, &vote)?;
|
||||
|
||||
trace!(target: "bft", "importing vote {:?} from {}", vote.vote, Hash::from(vote.sender.0));
|
||||
vote
|
||||
}),
|
||||
}),
|
||||
msg::BftMessage::Auxiliary(a) => {
|
||||
let justification = bft::UncheckedJustification::from(a);
|
||||
// TODO: get proper error
|
||||
let justification: Result<_, bft::Error> = bft::check_prepare_justification::<Block>(authorities, msg.parent_hash, justification)
|
||||
.map_err(|_| bft::ErrorKind::InvalidJustification.into());
|
||||
bft::generic::Communication::Auxiliary(justification?)
|
||||
},
|
||||
}))
|
||||
}
|
||||
|
||||
// task that processes all gossipped consensus messages,
|
||||
// checking signatures
|
||||
struct MessageProcessTask<P: PolkadotApi> {
|
||||
inner_stream: mpsc::UnboundedReceiver<ConsensusMessage<Block>>,
|
||||
bft_messages: mpsc::UnboundedSender<bft::Communication<Block>>,
|
||||
validators: Vec<SessionKey>,
|
||||
table_router: Router<P>,
|
||||
}
|
||||
|
||||
impl<P: LocalPolkadotApi + Send + Sync + 'static> MessageProcessTask<P> {
|
||||
fn process_message(&self, msg: ConsensusMessage<Block>) -> Option<Async<()>> {
|
||||
match msg {
|
||||
ConsensusMessage::Bft(msg) => {
|
||||
let local_id = self.table_router.session_key();
|
||||
match process_bft_message(msg, &local_id, &self.validators[..]) {
|
||||
Ok(Some(msg)) => {
|
||||
if let Err(_) = self.bft_messages.unbounded_send(msg) {
|
||||
// if the BFT receiving stream has ended then
|
||||
// we should just bail.
|
||||
trace!(target: "bft", "BFT message stream appears to have closed");
|
||||
return Some(Async::Ready(()));
|
||||
}
|
||||
}
|
||||
Ok(None) => {} // ignored local message
|
||||
Err(e) => {
|
||||
debug!("Message validation failed: {:?}", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
ConsensusMessage::ChainSpecific(msg, _) => {
|
||||
if let Ok(Message::Statement(parent_hash, statement)) = ::serde_json::from_slice(&msg) {
|
||||
if ::polkadot_consensus::check_statement(&statement.statement, &statement.signature, statement.sender, &parent_hash) {
|
||||
self.table_router.import_statement(statement);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
impl<P: LocalPolkadotApi + Send + Sync + 'static> Future for MessageProcessTask<P> {
|
||||
type Item = ();
|
||||
type Error = ();
|
||||
|
||||
fn poll(&mut self) -> Poll<(), ()> {
|
||||
loop {
|
||||
match self.inner_stream.poll() {
|
||||
Ok(Async::Ready(Some(val))) => if let Some(async) = self.process_message(val) {
|
||||
return Ok(async);
|
||||
},
|
||||
Ok(Async::Ready(None)) => return Ok(Async::Ready(())),
|
||||
Ok(Async::NotReady) => (),
|
||||
Err(e) => debug!(target: "p_net", "Error getting consensus message: {:?}", e),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Input stream from the consensus network.
|
||||
pub struct InputAdapter {
|
||||
input: mpsc::UnboundedReceiver<bft::Communication<Block>>,
|
||||
}
|
||||
|
||||
impl Stream for InputAdapter {
|
||||
type Item = bft::Communication<Block>;
|
||||
type Error = ::polkadot_consensus::Error;
|
||||
|
||||
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
|
||||
match self.input.poll() {
|
||||
Err(_) | Ok(Async::Ready(None)) => Err(bft::InputStreamConcluded.into()),
|
||||
Ok(x) => Ok(x)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Wrapper around the network service
|
||||
pub struct ConsensusNetwork<P> {
|
||||
network: Arc<NetworkService>,
|
||||
api: Arc<P>,
|
||||
}
|
||||
|
||||
impl<P> ConsensusNetwork<P> {
|
||||
/// Create a new consensus networking object.
|
||||
pub fn new(network: Arc<NetworkService>, api: Arc<P>) -> Self {
|
||||
ConsensusNetwork { network, api }
|
||||
}
|
||||
}
|
||||
|
||||
impl<P> Clone for ConsensusNetwork<P> {
|
||||
fn clone(&self) -> Self {
|
||||
ConsensusNetwork {
|
||||
network: self.network.clone(),
|
||||
api: self.api.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A long-lived network which can create parachain statement and BFT message routing processes on demand.
|
||||
impl<P: LocalPolkadotApi + Send + Sync + 'static> Network for ConsensusNetwork<P> {
|
||||
type TableRouter = Router<P>;
|
||||
/// The input stream of BFT messages. Should never logically conclude.
|
||||
type Input = InputAdapter;
|
||||
/// The output sink of BFT messages. Messages sent here should eventually pass to all
|
||||
/// current validators.
|
||||
type Output = BftSink<::polkadot_consensus::Error>;
|
||||
|
||||
/// Instantiate a table router using the given shared table.
|
||||
fn communication_for(&self, validators: &[SessionKey], table: Arc<SharedTable>, task_executor: TaskExecutor) -> (Self::TableRouter, Self::Input, Self::Output) {
|
||||
let parent_hash = table.consensus_parent_hash().clone();
|
||||
|
||||
let sink = BftSink {
|
||||
network: self.network.clone(),
|
||||
parent_hash,
|
||||
_marker: Default::default(),
|
||||
};
|
||||
|
||||
let (bft_send, bft_recv) = mpsc::unbounded();
|
||||
|
||||
let knowledge = Arc::new(Mutex::new(Knowledge::new()));
|
||||
|
||||
let local_session_key = table.session_key();
|
||||
let table_router = Router::new(
|
||||
table,
|
||||
self.network.clone(),
|
||||
self.api.clone(),
|
||||
task_executor.clone(),
|
||||
parent_hash,
|
||||
knowledge.clone(),
|
||||
);
|
||||
|
||||
// spin up a task in the background that processes all incoming statements
|
||||
// TODO: propagate statements on a timer?
|
||||
let process_task = self.network.with_spec(|spec, ctx| {
|
||||
spec.new_consensus(ctx, CurrentConsensus {
|
||||
knowledge,
|
||||
parent_hash,
|
||||
local_session_key,
|
||||
session_keys: Default::default(),
|
||||
});
|
||||
|
||||
MessageProcessTask {
|
||||
inner_stream: spec.consensus_gossip.messages_for(parent_hash),
|
||||
bft_messages: bft_send,
|
||||
validators: validators.to_vec(),
|
||||
table_router: table_router.clone(),
|
||||
}
|
||||
});
|
||||
|
||||
match process_task {
|
||||
Some(task) => task_executor.spawn(task),
|
||||
None => warn!(target: "p_net", "Cannot process incoming messages: network appears to be down"),
|
||||
}
|
||||
|
||||
(table_router, InputAdapter { input: bft_recv }, sink)
|
||||
}
|
||||
}
|
||||
|
||||
impl<P: LocalPolkadotApi + Send + Sync + 'static> Collators for ConsensusNetwork<P> {
|
||||
type Error = ();
|
||||
type Collation = future::Empty<Collation, ()>;
|
||||
|
||||
fn collate(&self, _parachain: ParaId, _relay_parent: Hash) -> Self::Collation {
|
||||
future::empty()
|
||||
}
|
||||
|
||||
fn note_bad_collator(&self, _collator: AccountId) { }
|
||||
}
|
||||
@@ -0,0 +1,488 @@
|
||||
// Copyright 2017 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Polkadot-specific network implementation.
|
||||
//!
|
||||
//! This manages gossip of consensus messages for BFT and for parachain statements,
|
||||
//! parachain block and extrinsic data fetching, communication between collators and validators,
|
||||
//! and more.
|
||||
|
||||
extern crate serde;
|
||||
#[macro_use]
|
||||
extern crate serde_derive;
|
||||
extern crate serde_json;
|
||||
|
||||
extern crate substrate_bft as bft;
|
||||
extern crate substrate_codec as codec;
|
||||
extern crate substrate_network;
|
||||
extern crate substrate_primitives;
|
||||
|
||||
extern crate polkadot_api;
|
||||
extern crate polkadot_consensus;
|
||||
extern crate polkadot_primitives;
|
||||
|
||||
extern crate ed25519;
|
||||
extern crate futures;
|
||||
extern crate parking_lot;
|
||||
extern crate tokio;
|
||||
|
||||
#[macro_use]
|
||||
extern crate log;
|
||||
|
||||
mod router;
|
||||
pub mod consensus;
|
||||
|
||||
use codec::Slicable;
|
||||
use futures::sync::oneshot;
|
||||
use parking_lot::Mutex;
|
||||
use polkadot_consensus::{Statement, SignedStatement, GenericStatement};
|
||||
use polkadot_primitives::{Block, SessionKey, Hash};
|
||||
use polkadot_primitives::parachain::{Id as ParaId, BlockData, Extrinsic, CandidateReceipt};
|
||||
use substrate_network::{PeerId, RequestId, Context};
|
||||
use substrate_network::consensus_gossip::ConsensusGossip;
|
||||
use substrate_network::{message, generic_message};
|
||||
use substrate_network::specialization::Specialization;
|
||||
use substrate_network::StatusMessage as GenericFullStatus;
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::sync::Arc;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
|
||||
/// Polkadot protocol id.
|
||||
pub const DOT_PROTOCOL_ID: ::substrate_network::ProtocolId = *b"dot";
|
||||
|
||||
type FullStatus = GenericFullStatus<Block>;
|
||||
|
||||
/// Specialization of the network service for the polkadot protocol.
|
||||
pub type NetworkService = ::substrate_network::Service<Block, PolkadotProtocol>;
|
||||
|
||||
/// Status of a Polkadot node.
|
||||
#[derive(Debug, PartialEq, Eq, Clone)]
|
||||
pub struct Status {
|
||||
collating_for: Option<ParaId>,
|
||||
}
|
||||
|
||||
impl Slicable for Status {
|
||||
fn encode(&self) -> Vec<u8> {
|
||||
let mut v = Vec::new();
|
||||
match self.collating_for {
|
||||
Some(ref id) => {
|
||||
v.push(1);
|
||||
id.using_encoded(|s| v.extend(s));
|
||||
}
|
||||
None => {
|
||||
v.push(0);
|
||||
}
|
||||
}
|
||||
v
|
||||
}
|
||||
|
||||
fn decode<I: ::codec::Input>(input: &mut I) -> Option<Self> {
|
||||
let collating_for = match input.read_byte()? {
|
||||
0 => None,
|
||||
1 => Some(ParaId::decode(input)?),
|
||||
_ => return None,
|
||||
};
|
||||
Some(Status { collating_for })
|
||||
}
|
||||
}
|
||||
|
||||
struct BlockDataRequest {
|
||||
attempted_peers: HashSet<SessionKey>,
|
||||
consensus_parent: Hash,
|
||||
candidate_hash: Hash,
|
||||
block_data_hash: Hash,
|
||||
sender: oneshot::Sender<BlockData>,
|
||||
}
|
||||
|
||||
struct PeerInfo {
|
||||
status: Status,
|
||||
validator: bool,
|
||||
session_keys: HashMap<Hash, SessionKey>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct KnowledgeEntry {
|
||||
knows_block_data: Vec<SessionKey>,
|
||||
knows_extrinsic: Vec<SessionKey>,
|
||||
block_data: Option<BlockData>,
|
||||
extrinsic: Option<Extrinsic>,
|
||||
}
|
||||
|
||||
/// Tracks knowledge of peers.
|
||||
struct Knowledge {
|
||||
candidates: HashMap<Hash, KnowledgeEntry>,
|
||||
}
|
||||
|
||||
impl Knowledge {
|
||||
pub fn new() -> Self {
|
||||
Knowledge {
|
||||
candidates: HashMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn note_statement(&mut self, from: SessionKey, statement: &Statement) {
|
||||
match *statement {
|
||||
GenericStatement::Candidate(ref c) => {
|
||||
let mut entry = self.candidates.entry(c.hash()).or_insert_with(Default::default);
|
||||
entry.knows_block_data.push(from);
|
||||
entry.knows_extrinsic.push(from);
|
||||
}
|
||||
GenericStatement::Available(ref hash) => {
|
||||
let mut entry = self.candidates.entry(*hash).or_insert_with(Default::default);
|
||||
entry.knows_block_data.push(from);
|
||||
entry.knows_extrinsic.push(from);
|
||||
}
|
||||
GenericStatement::Valid(ref hash) | GenericStatement::Invalid(ref hash) => self.candidates.entry(*hash)
|
||||
.or_insert_with(Default::default)
|
||||
.knows_block_data
|
||||
.push(from),
|
||||
}
|
||||
}
|
||||
|
||||
fn note_candidate(&mut self, hash: Hash, block_data: Option<BlockData>, extrinsic: Option<Extrinsic>) {
|
||||
let entry = self.candidates.entry(hash).or_insert_with(Default::default);
|
||||
entry.block_data = entry.block_data.take().or(block_data);
|
||||
entry.extrinsic = entry.extrinsic.take().or(extrinsic);
|
||||
}
|
||||
}
|
||||
|
||||
struct CurrentConsensus {
|
||||
knowledge: Arc<Mutex<Knowledge>>,
|
||||
parent_hash: Hash,
|
||||
session_keys: HashMap<SessionKey, PeerId>,
|
||||
local_session_key: SessionKey,
|
||||
}
|
||||
|
||||
impl CurrentConsensus {
|
||||
// get locally stored block data for a candidate.
|
||||
fn block_data(&self, hash: &Hash) -> Option<BlockData> {
|
||||
self.knowledge.lock().candidates.get(hash)
|
||||
.and_then(|entry| entry.block_data.clone())
|
||||
}
|
||||
|
||||
fn peer_disconnected(&mut self, peer: &PeerInfo) {
|
||||
if let Some(key) = peer.session_keys.get(&self.parent_hash) {
|
||||
self.session_keys.remove(key);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Polkadot-specific messages.
|
||||
#[derive(Serialize, Deserialize)]
|
||||
pub enum Message {
|
||||
/// signed statement and localized parent hash.
|
||||
Statement(Hash, SignedStatement),
|
||||
/// Tell the peer your session key for the current block.
|
||||
// TODO: do this with a random challenge protocol
|
||||
SessionKey(Hash, SessionKey),
|
||||
/// Requesting parachain block data by candidate hash.
|
||||
RequestBlockData(RequestId, Hash),
|
||||
/// Provide block data by candidate hash or nothing if unknown.
|
||||
BlockData(RequestId, Option<BlockData>),
|
||||
}
|
||||
|
||||
fn send_polkadot_message(ctx: &mut Context<Block>, to: PeerId, message: Message) {
|
||||
let encoded = ::serde_json::to_vec(&message).expect("serialization of messages infallible; qed");
|
||||
ctx.send_message(to, generic_message::Message::ChainSpecific(encoded))
|
||||
}
|
||||
|
||||
/// Polkadot protocol attachment for substrate.
|
||||
pub struct PolkadotProtocol {
|
||||
peers: HashMap<PeerId, PeerInfo>,
|
||||
consensus_gossip: ConsensusGossip<Block>,
|
||||
collators: HashMap<ParaId, Vec<PeerId>>,
|
||||
collating_for: Option<ParaId>,
|
||||
live_consensus: Option<CurrentConsensus>,
|
||||
in_flight: HashMap<(RequestId, PeerId), BlockDataRequest>,
|
||||
pending: Vec<BlockDataRequest>,
|
||||
next_req_id: u64,
|
||||
}
|
||||
|
||||
impl PolkadotProtocol {
|
||||
/// Instantiate a polkadot protocol handler.
|
||||
pub fn new() -> Self {
|
||||
PolkadotProtocol {
|
||||
peers: HashMap::new(),
|
||||
consensus_gossip: ConsensusGossip::new(),
|
||||
collators: HashMap::new(),
|
||||
collating_for: None,
|
||||
live_consensus: None,
|
||||
in_flight: HashMap::new(),
|
||||
pending: Vec::new(),
|
||||
next_req_id: 1,
|
||||
}
|
||||
}
|
||||
|
||||
/// Send a statement to a validator.
|
||||
fn send_statement(&mut self, ctx: &mut Context<Block>, _val: SessionKey, parent_hash: Hash, statement: SignedStatement) {
|
||||
// TODO: something more targeted than gossip.
|
||||
let raw = ::serde_json::to_vec(&Message::Statement(parent_hash, statement))
|
||||
.expect("message serialization infallible; qed");
|
||||
|
||||
self.consensus_gossip.multicast_chain_specific(ctx, raw, parent_hash);
|
||||
}
|
||||
|
||||
/// Fetch block data by candidate receipt.
|
||||
fn fetch_block_data(&mut self, ctx: &mut Context<Block>, candidate: &CandidateReceipt, relay_parent: Hash) -> oneshot::Receiver<BlockData> {
|
||||
let (tx, rx) = oneshot::channel();
|
||||
|
||||
self.pending.push(BlockDataRequest {
|
||||
attempted_peers: Default::default(),
|
||||
consensus_parent: relay_parent,
|
||||
candidate_hash: candidate.hash(),
|
||||
block_data_hash: candidate.block_data_hash,
|
||||
sender: tx,
|
||||
});
|
||||
|
||||
self.dispatch_pending_requests(ctx);
|
||||
rx
|
||||
}
|
||||
|
||||
/// Note new consensus session.
|
||||
fn new_consensus(&mut self, ctx: &mut Context<Block>, mut consensus: CurrentConsensus) {
|
||||
let parent_hash = consensus.parent_hash;
|
||||
let old_parent = self.live_consensus.as_ref().map(|c| c.parent_hash);
|
||||
|
||||
for (id, info) in self.peers.iter_mut().filter(|&(_, ref info)| info.validator) {
|
||||
send_polkadot_message(
|
||||
ctx,
|
||||
*id,
|
||||
Message::SessionKey(parent_hash, consensus.local_session_key)
|
||||
);
|
||||
|
||||
if let Some(key) = info.session_keys.get(&parent_hash) {
|
||||
consensus.session_keys.insert(*key, *id);
|
||||
}
|
||||
|
||||
if let Some(ref old_parent) = old_parent {
|
||||
info.session_keys.remove(old_parent);
|
||||
}
|
||||
}
|
||||
|
||||
self.live_consensus = Some(consensus);
|
||||
self.consensus_gossip.collect_garbage(old_parent.as_ref());
|
||||
}
|
||||
|
||||
fn dispatch_pending_requests(&mut self, ctx: &mut Context<Block>) {
|
||||
let consensus = match self.live_consensus {
|
||||
Some(ref mut c) => c,
|
||||
None => {
|
||||
self.pending.clear();
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let knowledge = consensus.knowledge.lock();
|
||||
let mut new_pending = Vec::new();
|
||||
for mut pending in ::std::mem::replace(&mut self.pending, Vec::new()) {
|
||||
if pending.consensus_parent != consensus.parent_hash { continue }
|
||||
|
||||
if let Some(entry) = knowledge.candidates.get(&pending.candidate_hash) {
|
||||
// answer locally
|
||||
if let Some(ref data) = entry.block_data {
|
||||
let _ = pending.sender.send(data.clone());
|
||||
continue;
|
||||
}
|
||||
|
||||
let next_peer = entry.knows_block_data.iter()
|
||||
.filter_map(|x| consensus.session_keys.get(x).map(|id| (*x, *id)))
|
||||
.find(|&(ref key, _)| pending.attempted_peers.insert(*key))
|
||||
.map(|(_, id)| id);
|
||||
|
||||
// dispatch to peer
|
||||
if let Some(peer_id) = next_peer {
|
||||
let req_id = self.next_req_id;
|
||||
self.next_req_id += 1;
|
||||
|
||||
send_polkadot_message(
|
||||
ctx,
|
||||
peer_id,
|
||||
Message::RequestBlockData(req_id, pending.candidate_hash)
|
||||
);
|
||||
|
||||
self.in_flight.insert((req_id, peer_id), pending);
|
||||
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
new_pending.push(pending);
|
||||
}
|
||||
|
||||
self.pending = new_pending;
|
||||
}
|
||||
|
||||
fn on_polkadot_message(&mut self, ctx: &mut Context<Block>, peer_id: PeerId, raw: Vec<u8>, msg: Message) {
|
||||
match msg {
|
||||
Message::Statement(parent_hash, _statement) =>
|
||||
self.consensus_gossip.on_chain_specific(ctx, peer_id, raw, parent_hash),
|
||||
Message::SessionKey(parent_hash, key) => {
|
||||
{
|
||||
let info = match self.peers.get_mut(&peer_id) {
|
||||
Some(peer) => peer,
|
||||
None => return,
|
||||
};
|
||||
|
||||
if !info.validator {
|
||||
ctx.disable_peer(peer_id);
|
||||
return;
|
||||
}
|
||||
|
||||
match self.live_consensus {
|
||||
Some(ref mut consensus) if consensus.parent_hash == parent_hash => {
|
||||
consensus.session_keys.insert(key, peer_id);
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
info.session_keys.insert(parent_hash, key);
|
||||
}
|
||||
self.dispatch_pending_requests(ctx);
|
||||
}
|
||||
Message::RequestBlockData(req_id, hash) => {
|
||||
let block_data = self.live_consensus.as_ref()
|
||||
.and_then(|c| c.block_data(&hash));
|
||||
|
||||
send_polkadot_message(ctx, peer_id, Message::BlockData(req_id, block_data));
|
||||
}
|
||||
Message::BlockData(req_id, data) => self.on_block_data(ctx, peer_id, req_id, data),
|
||||
}
|
||||
}
|
||||
|
||||
fn on_block_data(&mut self, ctx: &mut Context<Block>, peer_id: PeerId, req_id: RequestId, data: Option<BlockData>) {
|
||||
match self.in_flight.remove(&(req_id, peer_id)) {
|
||||
Some(req) => {
|
||||
if let Some(data) = data {
|
||||
if data.hash() == req.block_data_hash {
|
||||
let _ = req.sender.send(data);
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
self.pending.push(req);
|
||||
self.dispatch_pending_requests(ctx);
|
||||
}
|
||||
None => ctx.disable_peer(peer_id),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Specialization<Block> for PolkadotProtocol {
|
||||
fn status(&self) -> Vec<u8> {
|
||||
Status { collating_for: self.collating_for.clone() }.encode()
|
||||
}
|
||||
|
||||
fn on_connect(&mut self, ctx: &mut Context<Block>, peer_id: PeerId, status: FullStatus) {
|
||||
let local_status = match Status::decode(&mut &status.chain_status[..]) {
|
||||
Some(status) => status,
|
||||
None => {
|
||||
ctx.disable_peer(peer_id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
if let Some(ref para_id) = local_status.collating_for {
|
||||
self.collators.entry(para_id.clone())
|
||||
.or_insert_with(Vec::new)
|
||||
.push(peer_id);
|
||||
}
|
||||
|
||||
let validator = status.roles.iter().any(|r| *r == message::Role::Authority);
|
||||
self.peers.insert(peer_id, PeerInfo {
|
||||
status: local_status,
|
||||
session_keys: Default::default(),
|
||||
validator,
|
||||
});
|
||||
|
||||
self.consensus_gossip.new_peer(ctx, peer_id, &status.roles);
|
||||
|
||||
if let (true, &Some(ref consensus)) = (validator, &self.live_consensus) {
|
||||
send_polkadot_message(
|
||||
ctx,
|
||||
peer_id,
|
||||
Message::SessionKey(consensus.parent_hash, consensus.local_session_key)
|
||||
);
|
||||
}
|
||||
|
||||
self.dispatch_pending_requests(ctx);
|
||||
}
|
||||
|
||||
fn on_disconnect(&mut self, ctx: &mut Context<Block>, peer_id: PeerId) {
|
||||
if let Some(info) = self.peers.remove(&peer_id) {
|
||||
if let Some(collators) = info.status.collating_for.and_then(|id| self.collators.get_mut(&id)) {
|
||||
if let Some(pos) = collators.iter().position(|x| x == &peer_id) {
|
||||
collators.swap_remove(pos);
|
||||
}
|
||||
}
|
||||
|
||||
if let (true, &mut Some(ref mut consensus)) = (info.validator, &mut self.live_consensus) {
|
||||
consensus.peer_disconnected(&info);
|
||||
}
|
||||
|
||||
{
|
||||
let pending = &mut self.pending;
|
||||
self.in_flight.retain(|&(_, ref peer), val| {
|
||||
let retain = peer != &peer_id;
|
||||
if !retain {
|
||||
let (sender, _) = oneshot::channel();
|
||||
pending.push(::std::mem::replace(val, BlockDataRequest {
|
||||
attempted_peers: Default::default(),
|
||||
consensus_parent: Default::default(),
|
||||
candidate_hash: Default::default(),
|
||||
block_data_hash: Default::default(),
|
||||
sender,
|
||||
}));
|
||||
}
|
||||
|
||||
retain
|
||||
});
|
||||
}
|
||||
self.consensus_gossip.peer_disconnected(ctx, peer_id);
|
||||
self.dispatch_pending_requests(ctx);
|
||||
}
|
||||
}
|
||||
|
||||
fn on_message(&mut self, ctx: &mut Context<Block>, peer_id: PeerId, message: message::Message<Block>) {
|
||||
match message {
|
||||
generic_message::Message::BftMessage(msg) => {
|
||||
// TODO: check signature here? what if relevant block is unknown?
|
||||
self.consensus_gossip.on_bft_message(ctx, peer_id, msg)
|
||||
}
|
||||
generic_message::Message::ChainSpecific(raw) => {
|
||||
match serde_json::from_slice(&raw) {
|
||||
Ok(msg) => self.on_polkadot_message(ctx, peer_id, raw, msg),
|
||||
Err(e) => {
|
||||
trace!(target: "p_net", "Bad message from {}: {}", peer_id, e);
|
||||
ctx.disable_peer(peer_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
fn on_abort(&mut self) {
|
||||
self.consensus_gossip.abort();
|
||||
}
|
||||
|
||||
fn maintain_peers(&mut self, ctx: &mut Context<Block>) {
|
||||
self.consensus_gossip.collect_garbage(None);
|
||||
self.dispatch_pending_requests(ctx);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,349 @@
|
||||
// Copyright 2017 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Statement routing and consensus table router implementation.
|
||||
//!
|
||||
//! During the consensus process, validators exchange statements on validity and availability
|
||||
//! of parachain candidates.
|
||||
//! The `Router` in this file hooks into the underlying network to fulfill
|
||||
//! the `TableRouter` trait from `polkadot-consensus`, which is expected to call into a shared statement table
|
||||
//! and dispatch evaluation work as necessary when new statements come in.
|
||||
|
||||
use polkadot_api::{PolkadotApi, LocalPolkadotApi};
|
||||
use polkadot_consensus::{SharedTable, TableRouter, SignedStatement, GenericStatement, StatementProducer};
|
||||
use polkadot_primitives::{Hash, BlockId, SessionKey};
|
||||
use polkadot_primitives::parachain::{BlockData, Extrinsic, CandidateReceipt, Id as ParaId};
|
||||
|
||||
use futures::prelude::*;
|
||||
use tokio::runtime::TaskExecutor;
|
||||
use parking_lot::Mutex;
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::sync::Arc;
|
||||
|
||||
use super::{NetworkService, Knowledge};
|
||||
|
||||
/// Table routing implementation.
|
||||
pub struct Router<P: PolkadotApi> {
|
||||
table: Arc<SharedTable>,
|
||||
network: Arc<NetworkService>,
|
||||
api: Arc<P>,
|
||||
task_executor: TaskExecutor,
|
||||
parent_hash: Hash,
|
||||
knowledge: Arc<Mutex<Knowledge>>,
|
||||
deferred_statements: Arc<Mutex<DeferredStatements>>,
|
||||
}
|
||||
|
||||
impl<P: PolkadotApi> Router<P> {
|
||||
pub(crate) fn new(
|
||||
table: Arc<SharedTable>,
|
||||
network: Arc<NetworkService>,
|
||||
api: Arc<P>,
|
||||
task_executor: TaskExecutor,
|
||||
parent_hash: Hash,
|
||||
knowledge: Arc<Mutex<Knowledge>>,
|
||||
) -> Self {
|
||||
Router {
|
||||
table,
|
||||
network,
|
||||
api,
|
||||
task_executor,
|
||||
parent_hash,
|
||||
knowledge,
|
||||
deferred_statements: Arc::new(Mutex::new(DeferredStatements::new())),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn session_key(&self) -> SessionKey {
|
||||
self.table.session_key()
|
||||
}
|
||||
}
|
||||
|
||||
impl<P: PolkadotApi> Clone for Router<P> {
|
||||
fn clone(&self) -> Self {
|
||||
Router {
|
||||
table: self.table.clone(),
|
||||
network: self.network.clone(),
|
||||
api: self.api.clone(),
|
||||
task_executor: self.task_executor.clone(),
|
||||
parent_hash: self.parent_hash.clone(),
|
||||
deferred_statements: self.deferred_statements.clone(),
|
||||
knowledge: self.knowledge.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<P: LocalPolkadotApi + Send + Sync + 'static> Router<P> {
|
||||
/// Import a statement whose signature has been checked already.
|
||||
pub(crate) fn import_statement(&self, statement: SignedStatement) {
|
||||
// defer any statements for which we haven't imported the candidate yet
|
||||
let (c_hash, parachain_index) = {
|
||||
let candidate_data = match statement.statement {
|
||||
GenericStatement::Candidate(ref c) => Some((c.hash(), c.parachain_index)),
|
||||
GenericStatement::Valid(ref hash)
|
||||
| GenericStatement::Invalid(ref hash)
|
||||
| GenericStatement::Available(ref hash)
|
||||
=> self.table.with_candidate(hash, |c| c.map(|c| (*hash, c.parachain_index))),
|
||||
};
|
||||
match candidate_data {
|
||||
Some(x) => x,
|
||||
None => {
|
||||
self.deferred_statements.lock().push(statement);
|
||||
return;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// import all statements pending on this candidate
|
||||
let (mut statements, _traces) = if let GenericStatement::Candidate(_) = statement.statement {
|
||||
self.deferred_statements.lock().get_deferred(&c_hash)
|
||||
} else {
|
||||
(Vec::new(), Vec::new())
|
||||
};
|
||||
|
||||
// prepend the candidate statement.
|
||||
statements.insert(0, statement);
|
||||
let producers: Vec<_> = self.table.import_remote_statements(
|
||||
self,
|
||||
statements.iter().cloned(),
|
||||
);
|
||||
// dispatch future work as necessary.
|
||||
for (producer, statement) in producers.into_iter().zip(statements) {
|
||||
let producer = match producer {
|
||||
Some(p) => p,
|
||||
None => continue, // statement redundant
|
||||
};
|
||||
|
||||
self.knowledge.lock().note_statement(statement.sender, &statement.statement);
|
||||
self.dispatch_work(c_hash, producer, parachain_index);
|
||||
}
|
||||
}
|
||||
|
||||
fn dispatch_work<D, E>(&self, candidate_hash: Hash, producer: StatementProducer<D, E>, parachain: ParaId) where
|
||||
D: Future<Item=BlockData,Error=()> + Send + 'static,
|
||||
E: Future<Item=Extrinsic,Error=()> + Send + 'static,
|
||||
{
|
||||
let parent_hash = self.parent_hash.clone();
|
||||
|
||||
let api = self.api.clone();
|
||||
let validate = move |collation| -> Option<bool> {
|
||||
let id = BlockId::hash(parent_hash);
|
||||
match ::polkadot_consensus::validate_collation(&*api, &id, &collation) {
|
||||
Ok(()) => Some(true),
|
||||
Err(e) => {
|
||||
debug!(target: "p_net", "Encountered bad collation: {}", e);
|
||||
Some(false)
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
let table = self.table.clone();
|
||||
let network = self.network.clone();
|
||||
let knowledge = self.knowledge.clone();
|
||||
|
||||
let work = producer.prime(validate).map(move |produced| {
|
||||
// store the data before broadcasting statements, so other peers can fetch.
|
||||
knowledge.lock().note_candidate(candidate_hash, produced.block_data, produced.extrinsic);
|
||||
|
||||
// propagate the statements
|
||||
if let Some(validity) = produced.validity {
|
||||
let signed = table.sign_and_import(validity.clone());
|
||||
route_statement(&*network, &*table, parachain, parent_hash, signed);
|
||||
}
|
||||
|
||||
if let Some(availability) = produced.availability {
|
||||
let signed = table.sign_and_import(availability);
|
||||
route_statement(&*network, &*table, parachain, parent_hash, signed);
|
||||
}
|
||||
});
|
||||
|
||||
self.task_executor.spawn(work);
|
||||
}
|
||||
}
|
||||
|
||||
impl<P: LocalPolkadotApi + Send> TableRouter for Router<P> {
|
||||
type Error = ();
|
||||
type FetchCandidate = BlockDataReceiver;
|
||||
type FetchExtrinsic = Result<Extrinsic, Self::Error>;
|
||||
|
||||
fn local_candidate(&self, receipt: CandidateReceipt, block_data: BlockData, extrinsic: Extrinsic) {
|
||||
// give to network to make available.
|
||||
let hash = receipt.hash();
|
||||
let para_id = receipt.parachain_index;
|
||||
let signed = self.table.sign_and_import(GenericStatement::Candidate(receipt));
|
||||
|
||||
self.knowledge.lock().note_candidate(hash, Some(block_data), Some(extrinsic));
|
||||
route_statement(&*self.network, &*self.table, para_id, self.parent_hash, signed);
|
||||
}
|
||||
|
||||
fn fetch_block_data(&self, candidate: &CandidateReceipt) -> BlockDataReceiver {
|
||||
let parent_hash = self.parent_hash;
|
||||
let rx = self.network.with_spec(|spec, ctx| { spec.fetch_block_data(ctx, candidate, parent_hash) });
|
||||
BlockDataReceiver { inner: rx }
|
||||
}
|
||||
|
||||
fn fetch_extrinsic_data(&self, _candidate: &CandidateReceipt) -> Self::FetchExtrinsic {
|
||||
Ok(Extrinsic)
|
||||
}
|
||||
}
|
||||
|
||||
/// Receiver for block data.
|
||||
pub struct BlockDataReceiver {
|
||||
inner: Option<::futures::sync::oneshot::Receiver<BlockData>>,
|
||||
}
|
||||
|
||||
impl Future for BlockDataReceiver {
|
||||
type Item = BlockData;
|
||||
type Error = ();
|
||||
|
||||
fn poll(&mut self) -> Poll<BlockData, ()> {
|
||||
match self.inner {
|
||||
Some(ref mut inner) => inner.poll().map_err(|_| ()),
|
||||
None => return Err(()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// get statement to relevant validators.
|
||||
fn route_statement(network: &NetworkService, table: &SharedTable, para_id: ParaId, parent_hash: Hash, statement: SignedStatement) {
|
||||
let broadcast = |i: &mut Iterator<Item=&SessionKey>| {
|
||||
let local_key = table.session_key();
|
||||
network.with_spec(|spec, ctx| {
|
||||
for val in i.filter(|&x| x != &local_key) {
|
||||
spec.send_statement(ctx, *val, parent_hash, statement.clone());
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
let g_info = table
|
||||
.group_info()
|
||||
.get(¶_id)
|
||||
.expect("statements only produced about groups which exist");
|
||||
|
||||
match statement.statement {
|
||||
GenericStatement::Candidate(_) =>
|
||||
broadcast(&mut g_info.validity_guarantors.iter().chain(g_info.availability_guarantors.iter())),
|
||||
GenericStatement::Valid(_) | GenericStatement::Invalid(_) =>
|
||||
broadcast(&mut g_info.validity_guarantors.iter()),
|
||||
GenericStatement::Available(_) =>
|
||||
broadcast(&mut g_info.availability_guarantors.iter()),
|
||||
}
|
||||
}
|
||||
|
||||
// A unique trace for valid statements issued by a validator.
|
||||
#[derive(Hash, PartialEq, Eq, Clone, Debug)]
|
||||
enum StatementTrace {
|
||||
Valid(SessionKey, Hash),
|
||||
Invalid(SessionKey, Hash),
|
||||
Available(SessionKey, Hash),
|
||||
}
|
||||
|
||||
// helper for deferring statements whose associated candidate is unknown.
|
||||
struct DeferredStatements {
|
||||
deferred: HashMap<Hash, Vec<SignedStatement>>,
|
||||
known_traces: HashSet<StatementTrace>,
|
||||
}
|
||||
|
||||
impl DeferredStatements {
|
||||
fn new() -> Self {
|
||||
DeferredStatements {
|
||||
deferred: HashMap::new(),
|
||||
known_traces: HashSet::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn push(&mut self, statement: SignedStatement) {
|
||||
let (hash, trace) = match statement.statement {
|
||||
GenericStatement::Candidate(_) => return,
|
||||
GenericStatement::Valid(hash) => (hash, StatementTrace::Valid(statement.sender, hash)),
|
||||
GenericStatement::Invalid(hash) => (hash, StatementTrace::Invalid(statement.sender, hash)),
|
||||
GenericStatement::Available(hash) => (hash, StatementTrace::Available(statement.sender, hash)),
|
||||
};
|
||||
|
||||
if self.known_traces.insert(trace) {
|
||||
self.deferred.entry(hash).or_insert_with(Vec::new).push(statement);
|
||||
}
|
||||
}
|
||||
|
||||
fn get_deferred(&mut self, hash: &Hash) -> (Vec<SignedStatement>, Vec<StatementTrace>) {
|
||||
match self.deferred.remove(hash) {
|
||||
None => (Vec::new(), Vec::new()),
|
||||
Some(deferred) => {
|
||||
let mut traces = Vec::new();
|
||||
for statement in deferred.iter() {
|
||||
let trace = match statement.statement {
|
||||
GenericStatement::Candidate(_) => continue,
|
||||
GenericStatement::Valid(hash) => StatementTrace::Valid(statement.sender, hash),
|
||||
GenericStatement::Invalid(hash) => StatementTrace::Invalid(statement.sender, hash),
|
||||
GenericStatement::Available(hash) => StatementTrace::Available(statement.sender, hash),
|
||||
};
|
||||
|
||||
self.known_traces.remove(&trace);
|
||||
traces.push(trace);
|
||||
}
|
||||
|
||||
(deferred, traces)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use substrate_primitives::H512;
|
||||
|
||||
#[test]
|
||||
fn deferred_statements_works() {
|
||||
let mut deferred = DeferredStatements::new();
|
||||
let hash = [1; 32].into();
|
||||
let sig = H512([2; 64]).into();
|
||||
let sender = [255; 32].into();
|
||||
|
||||
let statement = SignedStatement {
|
||||
statement: GenericStatement::Valid(hash),
|
||||
sender,
|
||||
signature: sig,
|
||||
};
|
||||
|
||||
// pre-push.
|
||||
{
|
||||
let (signed, traces) = deferred.get_deferred(&hash);
|
||||
assert!(signed.is_empty());
|
||||
assert!(traces.is_empty());
|
||||
}
|
||||
|
||||
deferred.push(statement.clone());
|
||||
deferred.push(statement.clone());
|
||||
|
||||
// draining: second push should have been ignored.
|
||||
{
|
||||
let (signed, traces) = deferred.get_deferred(&hash);
|
||||
assert_eq!(signed.len(), 1);
|
||||
|
||||
assert_eq!(traces.len(), 1);
|
||||
assert_eq!(signed[0].clone(), statement);
|
||||
assert_eq!(traces[0].clone(), StatementTrace::Valid(sender, hash));
|
||||
}
|
||||
|
||||
// after draining
|
||||
{
|
||||
let (signed, traces) = deferred.get_deferred(&hash);
|
||||
assert!(signed.is_empty());
|
||||
assert!(traces.is_empty());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,204 @@
|
||||
// Copyright 2018 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Polkadot.
|
||||
|
||||
// Polkadot is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Polkadot is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Tests for polkadot and consensus network.
|
||||
|
||||
use super::{PolkadotProtocol, Status, CurrentConsensus, Knowledge, Message, FullStatus};
|
||||
|
||||
use parking_lot::Mutex;
|
||||
use polkadot_consensus::GenericStatement;
|
||||
use polkadot_primitives::{Block, Hash, SessionKey};
|
||||
use polkadot_primitives::parachain::{CandidateReceipt, HeadData, BlockData};
|
||||
use codec::Slicable;
|
||||
use substrate_network::{PeerId, PeerInfo, ClientHandle, Context, message::Message as SubstrateMessage, message::Role, specialization::Specialization, generic_message::Message as GenericMessage};
|
||||
|
||||
use std::sync::Arc;
|
||||
use futures::Future;
|
||||
|
||||
#[derive(Default)]
|
||||
struct TestContext {
|
||||
disabled: Vec<PeerId>,
|
||||
disconnected: Vec<PeerId>,
|
||||
messages: Vec<(PeerId, SubstrateMessage<Block>)>,
|
||||
}
|
||||
|
||||
impl Context<Block> for TestContext {
|
||||
fn client(&self) -> &ClientHandle<Block> {
|
||||
unimplemented!()
|
||||
}
|
||||
|
||||
fn disable_peer(&mut self, peer: PeerId) {
|
||||
self.disabled.push(peer);
|
||||
}
|
||||
|
||||
fn disconnect_peer(&mut self, peer: PeerId) {
|
||||
self.disconnected.push(peer);
|
||||
}
|
||||
|
||||
fn peer_info(&self, _peer: PeerId) -> Option<PeerInfo<Block>> {
|
||||
unimplemented!()
|
||||
}
|
||||
|
||||
fn send_message(&mut self, peer_id: PeerId, data: SubstrateMessage<Block>) {
|
||||
self.messages.push((peer_id, data))
|
||||
}
|
||||
}
|
||||
|
||||
impl TestContext {
|
||||
fn has_message(&self, to: PeerId, message: Message) -> bool {
|
||||
use substrate_network::generic_message::Message as GenericMessage;
|
||||
|
||||
let encoded = ::serde_json::to_vec(&message).unwrap();
|
||||
self.messages.iter().any(|&(ref peer, ref msg)| match msg {
|
||||
GenericMessage::ChainSpecific(ref data) => peer == &to && data == &encoded,
|
||||
_ => false,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn make_status(status: &Status, roles: Vec<Role>) -> FullStatus {
|
||||
FullStatus {
|
||||
version: 1,
|
||||
roles,
|
||||
best_number: 0,
|
||||
best_hash: Default::default(),
|
||||
genesis_hash: Default::default(),
|
||||
chain_status: status.encode(),
|
||||
}
|
||||
}
|
||||
|
||||
fn make_consensus(parent_hash: Hash, local_key: SessionKey) -> (CurrentConsensus, Arc<Mutex<Knowledge>>) {
|
||||
let knowledge = Arc::new(Mutex::new(Knowledge::new()));
|
||||
let c = CurrentConsensus {
|
||||
knowledge: knowledge.clone(),
|
||||
parent_hash,
|
||||
session_keys: Default::default(),
|
||||
local_session_key: local_key,
|
||||
};
|
||||
|
||||
(c, knowledge)
|
||||
}
|
||||
|
||||
fn on_message(protocol: &mut PolkadotProtocol, ctx: &mut TestContext, from: PeerId, message: Message) {
|
||||
let encoded = ::serde_json::to_vec(&message).unwrap();
|
||||
protocol.on_message(ctx, from, GenericMessage::ChainSpecific(encoded));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sends_session_key() {
|
||||
let mut protocol = PolkadotProtocol::new();
|
||||
|
||||
let peer_a = 1;
|
||||
let peer_b = 2;
|
||||
let parent_hash = [0; 32].into();
|
||||
let local_key = [1; 32].into();
|
||||
|
||||
let status = Status { collating_for: None };
|
||||
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
protocol.on_connect(&mut ctx, peer_a, make_status(&status, vec![Role::Authority]));
|
||||
assert!(ctx.messages.is_empty());
|
||||
}
|
||||
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
let (consensus, _knowledge) = make_consensus(parent_hash, local_key);
|
||||
protocol.new_consensus(&mut ctx, consensus);
|
||||
|
||||
assert!(ctx.has_message(peer_a, Message::SessionKey(parent_hash, local_key)));
|
||||
}
|
||||
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
protocol.on_connect(&mut ctx, peer_b, make_status(&status, vec![Role::Authority]));
|
||||
assert!(ctx.has_message(peer_b, Message::SessionKey(parent_hash, local_key)));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fetches_from_those_with_knowledge() {
|
||||
let mut protocol = PolkadotProtocol::new();
|
||||
|
||||
let peer_a = 1;
|
||||
let peer_b = 2;
|
||||
let parent_hash = [0; 32].into();
|
||||
let local_key = [1; 32].into();
|
||||
|
||||
let block_data = BlockData(vec![1, 2, 3, 4]);
|
||||
let block_data_hash = block_data.hash();
|
||||
let candidate_receipt = CandidateReceipt {
|
||||
parachain_index: 5.into(),
|
||||
collator: [255; 32].into(),
|
||||
head_data: HeadData(vec![9, 9, 9]),
|
||||
balance_uploads: Vec::new(),
|
||||
egress_queue_roots: Vec::new(),
|
||||
fees: 1_000_000,
|
||||
block_data_hash,
|
||||
};
|
||||
|
||||
let candidate_hash = candidate_receipt.hash();
|
||||
let a_key = [3; 32].into();
|
||||
let b_key = [4; 32].into();
|
||||
|
||||
let status = Status { collating_for: None };
|
||||
|
||||
let (consensus, knowledge) = make_consensus(parent_hash, local_key);
|
||||
protocol.new_consensus(&mut TestContext::default(), consensus);
|
||||
|
||||
knowledge.lock().note_statement(a_key, &GenericStatement::Valid(candidate_hash));
|
||||
let recv = protocol.fetch_block_data(&mut TestContext::default(), &candidate_receipt, parent_hash);
|
||||
|
||||
// connect peer A
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
protocol.on_connect(&mut ctx, peer_a, make_status(&status, vec![Role::Authority]));
|
||||
assert!(ctx.has_message(peer_a, Message::SessionKey(parent_hash, local_key)));
|
||||
}
|
||||
|
||||
// peer A gives session key and gets asked for data.
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
on_message(&mut protocol, &mut ctx, peer_a, Message::SessionKey(parent_hash, a_key));
|
||||
assert!(ctx.has_message(peer_a, Message::RequestBlockData(1, candidate_hash)));
|
||||
}
|
||||
|
||||
knowledge.lock().note_statement(b_key, &GenericStatement::Valid(candidate_hash));
|
||||
|
||||
// peer B connects and sends session key. request already assigned to A
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
protocol.on_connect(&mut ctx, peer_b, make_status(&status, vec![Role::Authority]));
|
||||
on_message(&mut protocol, &mut ctx, peer_b, Message::SessionKey(parent_hash, b_key));
|
||||
assert!(!ctx.has_message(peer_b, Message::RequestBlockData(2, candidate_hash)));
|
||||
|
||||
}
|
||||
|
||||
// peer A disconnects, triggering reassignment
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
protocol.on_disconnect(&mut ctx, peer_a);
|
||||
assert!(ctx.has_message(peer_b, Message::RequestBlockData(2, candidate_hash)));
|
||||
}
|
||||
|
||||
// peer B comes back with block data.
|
||||
{
|
||||
let mut ctx = TestContext::default();
|
||||
on_message(&mut protocol, &mut ctx, peer_b, Message::BlockData(2, Some(block_data.clone())));
|
||||
drop(protocol);
|
||||
assert_eq!(recv.wait().unwrap(), block_data);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user