pezkuwichain/pezkuwi-subxt
1
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-11-01 00:19:49 +00:00
Code Releases Activity

Minimal parachains part 2: Parachain statement and data routing (#173)

Browse Source 
* dynamic inclusion threshold calculator

* collators interface

* collation helpers

* initial proposal-creation future

* create proposer when asked to propose

* remove local_availability duty

* statement table tracks includable parachain count

* beginnings of timing future

* finish proposal logic

* remove stray println

* extract shared table to separate module

* change ordering

* includability tracking

* fix doc

* initial changes to parachains module

* initialise dummy block before API calls

* give polkadot control over round proposer based on random seed

* propose only after enough candidates

* flesh out parachains module a bit more

* set_heads

* actually introduce set_heads to runtime

* update block_builder to accept parachains

* split block validity errors from real errors in evaluation

* update WASM runtimes

* polkadot-api methods for parachains additions

* delay evaluation until candidates are ready

* comments

* fix dynamic inclusion with zero initial

* test for includability tracker

* wasm validation of parachain candidates

* move primitives to primitives crate

* remove runtime-std dependency from codec

* adjust doc

* polkadot-parachain-primitives

* kill legacy polkadot-validator crate

* basic-add test chain

* test for basic_add parachain

* move to test-chains dir

* use wasm-build

* new wasm directory layout

* reorganize a bit more

* Fix for rh-minimal-parachain (#141)

* Remove extern "C"

We already encountered such behavior (bug?) in pwasm-std, I believe.

* Fix `panic_fmt` signature by adding `_col`

Wrong `panic_fmt` signature can inhibit some optimizations in LTO mode.

* Add linker flags and use wasm-gc in build script

Pass --import-memory to LLD to emit wasm binary with imported memory.

Also use wasm-gc instead of wasm-build.

* Fix effective_max.

I'm not sure why it was the way it was actually.

* Recompile wasm.

* Fix indent

* more basic_add tests

* validate parachain WASM

* produce statements on receiving statements

* tests for reactive statement production

* fix build

* add OOM lang item to runtime-io

* use dynamic_inclusion when evaluating as well

* fix update_includable_count

* remove dead code

* grumbles

* actually defer round_proposer logic

* update wasm

* address a few more grumbles

* schedule collation work as soon as BFT is started

* impl future in collator

* fix comment

* governance proposals for adding and removing parachains

* bump protocol version

* tear out polkadot-specific pieces of substrate-network

* extract out polkadot-specific stuff from substrate-network

* begin polkadot network subsystem

* grumbles

* update WASM checkins

* parse status from polkadot peer

* allow invoke of network specialization

* begin statement router implementation

* remove dependency on tokio-timer

* fix sanity check and have proposer factory create communication streams

* pull out statement routing from consensus library

* fix comments

* adjust typedefs

* extract consensus_gossip out of main network protocol handler

* port substrate-bft to new tokio

* port polkadot-consensus to new tokio

* fix typo

* start message processing task

* initial consensus network implementation

* remove known tracking from statement-table crate

* extract router into separate module

* defer statements until later

* double signature is invalid

* propagating statements

* grumbles

* request block data

* fix compilation

* embed new consensus network into service

* port demo CLI to tokio

* all test crates compile

* some tests for fetching block data

* whitespace

* adjusting some tokio stuff

* update exit-future

* remove overly noisy warning

* clean up collation work a bit

* address review grumbles

* fix lock order in protocol handler

* rebuild wasm artifacts

* tag AuthorityId::from_slice for std only

* address formatting grumbles

* rename event_loop to executor

* some more docs for polkadot-network crate
This commit is contained in:
Robert Habermeier authored and GitHub committed 2018-07-06 14:17:03 +02:00
1 parent 5355956314
commit be5ff4e62f
62 files changed
+3424 -2134
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

substrate/substrate/bft/Cargo.toml
+1 -2
View File
@@ -10,7 +10,7 @@ substrate-primitives = { path = "../primitives" }
substrate-runtime-support = { path = "../runtime-support" }
substrate-runtime-primitives = { path = "../runtime/primitives" }
ed25519 = { path = "../ed25519" }
tokio-timer = "0.1.2"
tokio = "0.1.7"
parking_lot = "0.4"
error-chain = "0.12"
log = "0.3"
@@ -18,4 +18,3 @@ log = "0.3"
[dev-dependencies]
substrate-keyring = { path = "../keyring" }
substrate-executor = { path = "../executor" }
tokio-core = "0.1.12"
substrate/substrate/bft/src/error.rs
+3 -3
View File
@@ -31,9 +31,9 @@ error_chain! {
}
/// Unable to schedule wakeup.
FaultyTimer {
description("Faulty timer: unable to schedule wakeup"),
display("Faulty timer: unable to schedule wakeup"),
FaultyTimer(e: ::tokio::timer::Error) {
description("Timer error"),
display("Timer error: {}", e),
}
/// Unable to propose a block.
substrate/substrate/bft/src/generic/mod.rs
+1 -11
View File
@@ -812,17 +812,7 @@ impl<C: Context, I, O> Agreement<C, I, O> {
/// `max_faulty` is the maximum number of faulty nodes. Should be less than
/// 1/3 of `nodes`, otherwise agreement may never be reached.
///
/// The input stream should never logically conclude. The logic here assumes
/// that messages flushed to the output stream will eventually reach other nodes.
///
/// Note that it is possible to witness agreement being reached without ever
/// seeing the candidate. Any candidates seen will be checked for validity.
///
/// Although technically the agreement will always complete (given the eventual
/// delivery of messages), in practice it is possible for this future to
/// conclude without having witnessed the conclusion.
/// In general, this future should be pre-empted by the import of a justification
/// set for this block height.
/// The input and output streams should follow the constraints documented in the crate root.
pub fn agree<C: Context, I, O>(context: C, nodes: usize, max_faulty: usize, input: I, output: O)
-> Agreement<C, I, O>
where
substrate/substrate/bft/src/generic/tests.rs
+251 -286
View File
@@ -21,13 +21,14 @@ use super::*;
use std::collections::BTreeSet;
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use std::time::{Instant, Duration};
use futures::prelude::*;
use futures::sync::{oneshot, mpsc};
use futures::future::FutureResult;
use futures::sync::mpsc;
use futures::future::{self, FutureResult};
use tokio_timer::{self, Timer};
use tokio::timer::Delay;
use tokio::runtime;
use tokio::prelude::*;
const ROUND_DURATION: Duration = Duration::from_millis(50);
@@ -60,8 +61,8 @@ impl<T: Clone + Send + 'static> Network<T> {
(network, inputs, outputs)
}
fn route_on_thread(self) {
::std::thread::spawn(move || { let _ = self.wait(); });
fn route_in_background(self) {
::tokio::executor::spawn(self);
}
}
@@ -94,7 +95,7 @@ impl<T: Clone> Future for Network<T> {
#[derive(Debug, PartialEq, Eq, Clone, Hash)]
struct Candidate(usize);
#[derive(Debug, PartialEq, Eq, Clone, Hash)]
#[derive(Debug, PartialEq, Eq, Clone, Copy, Hash)]
struct Digest(usize);
#[derive(Debug, PartialEq, Eq, Clone, Hash)]
@@ -117,7 +118,6 @@ struct TestContext {
proposal: Mutex<usize>,
node_count: usize,
current_round: Arc<AtomicUsize>,
timer: Timer,
evaluated: Mutex<BTreeSet<usize>>,
}
@@ -194,7 +194,7 @@ impl Context for TestContext {
}
let current_round = self.current_round.clone();
let timeout = self.timer.sleep(round_duration)
let timeout = Delay::new(Instant::now() + round_duration)
.map(move |_| {
current_round.compare_and_swap(round, round + 1, Ordering::SeqCst);
})
@@ -205,278 +205,37 @@ impl Context for TestContext {
}
}
fn timeout_in(t: Duration) -> oneshot::Receiver<()> {
let (tx, rx) = oneshot::channel();
::std::thread::spawn(move || {
::std::thread::sleep(t);
let _ = tx.send(());
});
rx
fn test_harness<F, X, T, U>(f: F) -> Result<T, U> where
F: FnOnce() -> X,
X: IntoFuture<Item=T,Error=U>,
{
let mut runtime = runtime::current_thread::Runtime::new().unwrap();
runtime.block_on(future::lazy(f))
}
#[test]
fn consensus_completes_with_minimum_good() {
let node_count = 10;
let max_faulty = 3;
let results = test_harness(|| {
let node_count = 10;
let max_faulty = 3;
let timer = tokio_timer::wheel().tick_duration(ROUND_DURATION).build();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_in_background();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_on_thread();
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty)
.enumerate()
.map(|(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(0)),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty)
.enumerate()
.map(|(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(0)),
timer: timer.clone(),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
)
})
.collect::<Vec<_>>();
let timeout = timeout_in(Duration::from_millis(500)).map_err(|_| Error);
let results = ::futures::future::join_all(nodes)
.map(Some)
.select(timeout.map(|_| None))
.wait()
.map(|(i, _)| i)
.map_err(|(e, _)| e)
.expect("to complete")
.expect("to not time out");
for result in &results {
assert_eq!(&result.justification.digest, &results[0].justification.digest);
}
}
#[test]
fn consensus_completes_with_minimum_good_all_initial_proposals_bad() {
let node_count = 10;
let max_faulty = 3;
let timer = tokio_timer::wheel().tick_duration(ROUND_DURATION).build();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_on_thread();
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty)
.enumerate()
.map(|(i, (tx, rx))| {
// the first 5 proposals are going to be bad.
let proposal = if i < 5 {
i * 3 // proposals considered bad in the tests if they are % 3
} else {
(i * 3) + 1
};
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(proposal),
current_round: Arc::new(AtomicUsize::new(0)),
timer: timer.clone(),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
)
})
.collect::<Vec<_>>();
let timeout = timeout_in(Duration::from_millis(500)).map_err(|_| Error);
let results = ::futures::future::join_all(nodes)
.map(Some)
.select(timeout.map(|_| None))
.wait()
.map(|(i, _)| i)
.map_err(|(e, _)| e)
.expect("to complete")
.expect("to not time out");
for result in &results {
assert_eq!(&result.justification.digest, &results[0].justification.digest);
}
}
#[test]
fn consensus_does_not_complete_without_enough_nodes() {
let node_count = 10;
let max_faulty = 3;
let timer = tokio_timer::wheel().tick_duration(ROUND_DURATION).build();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_on_thread();
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty - 1)
.enumerate()
.map(|(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(0)),
timer: timer.clone(),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
)
})
.collect::<Vec<_>>();
let timeout = timeout_in(Duration::from_millis(500)).map_err(|_| Error);
let result = ::futures::future::join_all(nodes)
.map(Some)
.select(timeout.map(|_| None))
.wait()
.map(|(i, _)| i)
.map_err(|(e, _)| e)
.expect("to complete");
assert!(result.is_none(), "not enough online nodes");
}
#[test]
fn threshold_plus_one_locked_on_proposal_only_one_with_candidate() {
let node_count = 10;
let max_faulty = 3;
let locked_proposal = Candidate(999_999_999);
let locked_digest = Digest(999_999_999);
let locked_round = 1;
let justification = UncheckedJustification {
round_number: locked_round,
digest: locked_digest.clone(),
signatures: (0..7)
.map(|i| Signature(Message::Vote(Vote::Prepare(locked_round, locked_digest.clone())), AuthorityId(i)))
.collect()
}.check(7, |_, _, s| Some(s.1.clone())).unwrap();
let timer = tokio_timer::wheel().tick_duration(ROUND_DURATION).build();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_on_thread();
let nodes = net_send
.into_iter()
.zip(net_recv)
.enumerate()
.map(|(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(locked_round + 1)),
timer: timer.clone(),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
let mut agreement = agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
);
agreement.strategy.advance_to_round(
&agreement.context,
locked_round + 1
);
if i <= max_faulty {
agreement.strategy.locked = Some(Locked {
justification: justification.clone(),
})
}
if i == max_faulty {
agreement.strategy.notable_candidates.insert(
locked_digest.clone(),
locked_proposal.clone(),
);
}
agreement
})
.collect::<Vec<_>>();
let timeout = timeout_in(Duration::from_millis(1000)).map_err(|_| Error);
let results = ::futures::future::join_all(nodes)
.map(Some)
.select(timeout.map(|_| None))
.wait()
.map(|(i, _)| i)
.map_err(|(e, _)| e)
.expect("to complete")
.expect("to not time out");
for result in &results {
assert_eq!(&result.justification.digest, &locked_digest);
}
}
#[test]
fn consensus_completes_even_when_nodes_start_with_a_delay() {
let node_count = 10;
let max_faulty = 3;
let base_sleep = Duration::from_millis(75);
let timer = tokio_timer::wheel().tick_duration(ROUND_DURATION).build();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_on_thread();
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty)
.enumerate()
.map(|(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(0)),
timer: timer.clone(),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
let sleep_duration = base_sleep * i as u32;
timer.sleep(sleep_duration).map_err(|_| Error).and_then(move |_| {
agree(
ctx,
node_count,
@@ -485,18 +244,224 @@ fn consensus_completes_even_when_nodes_start_with_a_delay() {
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
)
})
})
.collect::<Vec<_>>();
.collect::<Vec<_>>();
let timeout = timeout_in(Duration::from_millis(750)).map_err(|_| Error);
let results = ::futures::future::join_all(nodes)
.map(Some)
.select(timeout.map(|_| None))
.wait()
.map(|(i, _)| i)
.map_err(|(e, _)| e)
.expect("to complete")
.expect("to not time out");
::futures::future::join_all(nodes)
.deadline(Instant::now() + Duration::from_millis(500))
}).unwrap();
for result in &results {
assert_eq!(&result.justification.digest, &results[0].justification.digest);
}
}
#[test]
fn consensus_completes_with_minimum_good_all_initial_proposals_bad() {
let results = test_harness(|| {
let node_count = 10;
let max_faulty = 3;
let (network, net_send, net_recv) = Network::new(node_count);
network.route_in_background();
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty)
.enumerate()
.map(|(i, (tx, rx))| {
// the first 5 proposals are going to be bad.
let proposal = if i < 5 {
i * 3 // proposals considered bad in the tests if they are % 3
} else {
(i * 3) + 1
};
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(proposal),
current_round: Arc::new(AtomicUsize::new(0)),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
)
})
.collect::<Vec<_>>();
::futures::future::join_all(nodes)
.deadline(Instant::now() + Duration::from_millis(500))
}).unwrap();
for result in &results {
assert_eq!(&result.justification.digest, &results[0].justification.digest);
}
}
#[test]
fn consensus_does_not_complete_without_enough_nodes() {
let result = test_harness(|| {
let node_count = 10;
let max_faulty = 3;
let (network, net_send, net_recv) = Network::new(node_count);
network.route_in_background();
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty - 1)
.enumerate()
.map(|(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(0)),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
)
})
.collect::<Vec<_>>();
::futures::future::join_all(nodes)
.deadline(Instant::now() + Duration::from_millis(500))
});
match result {
Ok(_) => panic!("completed wrongly"),
Err(ref e) if e.is_inner() => panic!("failed for wrong reason"),
Err(_) => {},
}
}
#[test]
fn threshold_plus_one_locked_on_proposal_only_one_with_candidate() {
let locked_digest = Digest(999_999_999);
let results = test_harness(move || {
let node_count = 10;
let max_faulty = 3;
let locked_proposal = Candidate(999_999_999);
let locked_round = 1;
let justification = UncheckedJustification {
round_number: locked_round,
digest: locked_digest.clone(),
signatures: (0..7)
.map(|i| Signature(Message::Vote(Vote::Prepare(locked_round, locked_digest.clone())), AuthorityId(i)))
.collect()
}.check(7, |_, _, s| Some(s.1.clone())).unwrap();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_in_background();
let nodes = net_send
.into_iter()
.zip(net_recv)
.enumerate()
.map(|(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(locked_round + 1)),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
let mut agreement = agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
);
agreement.strategy.advance_to_round(
&agreement.context,
locked_round + 1
);
if i <= max_faulty {
agreement.strategy.locked = Some(Locked {
justification: justification.clone(),
})
}
if i == max_faulty {
agreement.strategy.notable_candidates.insert(
locked_digest.clone(),
locked_proposal.clone(),
);
}
agreement
})
.collect::<Vec<_>>();
::futures::future::join_all(nodes)
.deadline(Instant::now() + Duration::from_millis(1000))
}).unwrap();
for result in &results {
assert_eq!(&result.justification.digest, &locked_digest);
}
}
#[test]
fn consensus_completes_even_when_nodes_start_with_a_delay() {
let results = test_harness(|| {
let node_count = 10;
let max_faulty = 3;
let base_sleep = Duration::from_millis(75);
let now = Instant::now();
let (network, net_send, net_recv) = Network::new(node_count);
network.route_in_background();
let nodes = net_send
.into_iter()
.zip(net_recv)
.take(node_count - max_faulty)
.enumerate()
.map(move |(i, (tx, rx))| {
let ctx = TestContext {
local_id: AuthorityId(i),
proposal: Mutex::new(i),
current_round: Arc::new(AtomicUsize::new(0)),
evaluated: Mutex::new(BTreeSet::new()),
node_count,
};
let sleep_duration = base_sleep * i as u32;
Delay::new(now + sleep_duration).map_err(|_| Error).and_then(move |_| {
agree(
ctx,
node_count,
max_faulty,
rx.map_err(|_| Error),
tx.sink_map_err(|_| Error).with(move |t| Ok((i, t))),
)
})
})
.collect::<Vec<_>>();
::futures::future::join_all(nodes)
.deadline(now + Duration::from_millis(750))
}).unwrap();
for result in &results {
assert_eq!(&result.justification.digest, &results[0].justification.digest);
substrate/substrate/bft/src/lib.rs
+64 -40
View File
@@ -15,6 +15,20 @@
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
//! BFT Agreement based on a rotating proposer in different rounds.
//!
//! Where this crate refers to input stream, should never logically conclude.
//! The logic in this crate assumes that messages flushed to the output stream
//! will eventually reach other nodes and that our own messages are not included
//! in the input stream.
//!
//! Note that it is possible to witness agreement being reached without ever
//! seeing the candidate. Any candidates seen will be checked for validity.
//!
//! Although technically the agreement will always complete (given the eventual
//! delivery of messages), in practice it is possible for this future to
//! conclude without having witnessed the conclusion.
//! In general, this future should be pre-empted by the import of a justification
//! set for this block height.
pub mod error;
pub mod generic;
@@ -24,7 +38,7 @@ extern crate substrate_primitives as primitives;
extern crate substrate_runtime_support as runtime_support;
extern crate substrate_runtime_primitives as runtime_primitives;
extern crate ed25519;
extern crate tokio_timer;
extern crate tokio;
extern crate parking_lot;
#[macro_use]
@@ -49,7 +63,7 @@ use primitives::AuthorityId;
use futures::{task, Async, Stream, Sink, Future, IntoFuture};
use futures::sync::oneshot;
use tokio_timer::Timer;
use tokio::timer::Delay;
use parking_lot::Mutex;
pub use generic::InputStreamConcluded;
@@ -108,16 +122,22 @@ pub type Communication<B> = generic::Communication<B, <B as Block>::Hash, Author
/// Misbehavior observed from BFT participants.
pub type Misbehavior<H> = generic::Misbehavior<H, LocalizedSignature>;
/// Proposer factory. Can be used to create a proposer instance.
pub trait ProposerFactory<B: Block> {
/// Environment producer for a BFT instance. Creates proposer instance and communication streams.
pub trait Environment<B: Block> {
/// The proposer type this creates.
type Proposer: Proposer<B>;
/// The input stream type.
type Input: Stream<Item=Communication<B>, Error=<Self::Proposer as Proposer<B>>::Error>;
/// The output stream type.
type Output: Sink<SinkItem=Communication<B>, SinkError=<Self::Proposer as Proposer<B>>::Error>;
/// Error which can occur upon creation.
type Error: From<Error>;
/// Initialize the proposal logic on top of a specific header.
/// Produces the proposer and message streams for this instance of BFT agreement.
// TODO: provide state context explicitly?
fn init(&self, parent_header: &B::Header, authorities: &[AuthorityId], sign_with: Arc<ed25519::Pair>) -> Result<Self::Proposer, Self::Error>;
fn init(&self, parent_header: &B::Header, authorities: &[AuthorityId], sign_with: Arc<ed25519::Pair>)
-> Result<(Self::Proposer, Self::Input, Self::Output), Self::Error>;
}
/// Logic for a proposer.
@@ -163,7 +183,6 @@ struct BftInstance<B: Block, P> {
key: Arc<ed25519::Pair>,
authorities: Vec<AuthorityId>,
parent_hash: B::Hash,
timer: Timer,
round_timeout_multiplier: u64,
proposer: P,
}
@@ -178,7 +197,7 @@ impl<B: Block, P: Proposer<B>> generic::Context for BftInstance<B, P>
type Digest = B::Hash;
type Signature = LocalizedSignature;
type Candidate = B;
type RoundTimeout = Box<Future<Item=(),Error=Self::Error> + Send>;
type RoundTimeout = Box<Future<Item=(),Error=Self::Error>>;
type CreateProposal = <P::Create as IntoFuture>::Future;
type EvaluateProposal = <P::Evaluate as IntoFuture>::Future;
@@ -207,16 +226,18 @@ impl<B: Block, P: Proposer<B>> generic::Context for BftInstance<B, P>
}
fn begin_round_timeout(&self, round: usize) -> Self::RoundTimeout {
use std::time::Duration;
use std::time::{Instant, Duration};
let round = ::std::cmp::min(63, round) as u32;
let timeout = 1u64.checked_shl(round)
.unwrap_or_else(u64::max_value)
.saturating_mul(self.round_timeout_multiplier);
Box::new(self.timer.sleep(Duration::from_secs(timeout))
.map_err(|_| Error::from(ErrorKind::FaultyTimer))
.map_err(Into::into))
let fut = Delay::new(Instant::now() + Duration::from_secs(timeout))
.map_err(|e| Error::from(ErrorKind::FaultyTimer(e)))
.map_err(Into::into);
Box::new(fut)
}
}
@@ -258,10 +279,8 @@ impl<B, P, I, InStream, OutSink> Future for BftFuture<B, P, I, InStream, OutSink
return Ok(Async::Ready(()))
}
// TODO: handle this error, at least by logging.
let committed = try_ready!(self.inner.poll().map_err(|e| {
warn!(target: "bft", "Error in BFT agreement: {}", e);
}));
// TODO: handle and log this error in a way which isn't noisy on exit.
let committed = try_ready!(self.inner.poll().map_err(|_| ()));
// If we didn't see the proposal (very unlikely),
// we will get the block from the network later.
@@ -312,10 +331,11 @@ impl Drop for AgreementHandle {
/// The BftService kicks off the agreement process on top of any blocks it
/// is notified of.
///
/// This assumes that it is being run in the context of a tokio runtime.
pub struct BftService<B: Block, P, I> {
client: Arc<I>,
live_agreement: Mutex<Option<(B::Hash, AgreementHandle)>>,
timer: Timer,
round_timeout_multiplier: u64,
key: Arc<ed25519::Pair>, // TODO: key changing over time.
factory: P,
@@ -324,8 +344,7 @@ pub struct BftService<B: Block, P, I> {
impl<B, P, I> BftService<B, P, I>
where
B: Block + Clone + Eq,
B::Hash: ::std::hash::Hash,
P: ProposerFactory<B>,
P: Environment<B>,
<P::Proposer as Proposer<B>>::Error: ::std::fmt::Display,
I: BlockImport<B> + Authorities<B>,
{
@@ -335,7 +354,6 @@ impl<B, P, I> BftService<B, P, I>
BftService {
client: client,
live_agreement: Mutex::new(None),
timer: Timer::default(),
round_timeout_multiplier: 4,
key: key, // TODO: key changing over time.
factory: factory,
@@ -353,10 +371,16 @@ impl<B, P, I> BftService<B, P, I>
/// If the local signing key is an authority, this will begin the consensus process to build a
/// block on top of it. If the executor fails to run the future, an error will be returned.
/// Returns `None` if the agreement on the block with given parent is already in progress.
pub fn build_upon<InStream, OutSink>(&self, header: &B::Header, input: InStream, output: OutSink)
-> Result<Option<BftFuture<B, <P as ProposerFactory<B>>::Proposer, I, InStream, OutSink>>, P::Error> where
InStream: Stream<Item=Communication<B>, Error=<<P as ProposerFactory<B>>::Proposer as Proposer<B>>::Error>,
OutSink: Sink<SinkItem=Communication<B>, SinkError=<<P as ProposerFactory<B>>::Proposer as Proposer<B>>::Error>,
pub fn build_upon(&self, header: &B::Header)
-> Result<Option<
BftFuture<
B,
<P as Environment<B>>::Proposer,
I,
<P as Environment<B>>::Input,
<P as Environment<B>>::Output,
>>, P::Error>
where
{
let hash = header.hash();
if self.live_agreement.lock().as_ref().map_or(false, |&(ref h, _)| h == &hash) {
@@ -377,13 +401,12 @@ impl<B, P, I> BftService<B, P, I>
Err(From::from(ErrorKind::InvalidAuthority(local_id)))?;
}
let proposer = self.factory.init(header, &authorities, self.key.clone())?;
let (proposer, input, output) = self.factory.init(header, &authorities, self.key.clone())?;
let bft_instance = BftInstance {
proposer,
parent_hash: hash.clone(),
round_timeout_multiplier: self.round_timeout_multiplier,
timer: self.timer.clone(),
key: self.key.clone(),
authorities: authorities,
};
@@ -592,13 +615,11 @@ mod tests {
use std::collections::HashSet;
use runtime_primitives::testing::{Block as GenericTestBlock, Header as TestHeader};
use primitives::H256;
use self::tokio_core::reactor::{Core};
use self::keyring::Keyring;
use futures::stream;
use futures::future::Executor;
use tokio::executor::current_thread;
extern crate substrate_keyring as keyring;
extern crate tokio_core;
type TestBlock = GenericTestBlock<()>;
@@ -638,12 +659,16 @@ mod tests {
struct DummyFactory;
struct DummyProposer(u64);
impl ProposerFactory<TestBlock> for DummyFactory {
impl Environment<TestBlock> for DummyFactory {
type Proposer = DummyProposer;
type Input = stream::Empty<Communication<TestBlock>, Error>;
type Output = Output<Error>;
type Error = Error;
fn init(&self, parent_header: &TestHeader, _authorities: &[AuthorityId], _sign_with: Arc<ed25519::Pair>) -> Result<DummyProposer, Error> {
Ok(DummyProposer(parent_header.number + 1))
fn init(&self, parent_header: &TestHeader, _authorities: &[AuthorityId], _sign_with: Arc<ed25519::Pair>)
-> Result<(DummyProposer, Self::Input, Self::Output), Error>
{
Ok((DummyProposer(parent_header.number + 1), stream::empty(), Output(::std::marker::PhantomData)))
}
}
@@ -677,7 +702,6 @@ mod tests {
BftService {
client: Arc::new(client),
live_agreement: Mutex::new(None),
timer: Timer::default(),
round_timeout_multiplier: 4,
key: Arc::new(Keyring::One.into()),
factory: DummyFactory
@@ -713,8 +737,6 @@ mod tests {
imported_heights: Mutex::new(HashSet::new()),
};
let mut core = Core::new().unwrap();
let service = make_service(client);
let first = from_block_number(2);
@@ -724,19 +746,21 @@ mod tests {
second.parent_hash = first_hash;
let second_hash = second.hash();
let bft = service.build_upon(&first, stream::empty(), Output(Default::default())).unwrap();
let bft = service.build_upon(&first).unwrap();
assert!(service.live_agreement.lock().as_ref().unwrap().0 == first_hash);
let mut core = current_thread::CurrentThread::new();
// turn the core so the future gets polled and sends its task to the
// service. otherwise it deadlocks.
core.handle().execute(bft.unwrap()).unwrap();
core.turn(Some(::std::time::Duration::from_millis(100)));
let bft = service.build_upon(&second, stream::empty(), Output(Default::default())).unwrap();
core.spawn(bft.unwrap());
core.run_timeout(::std::time::Duration::from_millis(100)).unwrap();
let bft = service.build_upon(&second).unwrap();
assert!(service.live_agreement.lock().as_ref().unwrap().0 != first_hash);
assert!(service.live_agreement.lock().as_ref().unwrap().0 == second_hash);
core.handle().execute(bft.unwrap()).unwrap();
core.turn(Some(::std::time::Duration::from_millis(100)));
core.spawn(bft.unwrap());
core.run_timeout(::std::time::Duration::from_millis(100)).unwrap();
}
#[test]
substrate/substrate/executor/wasm/target/wasm32-unknown-unknown/release/runtime_test.compact.wasm
BIN
View File
Binary file not shown.
substrate/substrate/executor/wasm/target/wasm32-unknown-unknown/release/runtime_test.wasm
BIN
View File
Binary file not shown.
substrate/substrate/network/src/chain.rs
+14 -13
View File
@@ -1,28 +1,29 @@
// Copyright 2017 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
//! Blockchain access trait
use client::{self, Client as PolkadotClient, ImportResult, ClientInfo, BlockStatus, BlockOrigin, CallExecutor};
use client::{self, Client as SubstrateClient, ImportResult, ClientInfo, BlockStatus, BlockOrigin, CallExecutor};
use client::error::Error;
use state_machine;
use runtime_primitives::traits::{Block as BlockT, Header as HeaderT};
use runtime_primitives::generic::BlockId;
use runtime_primitives::bft::Justification;
/// Local client abstraction for the network.
pub trait Client<Block: BlockT>: Send + Sync {
/// Import a new block. Parent is supposed to be existing in the blockchain.
fn import(&self, is_best: bool, header: Block::Header, justification: Justification<Block::Hash>, body: Option<Vec<Block::Extrinsic>>) -> Result<ImportResult, Error>;
@@ -49,7 +50,7 @@ pub trait Client<Block: BlockT>: Send + Sync {
fn execution_proof(&self, block: &Block::Hash, method: &str, data: &[u8]) -> Result<(Vec<u8>, Vec<Vec<u8>>), Error>;
}
impl<B, E, Block> Client<Block> for PolkadotClient<B, E, Block> where
impl<B, E, Block> Client<Block> for SubstrateClient<B, E, Block> where
B: client::backend::Backend<Block> + Send + Sync + 'static,
E: CallExecutor<Block> + Send + Sync + 'static,
Block: BlockT,
@@ -59,34 +60,34 @@ impl<B, E, Block> Client<Block> for PolkadotClient<B, E, Block> where
// TODO: defer justification check.
let justified_header = self.check_justification(header, justification.into())?;
let origin = if is_best { BlockOrigin::NetworkBroadcast } else { BlockOrigin::NetworkInitialSync };
(self as &PolkadotClient<B, E, Block>).import_block(origin, justified_header, body)
(self as &SubstrateClient<B, E, Block>).import_block(origin, justified_header, body)
}
fn info(&self) -> Result<ClientInfo<Block>, Error> {
(self as &PolkadotClient<B, E, Block>).info()
(self as &SubstrateClient<B, E, Block>).info()
}
fn block_status(&self, id: &BlockId<Block>) -> Result<BlockStatus, Error> {
(self as &PolkadotClient<B, E, Block>).block_status(id)
(self as &SubstrateClient<B, E, Block>).block_status(id)
}
fn block_hash(&self, block_number: <Block::Header as HeaderT>::Number) -> Result<Option<Block::Hash>, Error> {
(self as &PolkadotClient<B, E, Block>).block_hash(block_number)
(self as &SubstrateClient<B, E, Block>).block_hash(block_number)
}
fn header(&self, id: &BlockId<Block>) -> Result<Option<Block::Header>, Error> {
(self as &PolkadotClient<B, E, Block>).header(id)
(self as &SubstrateClient<B, E, Block>).header(id)
}
fn body(&self, id: &BlockId<Block>) -> Result<Option<Vec<Block::Extrinsic>>, Error> {
(self as &PolkadotClient<B, E, Block>).body(id)
(self as &SubstrateClient<B, E, Block>).body(id)
}
fn justification(&self, id: &BlockId<Block>) -> Result<Option<Justification<Block::Hash>>, Error> {
(self as &PolkadotClient<B, E, Block>).justification(id)
(self as &SubstrateClient<B, E, Block>).justification(id)
}
fn execution_proof(&self, block: &Block::Hash, method: &str, data: &[u8]) -> Result<(Vec<u8>, Vec<Vec<u8>>), Error> {
(self as &PolkadotClient<B, E, Block>).execution_proof(&BlockId::Hash(block.clone()), method, data)
(self as &SubstrateClient<B, E, Block>).execution_proof(&BlockId::Hash(block.clone()), method, data)
}
}
substrate/substrate/network/src/consensus.rs
-263
View File
@@ -1,263 +0,0 @@
// Copyright 2017 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.?
//! Consensus related bits of the network service.
use std::collections::{HashMap, HashSet};
use futures::sync::mpsc;
use std::time::{Instant, Duration};
use io::SyncIo;
use protocol::Protocol;
use network::PeerId;
use runtime_primitives::traits::{Block as BlockT, Header as HeaderT};
use runtime_primitives::generic::BlockId;
use message::{self, generic::Message as GenericMessage};
// TODO: Add additional spam/DoS attack protection.
const MESSAGE_LIFETIME: Duration = Duration::from_secs(600);
struct PeerConsensus<H> {
known_messages: HashSet<H>,
}
/// Consensus network protocol handler. Manages statements and candidate requests.
pub struct Consensus<B: BlockT> {
peers: HashMap<PeerId, PeerConsensus<B::Hash>>,
bft_message_sink: Option<(mpsc::UnboundedSender<message::LocalizedBftMessage<B>>, B::Hash)>,
messages: Vec<(B::Hash, Instant, message::Message<B>)>,
message_hashes: HashSet<B::Hash>,
}
impl<B: BlockT> Consensus<B> where B::Header: HeaderT<Number=u64> {
/// Create a new instance.
pub fn new() -> Self {
Consensus {
peers: HashMap::new(),
bft_message_sink: None,
messages: Default::default(),
message_hashes: Default::default(),
}
}
/// Closes all notification streams.
pub fn restart(&mut self) {
self.bft_message_sink = None;
}
/// Handle new connected peer.
pub fn new_peer(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId, roles: &[message::Role]) {
if roles.iter().any(|r| *r == message::Role::Validator) {
trace!(target:"sync", "Registering validator {}", peer_id);
// Send out all known messages.
// TODO: limit by size
let mut known_messages = HashSet::new();
for &(ref hash, _, ref message) in self.messages.iter() {
known_messages.insert(hash.clone());
protocol.send_message(io, peer_id, message.clone());
}
self.peers.insert(peer_id, PeerConsensus {
known_messages,
});
}
}
fn propagate(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, message: message::Message<B>, hash: B::Hash) {
for (id, ref mut peer) in self.peers.iter_mut() {
if peer.known_messages.insert(hash.clone()) {
protocol.send_message(io, *id, message.clone());
}
}
}
fn register_message(&mut self, hash: B::Hash, message: message::Message<B>) {
if self.message_hashes.insert(hash) {
self.messages.push((hash, Instant::now(), message));
}
}
pub fn on_bft_message(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId, message: message::LocalizedBftMessage<B>, hash: B::Hash) {
if self.message_hashes.contains(&hash) {
trace!(target:"sync", "Ignored already known BFT message from {}", peer_id);
return;
}
match (protocol.chain().info(), protocol.chain().header(&BlockId::Hash(message.parent_hash))) {
(_, Err(e)) | (Err(e), _) => {
debug!(target:"sync", "Error reading blockchain: {:?}", e);
return;
},
(Ok(info), Ok(Some(header))) => {
if header.number() < &info.chain.best_number {
trace!(target:"sync", "Ignored ancient BFT message from {}, hash={}", peer_id, message.parent_hash);
return;
}
},
(Ok(_), Ok(None)) => {},
}
if let Some(ref mut peer) = self.peers.get_mut(&peer_id) {
peer.known_messages.insert(hash);
// TODO: validate signature?
if let Some((sink, parent_hash)) = self.bft_message_sink.take() {
if message.parent_hash == parent_hash {
if let Err(e) = sink.unbounded_send(message.clone()) {
trace!(target:"sync", "Error broadcasting BFT message notification: {:?}", e);
} else {
self.bft_message_sink = Some((sink, parent_hash));
}
}
}
} else {
trace!(target:"sync", "Ignored BFT statement from unregistered peer {}", peer_id);
return;
}
let message = GenericMessage::BftMessage(message);
self.register_message(hash.clone(), message.clone());
// Propagate to other peers.
self.propagate(io, protocol, message, hash);
}
pub fn bft_messages(&mut self, parent_hash: B::Hash) -> mpsc::UnboundedReceiver<message::LocalizedBftMessage<B>> {
let (sink, stream) = mpsc::unbounded();
for &(_, _, ref message) in self.messages.iter() {
let bft_message = match *message {
GenericMessage::BftMessage(ref msg) => msg,
_ => continue,
};
if bft_message.parent_hash == parent_hash {
sink.unbounded_send(bft_message.clone()).expect("receiving end known to be open; qed");
}
}
self.bft_message_sink = Some((sink, parent_hash));
stream
}
pub fn send_bft_message(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, message: message::LocalizedBftMessage<B>) {
// Broadcast message to all validators.
trace!(target:"sync", "Broadcasting BFT message {:?}", message);
let message = GenericMessage::BftMessage(message);
let hash = Protocol::hash_message(&message);
self.register_message(hash.clone(), message.clone());
self.propagate(io, protocol, message, hash);
}
pub fn peer_disconnected(&mut self, _io: &mut SyncIo, _protocol: &Protocol<B>, peer_id: PeerId) {
self.peers.remove(&peer_id);
}
pub fn collect_garbage(&mut self, best_header: Option<&B::Header>) {
let hashes = &mut self.message_hashes;
let before = self.messages.len();
let now = Instant::now();
self.messages.retain(|&(ref hash, timestamp, ref message)| {
if timestamp >= now - MESSAGE_LIFETIME &&
best_header.map_or(true, |header|
match *message {
GenericMessage::BftMessage(ref msg) => &msg.parent_hash != header.parent_hash(),
_ => true,
})
{
true
} else {
hashes.remove(hash);
false
}
});
if self.messages.len() != before {
trace!(target:"sync", "Cleaned up {} stale messages", before - self.messages.len());
}
for (_, ref mut peer) in self.peers.iter_mut() {
peer.known_messages.retain(|h| hashes.contains(h));
}
}
}
#[cfg(test)]
mod tests {
use runtime_primitives::bft::Justification;
use runtime_primitives::testing::{H256, Header, Block as RawBlock};
use std::time::Instant;
use message::{self, generic::Message as GenericMessage};
use super::{Consensus, MESSAGE_LIFETIME};
type Block = RawBlock<u64>;
#[test]
fn collects_garbage() {
let prev_hash = H256::random();
let best_hash = H256::random();
let mut consensus = Consensus::<Block>::new();
let now = Instant::now();
let m1_hash = H256::random();
let m2_hash = H256::random();
let m1 = GenericMessage::BftMessage(message::LocalizedBftMessage {
parent_hash: prev_hash,
message: message::generic::BftMessage::Auxiliary(Justification {
round_number: 0,
hash: Default::default(),
signatures: Default::default(),
}),
});
let m2 = GenericMessage::BftMessage(message::LocalizedBftMessage {
parent_hash: best_hash,
message: message::generic::BftMessage::Auxiliary(Justification {
round_number: 0,
hash: Default::default(),
signatures: Default::default(),
}),
});
consensus.messages.push((m1_hash, now, m1));
consensus.messages.push((m2_hash, now, m2.clone()));
consensus.message_hashes.insert(m1_hash);
consensus.message_hashes.insert(m2_hash);
// nothing to collect
consensus.collect_garbage(None);
assert_eq!(consensus.messages.len(), 2);
assert_eq!(consensus.message_hashes.len(), 2);
// random header, nothing should be cleared
let mut header = Header {
parent_hash: H256::default(),
number: 0,
state_root: H256::default(),
extrinsics_root: H256::default(),
digest: Default::default(),
};
consensus.collect_garbage(Some(&header));
assert_eq!(consensus.messages.len(), 2);
assert_eq!(consensus.message_hashes.len(), 2);
// header that matches one of the messages
header.parent_hash = prev_hash;
consensus.collect_garbage(Some(&header));
assert_eq!(consensus.messages.len(), 1);
assert_eq!(consensus.message_hashes.len(), 1);
assert!(consensus.message_hashes.contains(&m2_hash));
// make timestamp expired
consensus.messages.clear();
consensus.messages.push((m2_hash, now - MESSAGE_LIFETIME, m2));
consensus.collect_garbage(None);
assert!(consensus.messages.is_empty());
assert!(consensus.message_hashes.is_empty());
}
}
substrate/substrate/network/src/consensus_gossip.rs
+377
View File
@@ -0,0 +1,377 @@
// Copyright 2017 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.?
//! Utility for gossip of network messages between authorities.
//! Handles chain-specific and standard BFT messages.
use std::collections::{HashMap, HashSet};
use futures::sync::mpsc;
use std::time::{Instant, Duration};
use network::PeerId;
use runtime_primitives::traits::{Block as BlockT, Header as HeaderT};
use runtime_primitives::generic::BlockId;
use message::{self, generic::Message as GenericMessage};
use protocol::Context;
// TODO: Add additional spam/DoS attack protection.
const MESSAGE_LIFETIME: Duration = Duration::from_secs(600);
struct PeerConsensus<H> {
known_messages: HashSet<H>,
}
/// Consensus messages.
#[derive(Debug, Clone, PartialEq)]
pub enum ConsensusMessage<B: BlockT> {
/// A message concerning BFT agreement
Bft(message::LocalizedBftMessage<B>),
/// A message concerning some chain-specific aspect of consensus
ChainSpecific(Vec<u8>, B::Hash),
}
struct MessageEntry<B: BlockT> {
hash: B::Hash,
message: ConsensusMessage<B>,
instant: Instant,
}
/// Consensus network protocol handler. Manages statements and candidate requests.
pub struct ConsensusGossip<B: BlockT> {
peers: HashMap<PeerId, PeerConsensus<B::Hash>>,
message_sink: Option<(mpsc::UnboundedSender<ConsensusMessage<B>>, B::Hash)>,
messages: Vec<MessageEntry<B>>,
message_hashes: HashSet<B::Hash>,
}
impl<B: BlockT> ConsensusGossip<B> where B::Header: HeaderT<Number=u64> {
/// Create a new instance.
pub fn new() -> Self {
ConsensusGossip {
peers: HashMap::new(),
message_sink: None,
messages: Default::default(),
message_hashes: Default::default(),
}
}
/// Closes all notification streams.
pub fn abort(&mut self) {
self.message_sink = None;
}
/// Handle new connected peer.
pub fn new_peer(&mut self, protocol: &mut Context<B>, peer_id: PeerId, roles: &[message::Role]) {
if roles.iter().any(|r| *r == message::Role::Authority) {
trace!(target:"gossip", "Registering authority {}", peer_id);
// Send out all known messages.
// TODO: limit by size
let mut known_messages = HashSet::new();
for entry in self.messages.iter() {
known_messages.insert(entry.hash);
let message = match entry.message {
ConsensusMessage::Bft(ref bft) => GenericMessage::BftMessage(bft.clone()),
ConsensusMessage::ChainSpecific(ref msg, _) => GenericMessage::ChainSpecific(msg.clone()),
};
protocol.send_message(peer_id, message);
}
self.peers.insert(peer_id, PeerConsensus {
known_messages,
});
}
}
fn propagate(&mut self, protocol: &mut Context<B>, message: message::Message<B>, hash: B::Hash) {
for (id, ref mut peer) in self.peers.iter_mut() {
if peer.known_messages.insert(hash.clone()) {
protocol.send_message(*id, message.clone());
}
}
}
fn register_message(&mut self, hash: B::Hash, message: ConsensusMessage<B>) {
if self.message_hashes.insert(hash) {
self.messages.push(MessageEntry {
hash,
instant: Instant::now(),
message,
});
}
}
/// Handles incoming BFT message, passing to stream and repropagating.
pub fn on_bft_message(&mut self, protocol: &mut Context<B>, peer_id: PeerId, message: message::LocalizedBftMessage<B>) {
if let Some((hash, message)) = self.handle_incoming(protocol, peer_id, ConsensusMessage::Bft(message)) {
// propagate to other peers.
self.multicast(protocol, message, Some(hash));
}
}
/// Handles incoming chain-specific message and repropagates
pub fn on_chain_specific(&mut self, protocol: &mut Context<B>, peer_id: PeerId, message: Vec<u8>, parent_hash: B::Hash) {
if let Some((hash, message)) = self.handle_incoming(protocol, peer_id, ConsensusMessage::ChainSpecific(message, parent_hash)) {
// propagate to other peers.
self.multicast(protocol, message, Some(hash));
}
}
/// Get a stream of messages relevant to consensus on top of a given parent hash.
pub fn messages_for(&mut self, parent_hash: B::Hash) -> mpsc::UnboundedReceiver<ConsensusMessage<B>> {
let (sink, stream) = mpsc::unbounded();
for entry in self.messages.iter() {
let message_matches = match entry.message {
ConsensusMessage::Bft(ref msg) => msg.parent_hash == parent_hash,
ConsensusMessage::ChainSpecific(_, ref h) => h == &parent_hash,
};
if message_matches {
sink.unbounded_send(entry.message.clone()).expect("receiving end known to be open; qed");
}
}
self.message_sink = Some((sink, parent_hash));
stream
}
/// Multicast a chain-specific message to other authorities.
pub fn multicast_chain_specific(&mut self, protocol: &mut Context<B>, message: Vec<u8>, parent_hash: B::Hash) {
trace!(target:"gossip", "sending chain-specific message");
self.multicast(protocol, ConsensusMessage::ChainSpecific(message, parent_hash), None);
}
/// Multicast a BFT message to other authorities
pub fn multicast_bft_message(&mut self, protocol: &mut Context<B>, message: message::LocalizedBftMessage<B>) {
// Broadcast message to all authorities.
trace!(target:"gossip", "Broadcasting BFT message {:?}", message);
self.multicast(protocol, ConsensusMessage::Bft(message), None);
}
/// Call when a peer has been disconnected to stop tracking gossip status.
pub fn peer_disconnected(&mut self, _protocol: &mut Context<B>, peer_id: PeerId) {
self.peers.remove(&peer_id);
}
/// Prune old or no longer relevant consensus messages.
/// Supply an optional block hash where consensus is known to have concluded.
pub fn collect_garbage(&mut self, best_hash: Option<&B::Hash>) {
let hashes = &mut self.message_hashes;
let before = self.messages.len();
let now = Instant::now();
self.messages.retain(|entry| {
if entry.instant + MESSAGE_LIFETIME >= now &&
best_hash.map_or(true, |parent_hash|
match entry.message {
ConsensusMessage::Bft(ref msg) => &msg.parent_hash != parent_hash,
ConsensusMessage::ChainSpecific(_, ref h) => h != parent_hash,
})
{
true
} else {
hashes.remove(&entry.hash);
false
}
});
if self.messages.len() != before {
trace!(target:"gossip", "Cleaned up {} stale messages", before - self.messages.len());
}
for (_, ref mut peer) in self.peers.iter_mut() {
peer.known_messages.retain(|h| hashes.contains(h));
}
}
fn handle_incoming(&mut self, protocol: &mut Context<B>, peer_id: PeerId, message: ConsensusMessage<B>) -> Option<(B::Hash, ConsensusMessage<B>)> {
let (hash, parent, message) = match message {
ConsensusMessage::Bft(msg) => {
let parent = msg.parent_hash;
let generic = GenericMessage::BftMessage(msg);
(
::protocol::hash_message(&generic),
parent,
match generic {
GenericMessage::BftMessage(msg) => ConsensusMessage::Bft(msg),
_ => panic!("`generic` is known to be the `BftMessage` variant; qed"),
}
)
}
ConsensusMessage::ChainSpecific(msg, parent) => {
let generic = GenericMessage::ChainSpecific(msg);
(
::protocol::hash_message::<B>(&generic),
parent,
match generic {
GenericMessage::ChainSpecific(msg) => ConsensusMessage::ChainSpecific(msg, parent),
_ => panic!("`generic` is known to be the `ChainSpecific` variant; qed"),
}
)
}
};
if self.message_hashes.contains(&hash) {
trace!(target:"gossip", "Ignored already known message from {}", peer_id);
return None;
}
match (protocol.client().info(), protocol.client().header(&BlockId::Hash(parent))) {
(_, Err(e)) | (Err(e), _) => {
debug!(target:"gossip", "Error reading blockchain: {:?}", e);
return None;
},
(Ok(info), Ok(Some(header))) => {
if header.number() < &info.chain.best_number {
trace!(target:"gossip", "Ignored ancient message from {}, hash={}", peer_id, parent);
return None;
}
},
(Ok(_), Ok(None)) => {},
}
if let Some(ref mut peer) = self.peers.get_mut(&peer_id) {
peer.known_messages.insert(hash);
if let Some((sink, parent_hash)) = self.message_sink.take() {
if parent == parent_hash {
if let Err(e) = sink.unbounded_send(message.clone()) {
trace!(target:"gossip", "Error broadcasting message notification: {:?}", e);
}
}
self.message_sink = Some((sink, parent_hash));
}
} else {
trace!(target:"gossip", "Ignored statement from unregistered peer {}", peer_id);
return None;
}
Some((hash, message))
}
fn multicast(&mut self, protocol: &mut Context<B>, message: ConsensusMessage<B>, hash: Option<B::Hash>) {
let generic = match message {
ConsensusMessage::Bft(ref message) => GenericMessage::BftMessage(message.clone()),
ConsensusMessage::ChainSpecific(ref message, _) => GenericMessage::ChainSpecific(message.clone()),
};
let hash = hash.unwrap_or_else(|| ::protocol::hash_message(&generic));
self.register_message(hash, message);
self.propagate(protocol, generic, hash);
}
}
#[cfg(test)]
mod tests {
use runtime_primitives::bft::Justification;
use runtime_primitives::testing::{H256, Header, Block as RawBlock};
use std::time::Instant;
use message::{self, generic};
use super::*;
type Block = RawBlock<u64>;
#[test]
fn collects_garbage() {
let prev_hash = H256::random();
let best_hash = H256::random();
let mut consensus = ConsensusGossip::<Block>::new();
let now = Instant::now();
let m1_hash = H256::random();
let m2_hash = H256::random();
let m1 = ConsensusMessage::Bft(message::LocalizedBftMessage {
parent_hash: prev_hash,
message: message::generic::BftMessage::Auxiliary(Justification {
round_number: 0,
hash: Default::default(),
signatures: Default::default(),
}),
});
let m2 = ConsensusMessage::ChainSpecific(vec![1, 2, 3], best_hash);
macro_rules! push_msg {
($hash:expr, $now: expr, $m:expr) => {
consensus.messages.push(MessageEntry {
hash: $hash,
instant: $now,
message: $m,
})
}
}
push_msg!(m1_hash, now, m1);
push_msg!(m2_hash, now, m2.clone());
consensus.message_hashes.insert(m1_hash);
consensus.message_hashes.insert(m2_hash);
// nothing to collect
consensus.collect_garbage(None);
assert_eq!(consensus.messages.len(), 2);
assert_eq!(consensus.message_hashes.len(), 2);
// random header, nothing should be cleared
let mut header = Header {
parent_hash: H256::default(),
number: 0,
state_root: H256::default(),
extrinsics_root: H256::default(),
digest: Default::default(),
};
consensus.collect_garbage(Some(&H256::default()));
assert_eq!(consensus.messages.len(), 2);
assert_eq!(consensus.message_hashes.len(), 2);
// header that matches one of the messages
header.parent_hash = prev_hash;
consensus.collect_garbage(Some(&prev_hash));
assert_eq!(consensus.messages.len(), 1);
assert_eq!(consensus.message_hashes.len(), 1);
assert!(consensus.message_hashes.contains(&m2_hash));
// make timestamp expired
consensus.messages.clear();
push_msg!(m2_hash, now - MESSAGE_LIFETIME, m2);
consensus.collect_garbage(None);
assert!(consensus.messages.is_empty());
assert!(consensus.message_hashes.is_empty());
}
#[test]
fn message_stream_include_those_sent_before_asking_for_stream() {
use futures::Stream;
let mut consensus = ConsensusGossip::new();
let bft_message = generic::BftMessage::Consensus(generic::SignedConsensusMessage::Vote(generic::SignedConsensusVote {
vote: generic::ConsensusVote::AdvanceRound(0),
sender: [0; 32].into(),
signature: Default::default(),
}));
let parent_hash = [1; 32].into();
let localized = ::message::LocalizedBftMessage::<Block> {
message: bft_message,
parent_hash: parent_hash,
};
let message = generic::Message::BftMessage(localized.clone());
let message_hash = ::protocol::hash_message::<Block>(&message);
let message = ConsensusMessage::Bft(localized);
consensus.register_message(message_hash, message.clone());
let stream = consensus.messages_for(parent_hash);
assert_eq!(stream.wait().next(), Some(Ok(message)));
}
}
substrate/substrate/network/src/lib.rs
+9 -7
View File
@@ -16,8 +16,8 @@
#![warn(missing_docs)]
//! Implements polkadot protocol version as specified here:
//! https://github.com/paritytech/polkadot/wiki/Network-protocol
//! Substrate-specific P2P networking: synchronizing blocks, propagating BFT messages.
//! Allows attachment of an optional subprotocol for chain-specific requests.
extern crate ethcore_network_devp2p as network_devp2p;
extern crate ethcore_network as network;
@@ -50,22 +50,24 @@ mod service;
mod sync;
mod protocol;
mod io;
mod message;
mod config;
mod chain;
mod blocks;
mod consensus;
mod on_demand;
pub mod consensus_gossip;
pub mod error;
pub mod message;
pub mod specialization;
#[cfg(test)] mod test;
pub use chain::Client as ClientHandle;
pub use service::{Service, FetchFuture, ConsensusService, BftMessageStream,
TransactionPool, Params, ManageNetwork, SyncProvider};
pub use protocol::{ProtocolStatus};
pub use protocol::{ProtocolStatus, PeerInfo, Context};
pub use sync::{Status as SyncStatus, SyncState};
pub use network::{NonReservedPeerMode, NetworkConfiguration, ConnectionFilter, ConnectionDirection};
pub use message::{generic as generic_message, BftMessage, LocalizedBftMessage, ConsensusVote, SignedConsensusVote, SignedConsensusMessage, SignedConsensusProposal};
pub use network::{NonReservedPeerMode, NetworkConfiguration, PeerId, ProtocolId, ConnectionFilter, ConnectionDirection};
pub use message::{generic as generic_message, RequestId, BftMessage, LocalizedBftMessage, ConsensusVote, SignedConsensusVote, SignedConsensusMessage, SignedConsensusProposal, Status as StatusMessage};
pub use error::Error;
pub use config::{Role, ProtocolConfig};
pub use on_demand::{OnDemand, OnDemandService, RemoteCallResponse};
substrate/substrate/network/src/message.rs
+16 -23
View File
@@ -1,18 +1,18 @@
// Copyright 2017 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// This file is part of Substrate.
// Polkadot is free software: you can redistribute it and/or modify
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.?
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
//! Network packet message types. These get serialized and put into the lower level protocol payload.
@@ -21,6 +21,7 @@ use service::Role as RoleFlags;
pub use self::generic::{BlockAnnounce, RemoteCallRequest, ConsensusVote, SignedConsensusVote, FromBlock, Body};
/// A unique ID of a request.
pub type RequestId = u64;
/// Type alias for using the message type using block type parameters.
@@ -88,14 +89,12 @@ pub type Transactions<E> = Vec<E>;
/// Configured node role.
#[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)]
pub enum Role {
/// Full relay chain client with no additional responsibilities.
/// Full node with no additional responsibilities.
Full,
/// Relay chain light client.
/// Light client.
Light,
/// Parachain validator.
Validator,
/// Parachain collator.
Collator,
Authority,
}
impl Role {
@@ -106,8 +105,7 @@ impl Role {
match *r {
Role::Full => flags = flags | RoleFlags::FULL,
Role::Light => flags = flags | RoleFlags::LIGHT,
Role::Validator => flags = flags | RoleFlags::VALIDATOR,
Role::Collator => flags = flags | RoleFlags::COLLATOR,
Role::Authority => flags = flags | RoleFlags::AUTHORITY,
}
}
flags
@@ -123,11 +121,8 @@ impl From<RoleFlags> for Vec<Role> where {
if !(flags & RoleFlags::LIGHT).is_empty() {
roles.push(Role::Light);
}
if !(flags & RoleFlags::VALIDATOR).is_empty() {
roles.push(Role::Validator);
}
if !(flags & RoleFlags::COLLATOR).is_empty() {
roles.push(Role::Collator);
if !(flags & RoleFlags::AUTHORITY).is_empty() {
roles.push(Role::Authority);
}
roles
}
@@ -157,8 +152,8 @@ pub enum Direction {
Descending,
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)]
/// Remote call response.
#[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)]
pub struct RemoteCallResponse {
/// Id of a request this response was made for.
pub id: RequestId,
@@ -358,6 +353,8 @@ pub mod generic {
RemoteCallRequest(RemoteCallRequest<Hash>),
/// Remote method call response.
RemoteCallResponse(RemoteCallResponse),
/// Chain-specific message
ChainSpecific(Vec<u8>),
}
/// Status sent on connection.
@@ -373,12 +370,8 @@ pub mod generic {
pub best_hash: Hash,
/// Genesis block hash.
pub genesis_hash: Hash,
/// Signatue of `best_hash` made with validator address. Required for the validator role.
pub validator_signature: Option<ed25519::Signature>,
/// Validator address. Required for the validator role.
pub validator_id: Option<AuthorityId>,
/// Parachain id. Required for the collator role.
pub parachain_id: Option<u64>,
/// Chain-specific status.
pub chain_status: Vec<u8>,
}
/// Request block data from a peer.
substrate/substrate/network/src/on_demand.rs
+6 -11
View File
@@ -169,7 +169,7 @@ impl<B, E> OnDemandService<B> for OnDemand<B, E> where
B::Header: HeaderT<Number=u64>,
{
fn on_connect(&self, peer: PeerId, role: service::Role) {
if !role.intersects(service::Role::FULL | service::Role::COLLATOR | service::Role::VALIDATOR) { // TODO: correct?
if !role.intersects(service::Role::FULL | service::Role::AUTHORITY) { // TODO: correct?
return;
}
@@ -296,9 +296,7 @@ impl<B, E> OnDemandCore<B, E> where
request.timestamp = Instant::now();
trace!(target: "sync", "Dispatching remote request {} to peer {}", request.id, peer);
service.execute_in_context(|ctx, protocol| {
protocol.send_message(ctx, peer, request.message())
});
service.execute_in_context(|ctx| ctx.send_message(peer, request.message()));
self.active_peers.insert(peer, request);
}
}
@@ -326,10 +324,8 @@ mod tests {
use parking_lot::RwLock;
use client;
use client::light::fetcher::{Fetcher, FetchChecker, RemoteCallRequest};
use io::NetSyncIo;
use message;
use network::PeerId;
use protocol::Protocol;
use service::{Role, ExecuteInContext};
use test::TestIo;
use super::{REQUEST_TIMEOUT, OnDemand, OnDemandService};
@@ -339,7 +335,7 @@ mod tests {
struct DummyFetchChecker { ok: bool }
impl ExecuteInContext<Block> for DummyExecutor {
fn execute_in_context<F: Fn(&mut NetSyncIo, &Protocol<Block>)>(&self, _closure: F) {}
fn execute_in_context<F: Fn(&mut ::protocol::Context<Block>)>(&self, _closure: F) {}
}
impl FetchChecker<Block> for DummyFetchChecker {
@@ -378,9 +374,8 @@ mod tests {
let (_, on_demand) = dummy(true);
on_demand.on_connect(0, Role::LIGHT);
on_demand.on_connect(1, Role::FULL);
on_demand.on_connect(2, Role::COLLATOR);
on_demand.on_connect(3, Role::VALIDATOR);
assert_eq!(vec![1, 2, 3], on_demand.core.lock().idle_peers.iter().cloned().collect::<Vec<_>>());
on_demand.on_connect(2, Role::AUTHORITY);
assert_eq!(vec![1, 2], on_demand.core.lock().idle_peers.iter().cloned().collect::<Vec<_>>());
}
#[test]
@@ -467,4 +462,4 @@ mod tests {
receive_call_response(&*on_demand, &mut network, 0, 0);
thread.join().unwrap();
}
}
}
substrate/substrate/network/src/protocol.rs
+200 -109
View File
@@ -18,7 +18,7 @@ use std::collections::{HashMap, HashSet};
use std::{mem, cmp};
use std::sync::Arc;
use std::time;
use parking_lot::{RwLock, Mutex};
use parking_lot::RwLock;
use serde_json;
use runtime_primitives::traits::{Block as BlockT, Header as HeaderT, Hashing, HashingFor};
use runtime_primitives::generic::BlockId;
@@ -26,9 +26,9 @@ use network::PeerId;
use message::{self, Message};
use message::generic::Message as GenericMessage;
use specialization::Specialization;
use sync::{ChainSync, Status as SyncStatus, SyncState};
use consensus::Consensus;
use service::{Role, TransactionPool, BftMessageStream};
use service::{Role, TransactionPool};
use config::ProtocolConfig;
use chain::Client;
use on_demand::OnDemandService;
@@ -36,26 +36,28 @@ use io::SyncIo;
use error;
const REQUEST_TIMEOUT_SEC: u64 = 40;
const PROTOCOL_VERSION: u32 = 0;
/// Current protocol version.
pub (crate) const CURRENT_VERSION: u32 = 1;
/// Current packet count.
pub (crate) const CURRENT_PACKET_COUNT: u8 = 1;
// Maximum allowed entries in `BlockResponse`
const MAX_BLOCK_DATA_RESPONSE: u32 = 128;
// Lock must always be taken in order declared here.
pub struct Protocol<B: BlockT> {
pub struct Protocol<B: BlockT, S: Specialization<B>> {
config: ProtocolConfig,
chain: Arc<Client<B>>,
on_demand: Option<Arc<OnDemandService<B>>>,
genesis_hash: B::Hash,
sync: RwLock<ChainSync<B>>,
consensus: Mutex<Consensus<B>>,
// All connected peers
peers: RwLock<HashMap<PeerId, Peer<B>>>,
specialization: RwLock<S>,
context_data: ContextData<B>,
// Connected peers pending Status message.
handshaking_peers: RwLock<HashMap<PeerId, time::Instant>>,
transaction_pool: Arc<TransactionPool<B>>,
}
/// Syncing status and statistics
#[derive(Clone)]
pub struct ProtocolStatus<B: BlockT> {
@@ -82,13 +84,14 @@ struct Peer<B: BlockT> {
/// Request timestamp
request_timestamp: Option<time::Instant>,
/// Holds a set of transactions known to this peer.
known_transactions: HashSet<B::Hash>,
known_extrinsics: HashSet<B::Hash>,
/// Holds a set of blocks known to this peer.
known_blocks: HashSet<B::Hash>,
/// Request counter,
next_request_id: message::RequestId,
}
/// Info about a peer's known state.
#[derive(Debug)]
pub struct PeerInfo<B: BlockT> {
/// Roles
@@ -101,36 +104,128 @@ pub struct PeerInfo<B: BlockT> {
pub best_number: <B::Header as HeaderT>::Number,
}
impl<B: BlockT> Protocol<B> where
B::Header: HeaderT<Number=u64>
{
/// Context for a network-specific handler.
pub trait Context<B: BlockT> {
/// Get a reference to the client.
fn client(&self) -> &::chain::Client<B>;
/// Disable a peer
fn disable_peer(&mut self, peer_id: PeerId);
/// Disconnect peer
fn disconnect_peer(&mut self, peer_id: PeerId);
/// Get peer info.
fn peer_info(&self, peer: PeerId) -> Option<PeerInfo<B>>;
/// Send a message to a peer.
fn send_message(&mut self, peer_id: PeerId, data: ::message::Message<B>);
}
/// Protocol context.
pub(crate) struct ProtocolContext<'a, B: 'a + BlockT> {
io: &'a mut SyncIo,
context_data: &'a ContextData<B>,
}
impl<'a, B: BlockT + 'a> ProtocolContext<'a, B> {
pub(crate) fn new(context_data: &'a ContextData<B>, io: &'a mut SyncIo) -> Self {
ProtocolContext {
io,
context_data,
}
}
/// Send a message to a peer.
pub fn send_message(&mut self, peer_id: PeerId, message: Message<B>) {
send_message(&self.context_data.peers, self.io, peer_id, message)
}
pub fn disable_peer(&mut self, peer_id: PeerId) {
self.io.disable_peer(peer_id);
}
pub fn disconnect_peer(&mut self, peer_id: PeerId) {
self.io.disconnect_peer(peer_id)
}
/// Get peer info.
pub fn peer_info(&self, peer: PeerId) -> Option<PeerInfo<B>> {
self.context_data.peers.read().get(&peer).map(|p| {
PeerInfo {
roles: p.roles,
protocol_version: p.protocol_version,
best_hash: p.best_hash,
best_number: p.best_number,
}
})
}
}
impl<'a, B: BlockT + 'a> Context<B> for ProtocolContext<'a, B> {
fn send_message(&mut self, peer_id: PeerId, message: Message<B>) {
ProtocolContext::send_message(self, peer_id, message);
}
fn disable_peer(&mut self, peer_id: PeerId) {
ProtocolContext::disable_peer(self, peer_id);
}
fn disconnect_peer(&mut self, peer_id: PeerId) {
ProtocolContext::disconnect_peer(self, peer_id);
}
fn peer_info(&self, peer_id: PeerId) -> Option<PeerInfo<B>> {
ProtocolContext::peer_info(self, peer_id)
}
fn client(&self) -> &Client<B> {
&*self.context_data.chain
}
}
/// Data necessary to create a context.
pub(crate) struct ContextData<B: BlockT> {
// All connected peers
peers: RwLock<HashMap<PeerId, Peer<B>>>,
chain: Arc<Client<B>>,
}
impl<B: BlockT, S: Specialization<B>> Protocol<B, S> where B::Header: HeaderT<Number=u64> {
/// Create a new instance.
pub fn new(
config: ProtocolConfig,
chain: Arc<Client<B>>,
on_demand: Option<Arc<OnDemandService<B>>>,
transaction_pool: Arc<TransactionPool<B>>
transaction_pool: Arc<TransactionPool<B>>,
specialization: S,
) -> error::Result<Self> {
let info = chain.info()?;
let sync = ChainSync::new(config.roles, &info);
let protocol = Protocol {
config: config,
chain: chain,
on_demand: on_demand,
context_data: ContextData {
peers: RwLock::new(HashMap::new()),
chain,
},
on_demand,
genesis_hash: info.chain.genesis_hash,
sync: RwLock::new(sync),
consensus: Mutex::new(Consensus::new()),
peers: RwLock::new(HashMap::new()),
specialization: RwLock::new(specialization),
handshaking_peers: RwLock::new(HashMap::new()),
transaction_pool: transaction_pool,
};
Ok(protocol)
}
pub(crate) fn context_data(&self) -> &ContextData<B> {
&self.context_data
}
/// Returns protocol status
pub fn status(&self) -> ProtocolStatus<B> {
let sync = self.sync.read();
let peers = self.peers.read();
let peers = self.context_data.peers.read();
ProtocolStatus {
sync: sync.status(),
num_peers: peers.values().count(),
@@ -154,7 +249,7 @@ impl<B: BlockT> Protocol<B> where
GenericMessage::BlockRequest(r) => self.on_block_request(io, peer_id, r),
GenericMessage::BlockResponse(r) => {
let request = {
let mut peers = self.peers.write();
let mut peers = self.context_data.peers.write();
if let Some(ref mut peer) = peers.get_mut(&peer_id) {
peer.request_timestamp = None;
match mem::replace(&mut peer.block_request, None) {
@@ -177,39 +272,16 @@ impl<B: BlockT> Protocol<B> where
}
self.on_block_response(io, peer_id, request, r);
},
GenericMessage::BlockAnnounce(announce) => {
self.on_block_announce(io, peer_id, announce);
},
GenericMessage::BftMessage(m) => self.on_bft_message(io, peer_id, m, HashingFor::<B>::hash(data)),
GenericMessage::Transactions(m) => self.on_transactions(io, peer_id, m),
GenericMessage::BlockAnnounce(announce) => self.on_block_announce(io, peer_id, announce),
GenericMessage::Transactions(m) => self.on_extrinsics(io, peer_id, m),
GenericMessage::RemoteCallRequest(request) => self.on_remote_call_request(io, peer_id, request),
GenericMessage::RemoteCallResponse(response) => self.on_remote_call_response(io, peer_id, response),
other => self.specialization.write().on_message(&mut ProtocolContext::new(&self.context_data, io), peer_id, other),
}
}
pub fn send_message(&self, io: &mut SyncIo, peer_id: PeerId, mut message: Message<B>) {
match &mut message {
&mut GenericMessage::BlockRequest(ref mut r) => {
let mut peers = self.peers.write();
if let Some(ref mut peer) = peers.get_mut(&peer_id) {
r.id = peer.next_request_id;
peer.next_request_id = peer.next_request_id + 1;
peer.block_request = Some(r.clone());
peer.request_timestamp = Some(time::Instant::now());
}
},
_ => (),
}
let data = serde_json::to_vec(&message).expect("Serializer is infallible; qed");
if let Err(e) = io.send(peer_id, data) {
debug!(target:"sync", "Error sending message: {:?}", e);
io.disconnect_peer(peer_id);
}
}
pub fn hash_message(message: &Message<B>) -> B::Hash {
let data = serde_json::to_vec(&message).expect("Serializer is infallible; qed");
HashingFor::<B>::hash(&data)
pub fn send_message(&self, io: &mut SyncIo, peer_id: PeerId, message: Message<B>) {
send_message::<B>(&self.context_data.peers, io, peer_id, message)
}
/// Called when a new peer is connected
@@ -222,15 +294,21 @@ impl<B: BlockT> Protocol<B> where
/// Called by peer when it is disconnecting
pub fn on_peer_disconnected(&self, io: &mut SyncIo, peer: PeerId) {
trace!(target: "sync", "Disconnecting {}: {}", peer, io.peer_info(peer));
// lock all the the peer lists so that add/remove peer events are in order
let mut sync = self.sync.write();
let mut spec = self.specialization.write();
let removed = {
let mut peers = self.peers.write();
let mut peers = self.context_data.peers.write();
let mut handshaking_peers = self.handshaking_peers.write();
handshaking_peers.remove(&peer);
peers.remove(&peer).is_some()
};
if removed {
self.consensus.lock().peer_disconnected(io, self, peer);
self.sync.write().peer_disconnected(io, self, peer);
let mut context = ProtocolContext::new(&self.context_data, io);
sync.peer_disconnected(&mut context, peer);
spec.on_disconnect(&mut context, peer);
self.on_demand.as_ref().map(|s| s.on_disconnect(peer));
}
}
@@ -254,17 +332,17 @@ impl<B: BlockT> Protocol<B> where
message::BlockAttribute::Justification => get_justification = true,
}
}
while let Some(header) = self.chain.header(&id).unwrap_or(None) {
while let Some(header) = self.context_data.chain.header(&id).unwrap_or(None) {
if blocks.len() >= max{
break;
}
let number = header.number().clone();
let hash = header.hash();
let justification = if get_justification { self.chain.justification(&BlockId::Hash(hash)).unwrap_or(None) } else { None };
let justification = if get_justification { self.context_data.chain.justification(&BlockId::Hash(hash)).unwrap_or(None) } else { None };
let block_data = message::generic::BlockData {
hash: hash,
header: if get_header { Some(header) } else { None },
body: (if get_body { self.chain.body(&BlockId::Hash(hash)).unwrap_or(None) } else { None }).map(|body| message::Body::Extrinsics(body)),
body: (if get_body { self.context_data.chain.body(&BlockId::Hash(hash)).unwrap_or(None) } else { None }).map(|body| message::Body::Extrinsics(body)),
receipt: None,
message_queue: None,
justification: justification.map(|j| message::generic::BlockJustification::V2(j)),
@@ -291,36 +369,20 @@ impl<B: BlockT> Protocol<B> where
fn on_block_response(&self, io: &mut SyncIo, peer: PeerId, request: message::BlockRequest<B>, response: message::BlockResponse<B>) {
// TODO: validate response
trace!(target: "sync", "BlockResponse {} from {} with {} blocks", response.id, peer, response.blocks.len());
self.sync.write().on_block_data(io, self, peer, request, response);
}
fn on_bft_message(&self, io: &mut SyncIo, peer: PeerId, message: message::LocalizedBftMessage<B>, hash: B::Hash) {
trace!(target: "sync", "BFT message from {}: {:?}", peer, message);
self.consensus.lock().on_bft_message(io, self, peer, message, hash);
}
/// See `ConsensusService` trait.
pub fn send_bft_message(&self, io: &mut SyncIo, message: message::LocalizedBftMessage<B>) {
self.consensus.lock().send_bft_message(io, self, message)
}
/// See `ConsensusService` trait.
pub fn bft_messages(&self, parent_hash: B::Hash) -> BftMessageStream<B> {
self.consensus.lock().bft_messages(parent_hash)
self.sync.write().on_block_data(&mut ProtocolContext::new(&self.context_data, io), peer, request, response);
}
/// Perform time based maintenance.
pub fn tick(&self, io: &mut SyncIo) {
self.maintain_peers(io);
self.on_demand.as_ref().map(|s| s.maintain_peers(io));
self.consensus.lock().collect_garbage(None);
}
fn maintain_peers(&self, io: &mut SyncIo) {
let tick = time::Instant::now();
let mut aborting = Vec::new();
{
let peers = self.peers.read();
let peers = self.context_data.peers.read();
let handshaking_peers = self.handshaking_peers.read();
for (peer_id, timestamp) in peers.iter()
.filter_map(|(id, peer)| peer.request_timestamp.as_ref().map(|r| (id, r)))
@@ -332,13 +394,15 @@ impl<B: BlockT> Protocol<B> where
}
}
}
self.specialization.write().maintain_peers(&mut ProtocolContext::new(&self.context_data, io));
for p in aborting {
self.on_peer_disconnected(io, p);
}
}
pub fn peer_info(&self, peer: PeerId) -> Option<PeerInfo<B>> {
self.peers.read().get(&peer).map(|p| {
self.context_data.peers.read().get(&peer).map(|p| {
PeerInfo {
roles: p.roles,
protocol_version: p.protocol_version,
@@ -357,7 +421,7 @@ impl<B: BlockT> Protocol<B> where
}
{
let mut peers = self.peers.write();
let mut peers = self.context_data.peers.write();
let mut handshaking_peers = self.handshaking_peers.write();
if peers.contains_key(&peer_id) {
debug!(target: "sync", "Unexpected status packet from {}:{}", peer_id, io.peer_info(peer_id));
@@ -368,7 +432,7 @@ impl<B: BlockT> Protocol<B> where
trace!(target: "sync", "Peer {} genesis hash mismatch (ours: {}, theirs: {})", peer_id, self.genesis_hash, status.genesis_hash);
return;
}
if status.version != PROTOCOL_VERSION {
if status.version != CURRENT_VERSION {
io.disable_peer(peer_id);
trace!(target: "sync", "Peer {} unsupported eth protocol ({})", peer_id, status.version);
return;
@@ -381,7 +445,7 @@ impl<B: BlockT> Protocol<B> where
best_number: status.best_number,
block_request: None,
request_timestamp: None,
known_transactions: HashSet::new(),
known_extrinsics: HashSet::new(),
known_blocks: HashSet::new(),
next_request_id: 0,
};
@@ -390,47 +454,48 @@ impl<B: BlockT> Protocol<B> where
debug!(target: "sync", "Connected {} {}", peer_id, io.peer_info(peer_id));
}
self.sync.write().new_peer(io, self, peer_id);
self.consensus.lock().new_peer(io, self, peer_id, &status.roles);
let mut context = ProtocolContext::new(&self.context_data, io);
self.sync.write().new_peer(&mut context, peer_id);
self.specialization.write().on_connect(&mut context, peer_id, status.clone());
self.on_demand.as_ref().map(|s| s.on_connect(peer_id, message::Role::as_flags(&status.roles)));
}
/// Called when peer sends us new transactions
fn on_transactions(&self, _io: &mut SyncIo, peer_id: PeerId, transactions: message::Transactions<B::Extrinsic>) {
// Accept transactions only when fully synced
/// Called when peer sends us new extrinsics
fn on_extrinsics(&self, _io: &mut SyncIo, peer_id: PeerId, extrinsics: message::Transactions<B::Extrinsic>) {
// Accept extrinsics only when fully synced
if self.sync.read().status().state != SyncState::Idle {
trace!(target: "sync", "{} Ignoring transactions while syncing", peer_id);
trace!(target: "sync", "{} Ignoring extrinsics while syncing", peer_id);
return;
}
trace!(target: "sync", "Received {} transactions from {}", transactions.len(), peer_id);
let mut peers = self.peers.write();
trace!(target: "sync", "Received {} extrinsics from {}", extrinsics.len(), peer_id);
let mut peers = self.context_data.peers.write();
if let Some(ref mut peer) = peers.get_mut(&peer_id) {
for t in transactions {
for t in extrinsics {
if let Some(hash) = self.transaction_pool.import(&t) {
peer.known_transactions.insert(hash);
peer.known_extrinsics.insert(hash);
}
}
}
}
/// Called when we propagate ready transactions to peers.
pub fn propagate_transactions(&self, io: &mut SyncIo) {
debug!(target: "sync", "Propagating transactions");
/// Called when we propagate ready extrinsics to peers.
pub fn propagate_extrinsics(&self, io: &mut SyncIo) {
debug!(target: "sync", "Propagating extrinsics");
// Accept transactions only when fully synced
if self.sync.read().status().state != SyncState::Idle {
return;
}
let transactions = self.transaction_pool.transactions();
let extrinsics = self.transaction_pool.transactions();
let mut propagated_to = HashMap::new();
let mut peers = self.peers.write();
let mut peers = self.context_data.peers.write();
for (peer_id, ref mut peer) in peers.iter_mut() {
let (hashes, to_send): (Vec<_>, Vec<_>) = transactions
let (hashes, to_send): (Vec<_>, Vec<_>) = extrinsics
.iter()
.cloned()
.filter(|&(hash, _)| peer.known_transactions.insert(hash))
.filter(|&(hash, _)| peer.known_extrinsics.insert(hash))
.unzip();
if !to_send.is_empty() {
@@ -453,16 +518,14 @@ impl<B: BlockT> Protocol<B> where
/// Send Status message
fn send_status(&self, io: &mut SyncIo, peer_id: PeerId) {
if let Ok(info) = self.chain.info() {
if let Ok(info) = self.context_data.chain.info() {
let status = message::generic::Status {
version: PROTOCOL_VERSION,
version: CURRENT_VERSION,
genesis_hash: info.chain.genesis_hash,
roles: self.config.roles.into(),
best_number: info.chain.best_number,
best_hash: info.chain.best_hash,
validator_signature: None,
validator_id: None,
parachain_id: None,
chain_status: self.specialization.read().status(),
};
self.send_message(io, peer_id, GenericMessage::Status(status))
}
@@ -470,24 +533,25 @@ impl<B: BlockT> Protocol<B> where
pub fn abort(&self) {
let mut sync = self.sync.write();
let mut peers = self.peers.write();
let mut spec = self.specialization.write();
let mut peers = self.context_data.peers.write();
let mut handshaking_peers = self.handshaking_peers.write();
sync.clear();
spec.on_abort();
peers.clear();
handshaking_peers.clear();
self.consensus.lock().restart();
}
pub fn on_block_announce(&self, io: &mut SyncIo, peer_id: PeerId, announce: message::BlockAnnounce<B::Header>) {
let header = announce.header;
let hash = header.hash();
{
let mut peers = self.peers.write();
let mut peers = self.context_data.peers.write();
if let Some(ref mut peer) = peers.get_mut(&peer_id) {
peer.known_blocks.insert(hash.clone());
}
}
self.sync.write().on_block_announce(io, self, peer_id, hash, &header);
self.sync.write().on_block_announce(&mut ProtocolContext::new(&self.context_data, io), peer_id, hash, &header);
}
pub fn on_block_imported(&self, io: &mut SyncIo, hash: B::Hash, header: &B::Header) {
@@ -499,7 +563,7 @@ impl<B: BlockT> Protocol<B> where
}
// send out block announcements
let mut peers = self.peers.write();
let mut peers = self.context_data.peers.write();
for (peer_id, ref mut peer) in peers.iter_mut() {
if peer.known_blocks.insert(hash.clone()) {
@@ -509,13 +573,11 @@ impl<B: BlockT> Protocol<B> where
}));
}
}
self.consensus.lock().collect_garbage(Some(&header));
}
fn on_remote_call_request(&self, io: &mut SyncIo, peer_id: PeerId, request: message::RemoteCallRequest<B::Hash>) {
trace!(target: "sync", "Remote call request {} from {} ({} at {})", request.id, peer_id, request.method, request.block);
let proof = match self.chain.execution_proof(&request.block, &request.method, &request.data) {
let proof = match self.context_data.chain.execution_proof(&request.block, &request.method, &request.data) {
Ok((_, proof)) => proof,
Err(error) => {
trace!(target: "sync", "Remote call request {} from {} ({} at {}) failed with: {}",
@@ -534,7 +596,36 @@ impl<B: BlockT> Protocol<B> where
self.on_demand.as_ref().map(|s| s.on_remote_call_response(io, peer_id, response));
}
pub fn chain(&self) -> &Client<B> {
&*self.chain
/// Execute a closure with access to a network context and specialization.
pub fn with_spec<F, U>(&self, io: &mut SyncIo, f: F) -> U
where F: FnOnce(&mut S, &mut Context<B>) -> U
{
f(&mut* self.specialization.write(), &mut ProtocolContext::new(&self.context_data, io))
}
}
fn send_message<B: BlockT>(peers: &RwLock<HashMap<PeerId, Peer<B>>>, io: &mut SyncIo, peer_id: PeerId, mut message: Message<B>) {
match &mut message {
&mut GenericMessage::BlockRequest(ref mut r) => {
let mut peers = peers.write();
if let Some(ref mut peer) = peers.get_mut(&peer_id) {
r.id = peer.next_request_id;
peer.next_request_id = peer.next_request_id + 1;
peer.block_request = Some(r.clone());
peer.request_timestamp = Some(time::Instant::now());
}
},
_ => (),
}
let data = serde_json::to_vec(&message).expect("Serializer is infallible; qed");
if let Err(e) = io.send(peer_id, data) {
debug!(target:"sync", "Error sending message: {:?}", e);
io.disconnect_peer(peer_id);
}
}
/// Hash a message.
pub(crate) fn hash_message<B: BlockT>(message: &Message<B>) -> B::Hash {
let data = serde_json::to_vec(&message).expect("Serializer is infallible; qed");
HashingFor::<B>::hash(&data)
}
substrate/substrate/network/src/service.rs
+55 -54
View File
@@ -1,18 +1,18 @@
// Copyright 2017 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.?
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
use std::collections::HashMap;
use std::sync::Arc;
@@ -24,19 +24,15 @@ NetworkConfiguration , NonReservedPeerMode, ErrorKind};
use network_devp2p::{NetworkService};
use core_io::{TimerToken};
use io::NetSyncIo;
use protocol::{Protocol, ProtocolStatus, PeerInfo as ProtocolPeerInfo};
use protocol::{Protocol, ProtocolContext, Context, ProtocolStatus, PeerInfo as ProtocolPeerInfo};
use config::{ProtocolConfig};
use error::Error;
use chain::Client;
use message::LocalizedBftMessage;
use specialization::Specialization;
use on_demand::OnDemandService;
use runtime_primitives::traits::{Block as BlockT, Header as HeaderT};
/// Polkadot devp2p protocol id
pub const DOT_PROTOCOL_ID: ProtocolId = *b"dot";
const V0_PACKET_COUNT: u8 = 1;
/// Type that represents fetch completion future.
pub type FetchFuture = oneshot::Receiver<Vec<u8>>;
/// Type that represents bft messages stream.
@@ -53,14 +49,12 @@ bitflags! {
pub struct Role: u32 {
/// No network.
const NONE = 0b00000000;
/// Full node, doe not participate in consensus.
/// Full node, does not participate in consensus.
const FULL = 0b00000001;
/// Light client node.
const LIGHT = 0b00000010;
/// Act as a validator.
const VALIDATOR = 0b00000100;
/// Act as a collator.
const COLLATOR = 0b00001000;
/// Act as an authority
const AUTHORITY = 0b00000100;
}
}
@@ -99,12 +93,12 @@ pub trait ConsensusService<B: BlockT>: Send + Sync {
/// Service able to execute closure in the network context.
pub trait ExecuteInContext<B: BlockT>: Send + Sync {
/// Execute closure in network context.
fn execute_in_context<F: Fn(&mut NetSyncIo, &Protocol<B>)>(&self, closure: F);
fn execute_in_context<F: Fn(&mut Context<B>)>(&self, closure: F);
}
/// devp2p Protocol handler
struct ProtocolHandler<B: BlockT> {
protocol: Protocol<B>,
struct ProtocolHandler<B: BlockT, S: Specialization<B>> {
protocol: Protocol<B, S>,
}
/// Peer connection information
@@ -125,7 +119,7 @@ pub struct PeerInfo<B: BlockT> {
}
/// Service initialization parameters.
pub struct Params<B: BlockT> {
pub struct Params<B: BlockT, S> {
/// Configuration.
pub config: ProtocolConfig,
/// Network layer configuration.
@@ -136,24 +130,35 @@ pub struct Params<B: BlockT> {
pub on_demand: Option<Arc<OnDemandService<B>>>,
/// Transaction pool.
pub transaction_pool: Arc<TransactionPool<B>>,
/// Protocol specialization.
pub specialization: S,
}
/// Polkadot network service. Handles network IO and manages connectivity.
pub struct Service<B: BlockT + 'static> where B::Header: HeaderT<Number=u64> {
pub struct Service<B: BlockT + 'static, S: Specialization<B>> where B::Header: HeaderT<Number=u64> {
/// Network service
network: NetworkService,
/// Devp2p protocol handler
handler: Arc<ProtocolHandler<B>>,
handler: Arc<ProtocolHandler<B, S>>,
/// Devp2p protocol ID.
protocol_id: ProtocolId,
}
impl<B: BlockT + 'static> Service<B> where B::Header: HeaderT<Number=u64> {
impl<B: BlockT + 'static, S: Specialization<B>> Service<B, S> where B::Header: HeaderT<Number=u64> {
/// Creates and register protocol with the network service
pub fn new(params: Params<B>) -> Result<Arc<Service<B>>, Error> {
pub fn new(params: Params<B, S>, protocol_id: ProtocolId) -> Result<Arc<Service<B, S>>, Error> {
let service = NetworkService::new(params.network_config.clone(), None)?;
let sync = Arc::new(Service {
network: service,
protocol_id,
handler: Arc::new(ProtocolHandler {
protocol: Protocol::new(params.config, params.chain, params.on_demand, params.transaction_pool)?,
protocol: Protocol::new(
params.config,
params.chain.clone(),
params.on_demand,
params.transaction_pool,
params.specialization
)?,
}),
});
@@ -162,18 +167,31 @@ impl<B: BlockT + 'static> Service<B> where B::Header: HeaderT<Number=u64> {
/// Called when a new block is imported by the client.
pub fn on_block_imported(&self, hash: B::Hash, header: &B::Header) {
self.network.with_context(DOT_PROTOCOL_ID, |context| {
self.network.with_context(self.protocol_id, |context| {
self.handler.protocol.on_block_imported(&mut NetSyncIo::new(context), hash, header)
});
}
/// Called when new transactons are imported by the client.
pub fn trigger_repropagate(&self) {
self.network.with_context(DOT_PROTOCOL_ID, |context| {
self.handler.protocol.propagate_transactions(&mut NetSyncIo::new(context));
self.network.with_context(self.protocol_id, |context| {
self.handler.protocol.propagate_extrinsics(&mut NetSyncIo::new(context));
});
}
/// Execute a closure with the chain-specific network specialization.
/// If the network is unavailable, this will return `None`.
pub fn with_spec<F, U>(&self, f: F) -> Option<U>
where F: FnOnce(&mut S, &mut Context<B>) -> U
{
let mut res = None;
self.network.with_context(self.protocol_id, |context| {
res = Some(self.handler.protocol.with_spec(&mut NetSyncIo::new(context), f))
});
res
}
fn start(&self) {
match self.network.start().map_err(|e| e.0.into()) {
Err(ErrorKind::Io(ref e)) if e.kind() == io::ErrorKind::AddrInUse =>
@@ -181,7 +199,7 @@ impl<B: BlockT + 'static> Service<B> where B::Header: HeaderT<Number=u64> {
Err(err) => warn!("Error starting network: {}", err),
_ => {},
};
self.network.register_protocol(self.handler.clone(), DOT_PROTOCOL_ID, &[(0, V0_PACKET_COUNT)])
self.network.register_protocol(self.handler.clone(), self.protocol_id, &[(::protocol::CURRENT_VERSION as u8, ::protocol::CURRENT_PACKET_COUNT)])
.unwrap_or_else(|e| warn!("Error registering polkadot protocol: {:?}", e));
}
@@ -191,21 +209,21 @@ impl<B: BlockT + 'static> Service<B> where B::Header: HeaderT<Number=u64> {
}
}
impl<B: BlockT + 'static> Drop for Service<B> where B::Header: HeaderT<Number=u64> {
impl<B: BlockT + 'static, S: Specialization<B>> Drop for Service<B, S> where B::Header: HeaderT<Number=u64> {
fn drop(&mut self) {
self.stop();
}
}
impl<B: BlockT + 'static> ExecuteInContext<B> for Service<B> where B::Header: HeaderT<Number=u64> {
fn execute_in_context<F: Fn(&mut NetSyncIo, &Protocol<B>)>(&self, closure: F) {
self.network.with_context(DOT_PROTOCOL_ID, |context| {
closure(&mut NetSyncIo::new(context), &self.handler.protocol)
impl<B: BlockT + 'static, S: Specialization<B>> ExecuteInContext<B> for Service<B, S> where B::Header: HeaderT<Number=u64> {
fn execute_in_context<F: Fn(&mut ::protocol::Context<B>)>(&self, closure: F) {
self.network.with_context(self.protocol_id, |context| {
closure(&mut ProtocolContext::new(self.handler.protocol.context_data(), &mut NetSyncIo::new(context)))
});
}
}
impl<B: BlockT + 'static> SyncProvider<B> for Service<B> where B::Header: HeaderT<Number=u64> {
impl<B: BlockT + 'static, S: Specialization<B>> SyncProvider<B> for Service<B, S> where B::Header: HeaderT<Number=u64> {
/// Get sync status
fn status(&self) -> ProtocolStatus<B> {
self.handler.protocol.status()
@@ -213,7 +231,7 @@ impl<B: BlockT + 'static> SyncProvider<B> for Service<B> where B::Header: Header
/// Get sync peers
fn peers(&self) -> Vec<PeerInfo<B>> {
self.network.with_context_eval(DOT_PROTOCOL_ID, |ctx| {
self.network.with_context_eval(self.protocol_id, |ctx| {
let peer_ids = self.network.connected_peers();
peer_ids.into_iter().filter_map(|peer_id| {
@@ -239,24 +257,7 @@ impl<B: BlockT + 'static> SyncProvider<B> for Service<B> where B::Header: Header
}
}
/// ConsensusService
impl<B: BlockT + 'static> ConsensusService<B> for Service<B> where B::Header: HeaderT<Number=u64> {
fn connect_to_authorities(&self, _addresses: &[String]) {
//TODO: implement me
}
fn bft_messages(&self, parent_hash: B::Hash) -> BftMessageStream<B> {
self.handler.protocol.bft_messages(parent_hash)
}
fn send_bft_message(&self, message: LocalizedBftMessage<B>) {
self.network.with_context(DOT_PROTOCOL_ID, |context| {
self.handler.protocol.send_bft_message(&mut NetSyncIo::new(context), message);
});
}
}
impl<B: BlockT + 'static> NetworkProtocolHandler for ProtocolHandler<B> where B::Header: HeaderT<Number=u64> {
impl<B: BlockT + 'static, S: Specialization<B>> NetworkProtocolHandler for ProtocolHandler<B, S> where B::Header: HeaderT<Number=u64> {
fn initialize(&self, io: &NetworkContext) {
io.register_timer(TICK_TOKEN, TICK_TIMEOUT)
.expect("Error registering sync timer");
@@ -280,7 +281,7 @@ impl<B: BlockT + 'static> NetworkProtocolHandler for ProtocolHandler<B> where B:
fn timeout(&self, io: &NetworkContext, timer: TimerToken) {
match timer {
TICK_TOKEN => self.protocol.tick(&mut NetSyncIo::new(io)),
PROPAGATE_TOKEN => self.protocol.propagate_transactions(&mut NetSyncIo::new(io)),
PROPAGATE_TOKEN => self.protocol.propagate_extrinsics(&mut NetSyncIo::new(io)),
_ => {}
}
}
@@ -303,7 +304,7 @@ pub trait ManageNetwork: Send + Sync {
}
impl<B: BlockT + 'static> ManageNetwork for Service<B> where B::Header: HeaderT<Number=u64> {
impl<B: BlockT + 'static, S: Specialization<B>> ManageNetwork for Service<B, S> where B::Header: HeaderT<Number=u64> {
fn accept_unreserved_peers(&self) {
self.network.set_non_reserved_mode(NonReservedPeerMode::Accept);
}
substrate/substrate/network/src/specialization.rs
+45
View File
@@ -0,0 +1,45 @@
// Copyright 2017 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
//! Specializations of the substrate network protocol to allow more complex forms of communication.
use ::PeerId;
use runtime_primitives::traits::Block as BlockT;
use protocol::Context;
/// A specialization of the substrate network protocol. Handles events and sends messages.
pub trait Specialization<B: BlockT>: Send + Sync + 'static {
/// Get the current specialization-status.
fn status(&self) -> Vec<u8>;
/// Called on start-up.
fn on_start(&mut self) { }
/// Called when a peer successfully handshakes.
fn on_connect(&mut self, ctx: &mut Context<B>, peer_id: PeerId, status: ::message::Status<B>);
/// Called when a peer is disconnected. If the peer ID is unknown, it should be ignored.
fn on_disconnect(&mut self, ctx: &mut Context<B>, peer_id: PeerId);
/// Called when a network-specific message arrives.
fn on_message(&mut self, ctx: &mut Context<B>, peer_id: PeerId, message: ::message::Message<B>);
/// Called on abort.
fn on_abort(&mut self) { }
/// Called periodically to maintain peers and handle timeouts.
fn maintain_peers(&mut self, _ctx: &mut Context<B>) { }
}
substrate/substrate/network/src/sync.rs
+47 -48
View File
@@ -15,8 +15,7 @@
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.?
use std::collections::HashMap;
use io::SyncIo;
use protocol::Protocol;
use protocol::Context;
use network::PeerId;
use client::{ImportResult, BlockStatus, ClientInfo};
use blocks::{self, BlockCollection};
@@ -76,12 +75,12 @@ impl<B: BlockT> ChainSync<B> where
B::Header: HeaderT<Number=u64>,
{
/// Create a new instance.
pub fn new(role: Role, info: &ClientInfo<B>) -> Self {
pub(crate) fn new(role: Role, info: &ClientInfo<B>) -> Self {
let mut required_block_attributes = vec![
message::BlockAttribute::Header,
message::BlockAttribute::Justification
];
if role.intersects(Role::FULL | Role::VALIDATOR | Role::COLLATOR) {
if role.intersects(Role::FULL) {
required_block_attributes.push(message::BlockAttribute::Body);
}
@@ -100,7 +99,7 @@ impl<B: BlockT> ChainSync<B> where
}
/// Returns sync status
pub fn status(&self) -> Status<B> {
pub(crate) fn status(&self) -> Status<B> {
let best_seen = self.best_seen_block();
let state = match &best_seen {
&Some(n) if n > self.best_queued_number && n - self.best_queued_number > 5 => SyncState::Downloading,
@@ -113,20 +112,20 @@ impl<B: BlockT> ChainSync<B> where
}
/// Handle new connected peer.
pub fn new_peer(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId) {
pub(crate) fn new_peer(&mut self, protocol: &mut Context<B>, peer_id: PeerId) {
if let Some(info) = protocol.peer_info(peer_id) {
match (protocol.chain().block_status(&BlockId::Hash(info.best_hash)), info.best_number) {
match (protocol.client().block_status(&BlockId::Hash(info.best_hash)), info.best_number) {
(Err(e), _) => {
debug!(target:"sync", "Error reading blockchain: {:?}", e);
io.disconnect_peer(peer_id);
protocol.disconnect_peer(peer_id);
},
(Ok(BlockStatus::KnownBad), _) => {
debug!(target:"sync", "New peer with known bad best block {} ({}).", info.best_hash, info.best_number);
io.disable_peer(peer_id);
protocol.disable_peer(peer_id);
},
(Ok(BlockStatus::Unknown), 0) => {
debug!(target:"sync", "New peer with unknown genesis hash {} ({}).", info.best_hash, info.best_number);
io.disable_peer(peer_id);
protocol.disable_peer(peer_id);
},
(Ok(BlockStatus::Unknown), _) => {
let our_best = self.best_queued_number;
@@ -139,7 +138,7 @@ impl<B: BlockT> ChainSync<B> where
best_number: info.best_number,
state: PeerSyncState::AncestorSearch(our_best),
});
Self::request_ancestry(io, protocol, peer_id, our_best)
Self::request_ancestry(protocol, peer_id, our_best)
} else {
// We are at genesis, just start downloading
debug!(target:"sync", "New peer with best hash {} ({}).", info.best_hash, info.best_number);
@@ -150,7 +149,7 @@ impl<B: BlockT> ChainSync<B> where
best_number: info.best_number,
state: PeerSyncState::Available,
});
self.download_new(io, protocol, peer_id)
self.download_new(protocol, peer_id)
}
},
(Ok(BlockStatus::Queued), _) | (Ok(BlockStatus::InChain), _) => {
@@ -167,7 +166,7 @@ impl<B: BlockT> ChainSync<B> where
}
}
pub fn on_block_data(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId, _request: message::BlockRequest<B>, response: message::BlockResponse<B>) {
pub(crate) fn on_block_data(&mut self, protocol: &mut Context<B>, peer_id: PeerId, _request: message::BlockRequest<B>, response: message::BlockResponse<B>) {
let count = response.blocks.len();
let mut imported: usize = 0;
let new_blocks = if let Some(ref mut peer) = self.peers.get_mut(&peer_id) {
@@ -190,7 +189,7 @@ impl<B: BlockT> ChainSync<B> where
match response.blocks.get(0) {
Some(ref block) => {
trace!(target: "sync", "Got ancestry block #{} ({}) from peer {}", n, block.hash, peer_id);
match protocol.chain().block_hash(n) {
match protocol.client().block_hash(n) {
Ok(Some(block_hash)) if block_hash == block.hash => {
if peer.common_number < n {
peer.common_hash = block.hash;
@@ -204,24 +203,24 @@ impl<B: BlockT> ChainSync<B> where
trace!(target:"sync", "Ancestry block mismatch for peer {}: theirs: {} ({}), ours: {:?}", peer_id, block.hash, n, our_best);
let n = n - 1;
peer.state = PeerSyncState::AncestorSearch(n);
Self::request_ancestry(io, protocol, peer_id, n);
Self::request_ancestry(protocol, peer_id, n);
return;
},
Ok(_) => { // genesis mismatch
trace!(target:"sync", "Ancestry search: genesis mismatch for peer {}", peer_id);
io.disable_peer(peer_id);
protocol.disable_peer(peer_id);
return;
},
Err(e) => {
debug!(target:"sync", "Error reading blockchain: {:?}", e);
io.disconnect_peer(peer_id);
protocol.disconnect_peer(peer_id);
return;
}
}
},
None => {
trace!(target:"sync", "Invalid response when searching for ancestor from {}", peer_id);
io.disconnect_peer(peer_id);
protocol.disconnect_peer(peer_id);
return;
}
}
@@ -245,17 +244,17 @@ impl<B: BlockT> ChainSync<B> where
let is_best = best_seen.as_ref().map_or(false, |n| number >= *n);
// check whether the block is known before importing.
match protocol.chain().block_status(&BlockId::Hash(hash)) {
match protocol.client().block_status(&BlockId::Hash(hash)) {
Ok(BlockStatus::InChain) => continue,
Ok(_) => {},
Err(e) => {
debug!(target: "sync", "Error importing block {}: {:?}: {:?}", number, hash, e);
self.restart(io, protocol);
self.restart(protocol);
return;
}
}
let result = protocol.chain().import(
let result = protocol.client().import(
is_best,
header,
justification.to_justification(),
@@ -277,42 +276,42 @@ impl<B: BlockT> ChainSync<B> where
},
Ok(ImportResult::UnknownParent) => {
debug!(target: "sync", "Block with unknown parent {}: {:?}, parent: {:?}", number, hash, parent);
self.restart(io, protocol);
self.restart(protocol);
return;
},
Ok(ImportResult::KnownBad) => {
debug!(target: "sync", "Bad block {}: {:?}", number, hash);
io.disable_peer(origin); //TODO: use persistent ID
self.restart(io, protocol);
protocol.disable_peer(origin); //TODO: use persistent ID
self.restart(protocol);
return;
}
Err(e) => {
debug!(target: "sync", "Error importing block {}: {:?}: {:?}", number, hash, e);
self.restart(io, protocol);
self.restart(protocol);
return;
}
}
},
(None, _) => {
debug!(target: "sync", "Header {} was not provided by {} ", block.hash, origin);
io.disable_peer(origin); //TODO: use persistent ID
protocol.disable_peer(origin); //TODO: use persistent ID
return;
},
(_, None) => {
debug!(target: "sync", "Justification set for block {} was not provided by {} ", block.hash, origin);
io.disable_peer(origin); //TODO: use persistent ID
protocol.disable_peer(origin); //TODO: use persistent ID
return;
}
}
}
trace!(target: "sync", "Imported {} of {}", imported, count);
self.maintain_sync(io, protocol);
self.maintain_sync(protocol);
}
fn maintain_sync(&mut self, io: &mut SyncIo, protocol: &Protocol<B>) {
fn maintain_sync(&mut self, protocol: &mut Context<B>) {
let peers: Vec<PeerId> = self.peers.keys().map(|p| *p).collect();
for peer in peers {
self.download_new(io, protocol, peer);
self.download_new(protocol, peer);
}
}
@@ -331,12 +330,12 @@ impl<B: BlockT> ChainSync<B> where
}
}
pub fn update_chain_info(&mut self, best_header: &B::Header) {
pub(crate) fn update_chain_info(&mut self, best_header: &B::Header) {
let hash = best_header.hash();
self.block_imported(&hash, best_header.number().clone())
}
pub fn on_block_announce(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId, hash: B::Hash, header: &B::Header) {
pub(crate) fn on_block_announce(&mut self, protocol: &mut Context<B>, peer_id: PeerId, hash: B::Hash, header: &B::Header) {
let number = *header.number();
if let Some(ref mut peer) = self.peers.get_mut(&peer_id) {
if number > peer.best_number {
@@ -357,35 +356,35 @@ impl<B: BlockT> ChainSync<B> where
trace!(target: "sync", "Ignoring unknown stale block announce from {}: {} {:?}", peer_id, hash, header);
} else {
trace!(target: "sync", "Downloading new stale block announced from {}: {} {:?}", peer_id, hash, header);
self.download_stale(io, protocol, peer_id, &hash);
self.download_stale(protocol, peer_id, &hash);
}
} else {
trace!(target: "sync", "Downloading new block announced from {}: {} {:?}", peer_id, hash, header);
self.download_new(io, protocol, peer_id);
self.download_new(protocol, peer_id);
}
} else {
trace!(target: "sync", "Known block announce from {}: {}", peer_id, hash);
}
}
fn is_known_or_already_downloading(&self, protocol: &Protocol<B>, hash: &B::Hash) -> bool {
fn is_known_or_already_downloading(&self, protocol: &mut Context<B>, hash: &B::Hash) -> bool {
self.peers.iter().any(|(_, p)| p.state == PeerSyncState::DownloadingStale(*hash))
|| protocol.chain().block_status(&BlockId::Hash(*hash)).ok().map_or(false, |s| s != BlockStatus::Unknown)
|| protocol.client().block_status(&BlockId::Hash(*hash)).ok().map_or(false, |s| s != BlockStatus::Unknown)
}
pub fn peer_disconnected(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId) {
pub(crate) fn peer_disconnected(&mut self, protocol: &mut Context<B>, peer_id: PeerId) {
self.blocks.clear_peer_download(peer_id);
self.peers.remove(&peer_id);
self.maintain_sync(io, protocol);
self.maintain_sync(protocol);
}
pub fn restart(&mut self, io: &mut SyncIo, protocol: &Protocol<B>) {
pub(crate) fn restart(&mut self, protocol: &mut Context<B>) {
self.blocks.clear();
let ids: Vec<PeerId> = self.peers.keys().map(|p| *p).collect();
for id in ids {
self.new_peer(io, protocol, id);
self.new_peer(protocol, id);
}
match protocol.chain().info() {
match protocol.client().info() {
Ok(info) => {
self.best_queued_hash = info.best_queued_hash.unwrap_or(info.chain.best_hash);
self.best_queued_number = info.best_queued_number.unwrap_or(info.chain.best_number);
@@ -398,13 +397,13 @@ impl<B: BlockT> ChainSync<B> where
}
}
pub fn clear(&mut self) {
pub(crate) fn clear(&mut self) {
self.blocks.clear();
self.peers.clear();
}
// Download old block.
fn download_stale(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId, hash: &B::Hash) {
fn download_stale(&mut self, protocol: &mut Context<B>, peer_id: PeerId, hash: &B::Hash) {
if let Some(ref mut peer) = self.peers.get_mut(&peer_id) {
match peer.state {
PeerSyncState::Available => {
@@ -417,7 +416,7 @@ impl<B: BlockT> ChainSync<B> where
max: Some(1),
};
peer.state = PeerSyncState::DownloadingStale(*hash);
protocol.send_message(io, peer_id, GenericMessage::BlockRequest(request));
protocol.send_message(peer_id, GenericMessage::BlockRequest(request));
},
_ => (),
}
@@ -425,7 +424,7 @@ impl<B: BlockT> ChainSync<B> where
}
// Issue a request for a peer to download new blocks, if any are available
fn download_new(&mut self, io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId) {
fn download_new(&mut self, protocol: &mut Context<B>, peer_id: PeerId) {
if let Some(ref mut peer) = self.peers.get_mut(&peer_id) {
trace!(target: "sync", "Considering new block download from {}, common block is {}, best is {:?}", peer_id, peer.common_number, peer.best_number);
match peer.state {
@@ -441,7 +440,7 @@ impl<B: BlockT> ChainSync<B> where
max: Some((range.end - range.start) as u32),
};
peer.state = PeerSyncState::DownloadingNew(range.start);
protocol.send_message(io, peer_id, GenericMessage::BlockRequest(request));
protocol.send_message(peer_id, GenericMessage::BlockRequest(request));
} else {
trace!(target: "sync", "Nothing to request");
}
@@ -451,7 +450,7 @@ impl<B: BlockT> ChainSync<B> where
}
}
fn request_ancestry(io: &mut SyncIo, protocol: &Protocol<B>, peer_id: PeerId, block: u64) {
fn request_ancestry(protocol: &mut Context<B>, peer_id: PeerId, block: u64) {
trace!(target: "sync", "Requesting ancestry block #{} from {}", block, peer_id);
let request = message::generic::BlockRequest {
id: 0,
@@ -461,6 +460,6 @@ impl<B: BlockT> ChainSync<B> where
direction: message::Direction::Ascending,
max: Some(1),
};
protocol.send_message(io, peer_id, GenericMessage::BlockRequest(request));
protocol.send_message(peer_id, GenericMessage::BlockRequest(request));
}
}
substrate/substrate/network/src/test/consensus.rs
-53
View File
@@ -1,53 +0,0 @@
// Copyright 2017 Parity Technologies (UK) Ltd.
// This file is part of Polkadot.
// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
use super::*;
use message::{Message, generic};
use futures::Stream;
use test_client::runtime::Block;
#[test]
fn bft_messages_include_those_sent_before_asking_for_stream() {
let mut config = ::config::ProtocolConfig::default();
config.roles = ::service::Role::VALIDATOR | ::service::Role::FULL;
let mut net = TestNet::new_with_config(2, config);
net.sync(); // necessary for handshaking
let peer = net.peer(0);
let mut io = TestIo::new(&peer.queue, None);
let bft_message = generic::BftMessage::Consensus(generic::SignedConsensusMessage::Vote(generic::SignedConsensusVote {
vote: generic::ConsensusVote::AdvanceRound(0),
sender: Default::default(),
signature: Default::default(),
}));
let parent_hash = peer.genesis_hash();
let localized = ::message::LocalizedBftMessage::<Block> {
message: bft_message,
parent_hash: parent_hash,
};
let message: Message<Block> = generic::Message::BftMessage(localized.clone());
let as_bytes = ::serde_json::to_vec(&message).unwrap();
peer.sync.handle_packet(&mut io, 1, &as_bytes[..]);
let stream = peer.sync.bft_messages(parent_hash);
assert_eq!(stream.wait().next(), Some(Ok(localized)));
}
substrate/substrate/network/src/test/mod.rs
+42 -29
View File
@@ -14,7 +14,6 @@
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.
mod consensus;
mod sync;
use std::collections::{VecDeque, HashSet, HashMap};
@@ -26,7 +25,7 @@ use client::block_builder::BlockBuilder;
use runtime_primitives::traits::Block as BlockT;
use runtime_primitives::generic::BlockId;
use io::SyncIo;
use protocol::Protocol;
use protocol::{Context, Protocol};
use config::ProtocolConfig;
use service::TransactionPool;
use network::{PeerId, SessionInfo, Error as NetworkError};
@@ -34,20 +33,39 @@ use keyring::Keyring;
use codec::Slicable;
use test_client::{self, TestClient};
use test_client::runtime::{Block, Hash, Transfer, Extrinsic};
use specialization::Specialization;
pub struct DummySpecialization;
impl Specialization<Block> for DummySpecialization {
fn status(&self) -> Vec<u8> { vec![] }
fn on_connect(&mut self, _ctx: &mut Context<Block>, _peer_id: PeerId, _status: ::message::Status<Block>) {
}
fn on_disconnect(&mut self, _ctx: &mut Context<Block>, _peer_id: PeerId) {
}
fn on_message(&mut self, _ctx: &mut Context<Block>, _peer_id: PeerId, _message: ::message::Message<Block>) {
}
}
pub struct TestIo<'p> {
pub queue: &'p RwLock<VecDeque<TestPacket>>,
pub sender: Option<PeerId>,
queue: &'p RwLock<VecDeque<TestPacket>>,
pub to_disconnect: HashSet<PeerId>,
pub packets: Vec<TestPacket>,
pub peers_info: HashMap<PeerId, String>,
packets: Vec<TestPacket>,
peers_info: HashMap<PeerId, String>,
_sender: Option<PeerId>,
}
impl<'p> TestIo<'p> where {
pub fn new(queue: &'p RwLock<VecDeque<TestPacket>>, sender: Option<PeerId>) -> TestIo<'p> {
TestIo {
queue: queue,
sender: sender,
_sender: sender,
to_disconnect: HashSet::new(),
packets: Vec::new(),
peers_info: HashMap::new(),
@@ -95,13 +113,13 @@ impl<'p> SyncIo for TestIo<'p> {
/// Mocked subprotocol packet
pub struct TestPacket {
pub data: Vec<u8>,
pub recipient: PeerId,
data: Vec<u8>,
recipient: PeerId,
}
pub struct Peer {
client: Arc<client::Client<test_client::Backend, test_client::Executor, Block>>,
pub sync: Protocol<Block>,
pub sync: Protocol<Block, DummySpecialization>,
pub queue: RwLock<VecDeque<TestPacket>>,
}
@@ -186,14 +204,9 @@ impl Peer {
self.generate_blocks(count, |_| ());
}
}
pub fn genesis_hash(&self) -> Hash {
let info = self.client.info().expect("In-mem client does not fail");
info.chain.genesis_hash
}
}
struct EmptyTransactionPool;
pub struct EmptyTransactionPool;
impl TransactionPool<Block> for EmptyTransactionPool {
fn transactions(&self) -> Vec<(Hash, Extrinsic)> {
@@ -208,17 +221,17 @@ impl TransactionPool<Block> for EmptyTransactionPool {
}
pub struct TestNet {
pub peers: Vec<Arc<Peer>>,
pub started: bool,
pub disconnect_events: Vec<(PeerId, PeerId)>, //disconnected (initiated by, to)
peers: Vec<Arc<Peer>>,
started: bool,
disconnect_events: Vec<(PeerId, PeerId)>, //disconnected (initiated by, to)
}
impl TestNet {
pub fn new(n: usize) -> Self {
fn new(n: usize) -> Self {
Self::new_with_config(n, ProtocolConfig::default())
}
pub fn new_with_config(n: usize, config: ProtocolConfig) -> Self {
fn new_with_config(n: usize, config: ProtocolConfig) -> Self {
let mut net = TestNet {
peers: Vec::new(),
started: false,
@@ -234,7 +247,7 @@ impl TestNet {
pub fn add_peer(&mut self, config: &ProtocolConfig) {
let client = Arc::new(test_client::new());
let tx_pool = Arc::new(EmptyTransactionPool);
let sync = Protocol::new(config.clone(), client.clone(), None, tx_pool).unwrap();
let sync = Protocol::new(config.clone(), client.clone(), None, tx_pool, DummySpecialization).unwrap();
self.peers.push(Arc::new(Peer {
sync: sync,
client: client,
@@ -246,7 +259,7 @@ impl TestNet {
&self.peers[i]
}
pub fn start(&mut self) {
fn start(&mut self) {
if self.started {
return;
}
@@ -261,7 +274,7 @@ impl TestNet {
self.started = true;
}
pub fn sync_step(&mut self) {
fn sync_step(&mut self) {
for peer in 0..self.peers.len() {
let packet = self.peers[peer].pending_message();
if let Some(packet) = packet {
@@ -286,15 +299,15 @@ impl TestNet {
}
}
pub fn sync_step_peer(&mut self, peer_num: usize) {
fn sync_step_peer(&mut self, peer_num: usize) {
self.peers[peer_num].sync_step();
}
pub fn restart_peer(&mut self, i: usize) {
fn restart_peer(&mut self, i: usize) {
self.peers[i].restart_sync();
}
pub fn sync(&mut self) -> u32 {
fn sync(&mut self) -> u32 {
self.start();
let mut total_steps = 0;
while !self.done() {
@@ -304,14 +317,14 @@ impl TestNet {
total_steps
}
pub fn sync_steps(&mut self, count: usize) {
fn sync_steps(&mut self, count: usize) {
self.start();
for _ in 0..count {
self.sync_step();
}
}
pub fn done(&self) -> bool {
fn done(&self) -> bool {
self.peers.iter().all(|p| p.is_done())
}
}
substrate/substrate/primitives/src/authority_id.rs
+1
View File
@@ -26,6 +26,7 @@ pub struct AuthorityId(pub [u8; 32]);
impl AuthorityId {
/// Create an id from a 32-byte slice. Panics with other lengths.
#[cfg(feature = "std")]
fn from_slice(data: &[u8]) -> Self {
let mut r = [0u8; 32];
r.copy_from_slice(data);
substrate/substrate/rpc/Cargo.toml
+1 -1
View File
@@ -17,7 +17,7 @@ substrate-extrinsic-pool = { path = "../extrinsic-pool" }
substrate-primitives = { path = "../primitives" }
substrate-runtime-primitives = { path = "../runtime/primitives" }
substrate-state-machine = { path = "../state-machine" }
tokio-core = "0.1.12"
tokio = "0.1.7"
[dev-dependencies]
assert_matches = "1.1"
substrate/substrate/rpc/src/chain/mod.rs
+3 -3
View File
@@ -27,7 +27,7 @@ use jsonrpc_macros::pubsub;
use jsonrpc_pubsub::SubscriptionId;
use rpc::Result as RpcResult;
use rpc::futures::{Future, Sink, Stream};
use tokio_core::reactor::Remote;
use tokio::runtime::TaskExecutor;
use subscriptions::Subscriptions;
@@ -72,10 +72,10 @@ pub struct Chain<B, E, Block: BlockT> {
impl<B, E, Block: BlockT> Chain<B, E, Block> {
/// Create new Chain API RPC handler.
pub fn new(client: Arc<Client<B, E, Block>>, remote: Remote) -> Self {
pub fn new(client: Arc<Client<B, E, Block>>, executor: TaskExecutor) -> Self {
Chain {
client,
subscriptions: Subscriptions::new(remote),
subscriptions: Subscriptions::new(executor),
}
}
}
substrate/substrate/rpc/src/chain/tests.rs
+7 -7
View File
@@ -22,8 +22,8 @@ use test_client::runtime::Header;
#[test]
fn should_return_header() {
let core = ::tokio_core::reactor::Core::new().unwrap();
let remote = core.remote();
let core = ::tokio::runtime::Runtime::new().unwrap();
let remote = core.executor();
let client = Chain {
client: Arc::new(test_client::new()),
@@ -48,8 +48,8 @@ fn should_return_header() {
#[test]
fn should_notify_about_latest_block() {
let mut core = ::tokio_core::reactor::Core::new().unwrap();
let remote = core.remote();
let mut core = ::tokio::runtime::Runtime::new().unwrap();
let remote = core.executor();
let (subscriber, id, transport) = pubsub::Subscriber::new_test("test");
{
@@ -61,17 +61,17 @@ fn should_notify_about_latest_block() {
api.subscribe_new_head(Default::default(), subscriber);
// assert id assigned
assert_eq!(core.run(id), Ok(Ok(SubscriptionId::Number(0))));
assert_eq!(core.block_on(id), Ok(Ok(SubscriptionId::Number(0))));
let builder = api.client.new_block().unwrap();
api.client.justify_and_import(BlockOrigin::Own, builder.bake().unwrap()).unwrap();
}
// assert notification send to transport
let (notification, next) = core.run(transport.into_future()).unwrap();
let (notification, next) = core.block_on(transport.into_future()).unwrap();
assert_eq!(notification, Some(
r#"{"jsonrpc":"2.0","method":"test","params":{"result":{"digest":{"logs":[]},"extrinsicsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","number":1,"parentHash":"0x50fb1e7f32e8ad17f553846f4338861e17eb95132e7c3b433e0429ffab2f8f13","stateRoot":"0x17dccc74bd9200b7ce5a2f6a1bf379f1cdcf91bca3d19c3d17e1478b8d404703"},"subscription":0}}"#.to_owned()
));
// no more notifications on this channel
assert_eq!(core.run(next.into_future()).unwrap().0, None);
assert_eq!(core.block_on(next.into_future()).unwrap().0, None);
}
substrate/substrate/rpc/src/lib.rs
+1 -1
View File
@@ -27,7 +27,7 @@ extern crate substrate_extrinsic_pool as extrinsic_pool;
extern crate substrate_primitives as primitives;
extern crate substrate_runtime_primitives as runtime_primitives;
extern crate substrate_state_machine as state_machine;
extern crate tokio_core;
extern crate tokio;
#[macro_use]
extern crate error_chain;
substrate/substrate/rpc/src/subscriptions.rs
+6 -7
View File
@@ -22,7 +22,7 @@ use jsonrpc_pubsub::SubscriptionId;
use parking_lot::Mutex;
use rpc::futures::sync::oneshot;
use rpc::futures::{Future, future};
use tokio_core::reactor::Remote;
use tokio::runtime::TaskExecutor;
type Id = u64;
@@ -34,16 +34,16 @@ type Id = u64;
pub struct Subscriptions {
next_id: AtomicUsize,
active_subscriptions: Mutex<HashMap<Id, oneshot::Sender<()>>>,
event_loop: Remote,
executor: TaskExecutor,
}
impl Subscriptions {
/// Creates new `Subscriptions` object.
pub fn new(event_loop: Remote) -> Self {
pub fn new(executor: TaskExecutor) -> Self {
Subscriptions {
next_id: Default::default(),
active_subscriptions: Default::default(),
event_loop,
executor,
}
}
@@ -63,11 +63,10 @@ impl Subscriptions {
let future = into_future(sink)
.into_future()
.select(rx.map_err(|e| warn!("Error timeing out: {:?}", e)))
.map(|_| ())
.map_err(|_| ());
.then(|_| Ok(()));
self.active_subscriptions.lock().insert(id, tx);
self.event_loop.spawn(|_| future);
self.executor.spawn(future);
}
}
substrate/substrate/runtime/primitives/src/traits.rs
+5 -1
View File
@@ -32,11 +32,15 @@ pub trait Lazy<T: ?Sized> {
fn get(&mut self) -> &T;
}
impl<'a> Lazy<[u8]> for &'a [u8] {
fn get(&mut self) -> &[u8] { &**self }
}
/// Means of signature verification.
pub trait Verify {
/// Type of the signer.
type Signer;
/// Verify a signature.
/// Verify a signature. Return `true` if signature is valid for the value.
fn verify<L: Lazy<[u8]>>(&self, msg: L, signer: &Self::Signer) -> bool;
}
substrate/substrate/test-runtime/wasm/target/wasm32-unknown-unknown/release/substrate_test_runtime.compact.wasm
BIN
View File
Binary file not shown.
substrate/substrate/test-runtime/wasm/target/wasm32-unknown-unknown/release/substrate_test_runtime.wasm
BIN
View File
Binary file not shown.