pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-05-31 02:51:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Squashed commit of slava-async-bridge:

Browse Source 
commit 657deb4cf4b90f24b9c5bfd62764b197776c262c
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 20:14:20 2020 -0500

    Move Slava's bridge code into relays folder

commit 4868c42c7da959dde7252766996b3ed4e408e439
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 20:01:06 2020 -0500

    Move files into `modules/ethereum`

commit d1093f3e4238acb1a1a020011452cb928d3f8d7a
Merge: 29dc6f9 bfd30ef
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 19:59:27 2020 -0500

    Merge branch 'master' of slava-async-bridge

commit 29dc6f97b1b7d1db99086d35a5336f43d2f0f8af
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 19:51:31 2020 -0500

    Squashed commit of the following:

    commit 265365920836bb1d286c9b48b1902a2de278fdd9
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Wed Jan 29 19:51:15 2020 -0500

        Move hc-jp-bridge repo to different folder

    commit 8271991e95320baba70bd1cb9c4234d0ffd5b638
    Merge: 57d0811 304cbc5
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Wed Jan 29 19:36:41 2020 -0500

        Merge branch 'hc-jp-bridge-module' of hc-jp-bridge-module

    commit 304cbc5f02d003ffa5404c1c01e461e5b8539888
    Author: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Wed Jan 29 00:38:27 2020 -0500

        Update bridge pallet to work with the (almost) lastest master (#4672)

        * Update decl_error usage

        * WIP: Update error handling to use DispatchResult

        * Get module compiling with new error handling

        * Make tests compile again

        Main change was updating the usage of InMemoryBackend

        * Move `sp-state-machine` into dev-dependencies

        * Bump dependencies to v2.0.0

        * Remove some stray comments

        * Appy code review suggestion

    commit 510cd6d96372688517496efa61773ea2839f8474
    Author: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Tue Dec 17 12:52:51 2019 -0500

        Move Bridge Pallet into FRAME (#4373)

        * Move `bridge` crate into `frame` folder

        * Make `bridge` pallet compile after `the-big-reorg`

    commit ab54e838ef75e6a3f68fd0944bf22598c10c552f
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Mon Nov 11 21:56:40 2019 +0100

        Use new StorageProof type from #3834

    commit 8fc8911fd1b4acc2274c6863fb3dba91b30c90af
    Author: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Tue Nov 5 00:50:34 2019 +0100

        Verify Ancestry between Headers (#3963)

        * Create module for checking ancestry proofs

        * Use Vec of Headers instead of a HashMap

        * Move the ancestry verification into the lib.rs file

        * Change the proof format to exclude `child` and `ancestor` headers

        * Add a testing function for building header chains

        * Rename AncestorNotFound error to InvalidAncestryProof

        * Use ancestor hash instead of header when verifying ancestry

        * Clean up some stuff missed in the merge

    commit dbe85738b68358b790cf927b34a804b965a88f96
    or: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Fri Nov 1 15:41:58 2019 +0100

        Check given Grandpa validator set against set found in storage (#3915)

        * Make StorageProofChecker happy

        * Update some tests

        * Check given validator set against set found in storage

        * Use Finality Grandpa's Authority Id and Weight

        * Add better error handling

        * Use error type from decl_error! macro

    commit 31b09216603d3e9c21144ce8c0b6bf59307a4f97
    or: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Wed Oct 23 14:55:37 2019 +0200

        Make tests work after the changes introduced in #3793 (#3874)

        * Make tests work after the changes introduced in #3793

        * Remove unneccessary import

    commit bce6d804aa86504599ff912387295c58f846cbf3
    Author: Jim Posen <jim.posen@gmail.com>
    Date:   Thu Oct 10 12:18:58 2019 +0200

        Logic for checking Substrate proofs from within runtime module. (#3783)

    commit a7013e94b6c772c1d45a7cacbb445f73f6554fca
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Fri Oct 4 15:21:00 2019 +0300

        Allow tracking of multiple bridges

    commit 3cf648242d631e32bd553a67df54bf5a48912839
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Tue Oct 1 14:55:04 2019 +0200

        Add BridgeId => Bridge mapping

    commit 001c74c45072213e01857d0a2454379b447c5a76
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Tue Oct 1 11:10:19 2019 +0200

        Get the mock runtime for tests set up

    commit 38443a1e8b424ed2f148eb95121d009f730e3b5a
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Fri Sep 27 14:52:53 2019 +0200

        Clean up some warnings

    commit bdc3b01401e89c7111f8bf71f84c50750d25089f
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Thu Sep 26 16:41:01 2019 +0200

        Add more skeleton code

    commit 26995efbf4bac2842eb2822322f7ad3c3e88feb8
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Wed Sep 25 15:16:57 2019 +0200

        Create `bridge` module skeleton

commit bfd30ef8363b1483ef1107ae1eb958a4e944c93b
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 12:10:53 2019 +0300

    actually use signer from CLI to sign Substrate transactions

commit 504028eac60d9d14ba95b506cd355b0d2f405ce0
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 12:02:22 2019 +0300

    go offline for a bit on connection error

commit 446d0c8d20187dfd1beb173958ea28f2ad97887d
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 11:25:50 2019 +0300

    enable info logs by default

commit d039c60ec72bc91adfdad85442bc99a93b7f8e8d
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 11:12:51 2019 +0300

    support basic CLI arguments

commit 65c6d48e23576f36e8541878b920a03730226392
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 15:37:48 2019 +0300

    fix restart

commit 96e94c1c4b22d732078f8c401b872c5f8246c3fe
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 14:57:53 2019 +0300

    license

commit 68f4191e6cdd211ac8975e0b79f8a6f46a3ca953
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 14:56:05 2019 +0300

    restart sync when Substrate reorgs && we are unlucky

commit 29887c446167d580d73cc03a0b71c31890cafb51
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:49:31 2019 +0300

    only read genesis hash once

commit 832492b8393fe2063adf9c58c2b9e060dc3e4efb
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:23:26 2019 +0300

    changed TODO

commit 9dbc130e5fa036ae63d973819daf30f4ed6ffb5b
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:16:56 2019 +0300

    removed obsolete exit future

commit d03408cd8284eb0c61e7e96429b4f6199353e030
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:16:17 2019 +0300

    removed obsolete TODOs + moved a couple of TODOs to runtime module

commit ed8bec44b79f9a2ce829e59f10181368b2f42139
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 12:37:05 2019 +0300

    explained TODO fix

commit aa9c4c66ec2904eeb6072d654718b0ac0b7d8803
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 12:28:09 2019 +0300

    fix tx outcome serialization

commit 126f8f5484dac8c4af588ae86dc8855919d6c822
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 12:05:05 2019 +0300

    prune old ethereum headers when Substrate best header is too far in the future

commit c7bd301e631a44fe3263e188d0956081aa84f31e
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Fri Dec 6 12:51:50 2019 +0300

    fix trace

commit 549bb7acdb30cfdafe6c8600f0410212539ea63d
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Fri Dec 6 12:51:26 2019 +0300

    tx hashes are already a part of Block response

commit 7864017909f87ea36955d605a924c3c88bc88df3
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Thu Dec 5 12:29:37 2019 +0300

    submit bunch of headers at once + some fixes

commit 96485f85d38c144f0771f02ba692216a60356665
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 17:22:13 2019 +0300

    print status messages

commit ae0ec4c087136db653339537daab7f96a8c21b65
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 17:06:00 2019 +0300

    continue actual Substrate client implementation

commit 8146293740d70b88904568ff8e5acdfbadf06fd3
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 13:49:30 2019 +0300

    fix IncompleteHeader condition

commit 767c6201157dabcccf7f62e643681ca298224fb1
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 10:55:06 2019 +0300

    actual Substrate client implementation

commit 221fd4ccd2b1eea12c9dacf800d80e15ec115c1b
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Nov 20 17:28:13 2019 +0300

    initial commit
This commit is contained in:
Hernando Castano
2020-01-29 20:14:48 -05:00
committed by Bastian Köcher
parent d530bf2199
commit 479c5bd99a
13 changed files with 9349 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bridges/relays/ethereum/src/cli.yml
+33
View File
@@ -0,0 +1,33 @@
name: ethsub-bridge
version: "0.1.0"
author: Parity Technologies <admin@parity.io>
about: Parity Ethereum (PoA) <-> Substrate bridge
args:
- eth-host:
long: eth-host
value_name: ETH_HOST
help: Connect to Ethereum node at given host.
takes_value: true
- eth-port:
long: eth-port
value_name: ETH_PORT
help: Connect to Ethereum node at given port.
takes_value: true
- sub-host:
long: sub-host
value_name: SUB_HOST
help: Connect to Substrate node at given host.
takes_value: true
- sub-port:
long: sub-port
value_name: SUB_PORT
help: Connect to Substrate node at given port.
takes_value: true
- sub-signer:
long: sub-signer
value_name: SUB_SIGNER
help: The SURI of secret key to use when transactions are submitted to the Substrate node.
- sub-signer-password:
long: sub-signer-password
value_name: SUB_SIGNER_PASSWORD
help: The password for the SURI of secret key to use when transactions are submitted to the Substrate node.
bridges/relays/ethereum/src/ethereum_client.rs
+170
View File
@@ -0,0 +1,170 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
use jsonrpsee_core::{client::ClientError, common::Params};
use jsonrpsee_http::{HttpClient, RequestError, http_client};
use serde::de::DeserializeOwned;
use serde_json::{from_value, to_value};
use crate::ethereum_sync_loop::MaybeConnectionError;
use crate::ethereum_types::{H256, Header, HeaderId, Receipt, U64};
/// Proof of hash serialization success.
const HASH_SERIALIZATION_PROOF: &'static str = "hash serialization never fails; qed";
/// Proof of integer serialization success.
const INT_SERIALIZATION_PROOF: &'static str = "integer serialization never fails; qed";
/// Proof of bool serialization success.
const BOOL_SERIALIZATION_PROOF: &'static str = "bool serialization never fails; qed";
/// Ethereum client type.
pub type Client = HttpClient;
/// All possible errors that can occur during interacting with Ethereum node.
#[derive(Debug)]
pub enum Error {
/// Request start failed.
StartRequestFailed(RequestError),
/// Request not found (should never occur?).
RequestNotFound,
/// Failed to receive response.
ResponseRetrievalFailed(ClientError<RequestError>),
/// Failed to parse response.
ResponseParseFailed(serde_json::Error),
/// We have received header with missing number and hash fields.
IncompleteHeader,
/// We have received receipt with missing gas_used field.
IncompleteReceipt,
}
impl MaybeConnectionError for Error {
fn is_connection_error(&self) -> bool {
match *self {
Error::StartRequestFailed(_) | Error::ResponseRetrievalFailed(_) => true,
_ => false,
}
}
}
/// Returns client that is able to call RPCs on Ethereum node.
pub fn client(uri: &str) -> Client {
http_client(uri)
}
/// Retrieve best known block number from Ethereum node.
pub async fn best_block_number(client: Client) -> (Client, Result<u64, Error>) {
let (client, result) = call_rpc::<U64>(
client,
"eth_blockNumber",
Params::None,
).await;
(client, result.map(|x| x.as_u64()))
}
/// Retrieve block header by its number from Ethereum node.
pub async fn header_by_number(client: Client, number: u64) -> (Client, Result<Header, Error>) {
let (client, header) = call_rpc(
client,
"eth_getBlockByNumber",
Params::Array(vec![
to_value(U64::from(number)).expect(INT_SERIALIZATION_PROOF),
to_value(false).expect(BOOL_SERIALIZATION_PROOF),
]),
).await;
(client, header.and_then(|header: Header| match header.number.is_some() && header.hash.is_some() {
true => Ok(header),
false => Err(Error::IncompleteHeader),
}))
}
/// Retrieve block header by its hash from Ethereum node.
pub async fn header_by_hash(client: Client, hash: H256) -> (Client, Result<Header, Error>) {
let (client, header) = call_rpc(
client,
"eth_getBlockByHash",
Params::Array(vec![
to_value(hash).expect(HASH_SERIALIZATION_PROOF),
to_value(false).expect(BOOL_SERIALIZATION_PROOF),
]),
).await;
(client, header.and_then(|header: Header| match header.number.is_none() && header.hash.is_none() {
true => Ok(header),
false => Err(Error::IncompleteHeader),
}))
}
/// Retrieve transactions receipts for given block.
pub async fn transactions_receipts(
mut client: Client,
id: HeaderId,
transacactions: Vec<H256>,
) -> (Client, Result<(HeaderId, Vec<Receipt>), Error>) {
let mut transactions_receipts = Vec::with_capacity(transacactions.len());
for transacaction in transacactions {
let (next_client, transaction_receipt) = transaction_receipt(client, transacaction).await;
let transaction_receipt = match transaction_receipt {
Ok(transaction_receipt) => transaction_receipt,
Err(error) => return (next_client, Err(error)),
};
transactions_receipts.push(transaction_receipt);
client = next_client;
}
(client, Ok((id, transactions_receipts)))
}
/// Retrieve transaction receipt by transaction hash.
async fn transaction_receipt(client: Client, hash: H256) -> (Client, Result<Receipt, Error>) {
let (client, receipt) = call_rpc::<Receipt>(
client,
"eth_getTransactionReceipt",
Params::Array(vec![
to_value(hash).expect(HASH_SERIALIZATION_PROOF),
]),
).await;
(client, receipt.and_then(|receipt| {
match receipt.gas_used.is_some() {
true => Ok(receipt),
false => Err(Error::IncompleteReceipt),
}
}))
}
/// Calls RPC on Ethereum node.
async fn call_rpc<T: DeserializeOwned>(
mut client: Client,
method: &'static str,
params: Params,
) -> (Client, Result<T, Error>) {
async fn do_call_rpc<T: DeserializeOwned>(
client: &mut Client,
method: &'static str,
params: Params,
) -> Result<T, Error> {
let request_id = client
.start_request(method, params)
.await
.map_err(Error::StartRequestFailed)?;
// WARN: if there'll be need for executing >1 request at a time, we should avoid
// calling request_by_id
let response = client
.request_by_id(request_id)
.ok_or(Error::RequestNotFound)?
.await
.map_err(Error::ResponseRetrievalFailed)?;
from_value(response).map_err(Error::ResponseParseFailed)
}
let result = do_call_rpc(&mut client, method, params).await;
(client, result)
}
bridges/relays/ethereum/src/ethereum_headers.rs
+761
View File
@@ -0,0 +1,761 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
use std::collections::{
BTreeMap, HashMap, HashSet,
btree_map::Entry as BTreeMapEntry,
hash_map::Entry as HashMapEntry,
};
use crate::ethereum_types::{H256, Header, HeaderId, HeaderStatus, QueuedHeader, Receipt};
type HeadersQueue = BTreeMap<u64, HashMap<H256, QueuedHeader>>;
type KnownHeaders = BTreeMap<u64, HashMap<H256, HeaderStatus>>;
/// Ethereum headers queue.
#[derive(Debug, Default)]
pub struct QueuedHeaders {
/// Headers that are received from Ethereum node, but we (native ethereum sync code) have
/// never seen their parents. So we need to check if we can/should submit this header.
maybe_orphan: HeadersQueue,
/// Headers that are received from Ethreum node, and we (native ethereum sync code) have
/// checked that Substrate runtime doesn't know their parents. So we need to submit parents
/// first.
orphan: HeadersQueue,
/// Headers that are ready to be submitted to Substrate runtime, but we need to check
/// whether submission requires transactions receipts to be provided.
maybe_receipts: HeadersQueue,
/// Headers that are ready to be submitted to Substrate runtime, but we need to retrieve
/// transactions receipts first.
receipts: HeadersQueue,
/// Headers that are ready to be submitted to Substrate runtime.
ready: HeadersQueue,
/// Headers that are (we believe) are currently submitted to Substrate runtime by our,
/// not-yet mined transactions.
submitted: HeadersQueue,
/// Pointers to all headers that we ever seen and we believe we can touch in the future.
known_headers: KnownHeaders,
/// Pruned blocks border. We do not store or accept any blocks with number less than
/// this number.
prune_border: u64,
}
impl QueuedHeaders {
/// Returns prune border.
#[cfg(test)]
pub fn prune_border(&self) -> u64 {
self.prune_border
}
/// Returns number of headers that are currently in given queue.
pub fn headers_in_status(&self, status: HeaderStatus) -> usize {
match status {
HeaderStatus::Unknown | HeaderStatus::Synced => return 0,
HeaderStatus::MaybeOrphan => self.maybe_orphan.values().fold(0, |total, headers| total + headers.len()),
HeaderStatus::Orphan => self.orphan.values().fold(0, |total, headers| total + headers.len()),
HeaderStatus::MaybeReceipts => self.maybe_receipts.values().fold(0, |total, headers| total + headers.len()),
HeaderStatus::Receipts => self.receipts.values().fold(0, |total, headers| total + headers.len()),
HeaderStatus::Ready => self.ready.values().fold(0, |total, headers| total + headers.len()),
HeaderStatus::Submitted => self.submitted.values().fold(0, |total, headers| total + headers.len()),
}
}
/// Returns number of headers that are currently in the queue.
pub fn total_headers(&self) -> usize {
self.maybe_orphan.values().fold(0, |total, headers| total + headers.len())
+ self.orphan.values().fold(0, |total, headers| total + headers.len())
+ self.maybe_receipts.values().fold(0, |total, headers| total + headers.len())
+ self.receipts.values().fold(0, |total, headers| total + headers.len())
+ self.ready.values().fold(0, |total, headers| total + headers.len())
}
/// Returns number of best block in the queue.
pub fn best_queued_number(&self) -> u64 {
std::cmp::max(
self.maybe_orphan.keys().next_back().cloned().unwrap_or(0),
std::cmp::max(
self.orphan.keys().next_back().cloned().unwrap_or(0),
std::cmp::max(
self.maybe_receipts.keys().next_back().cloned().unwrap_or(0),
std::cmp::max(
self.receipts.keys().next_back().cloned().unwrap_or(0),
self.ready.keys().next_back().cloned().unwrap_or(0),
),
),
),
)
}
/// Returns synchronization status of the header.
pub fn status(&self, id: &HeaderId) -> HeaderStatus {
self.known_headers
.get(&id.0)
.and_then(|x| x.get(&id.1))
.cloned()
.unwrap_or(HeaderStatus::Unknown)
}
/// Get oldest header from given queue.
pub fn header(&self, status: HeaderStatus) -> Option<&QueuedHeader> {
match status {
HeaderStatus::Unknown | HeaderStatus::Synced => return None,
HeaderStatus::MaybeOrphan => oldest_header(&self.maybe_orphan),
HeaderStatus::Orphan => oldest_header(&self.orphan),
HeaderStatus::MaybeReceipts => oldest_header(&self.maybe_receipts),
HeaderStatus::Receipts => oldest_header(&self.receipts),
HeaderStatus::Ready => oldest_header(&self.ready),
HeaderStatus::Submitted => oldest_header(&self.submitted),
}
}
/// Get oldest headers from given queue until functor will return false.
pub fn headers(&self, status: HeaderStatus, f: impl FnMut(&QueuedHeader) -> bool) -> Option<Vec<&QueuedHeader>> {
match status {
HeaderStatus::Unknown | HeaderStatus::Synced => return None,
HeaderStatus::MaybeOrphan => oldest_headers(&self.maybe_orphan, f),
HeaderStatus::Orphan => oldest_headers(&self.orphan, f),
HeaderStatus::MaybeReceipts => oldest_headers(&self.maybe_receipts, f),
HeaderStatus::Receipts => oldest_headers(&self.receipts, f),
HeaderStatus::Ready => oldest_headers(&self.ready, f),
HeaderStatus::Submitted => oldest_headers(&self.submitted, f),
}
}
/// Appends new header to the queue.
pub fn header_response(&mut self, header: Header) {
let id = (&header).into();
let status = self.status(&id);
if status != HeaderStatus::Unknown {
log::debug!(target: "bridge", "Ignoring new Ethereum header: {:?}. Status is {:?}.", id, status);
return;
}
if id.0 < self.prune_border {
log::debug!(target: "bridge", "Ignoring ancient new Ethereum header: {:?}.", id);
return;
}
let parent_id = HeaderId(id.0 - 1, header.parent_hash);
let parent_status = self.status(&parent_id);
let header = QueuedHeader::new(header);
let status = match parent_status {
HeaderStatus::Unknown | HeaderStatus::MaybeOrphan => {
insert_header(&mut self.maybe_orphan, id, header);
HeaderStatus::MaybeOrphan
},
HeaderStatus::Orphan => {
insert_header(&mut self.orphan, id, header);
HeaderStatus::Orphan
}
HeaderStatus::MaybeReceipts | HeaderStatus::Receipts | HeaderStatus::Ready
| HeaderStatus::Submitted | HeaderStatus::Synced => {
insert_header(&mut self.maybe_receipts, id, header);
HeaderStatus::MaybeReceipts
},
};
self.known_headers.entry(id.0).or_default().insert(id.1, status);
log::debug!(target: "bridge", "Queueing new Ethereum header: {:?}. Queue: {:?}.", id, status);
}
/// Receive Substrate best header.
pub fn substrate_best_header_response(&mut self, id: &HeaderId) {
// all ancestors of this header are now synced => let's remove them from
// queues
let mut current = *id;
loop {
let header = match self.status(&current) {
HeaderStatus::Unknown => break,
HeaderStatus::MaybeOrphan => remove_header(&mut self.maybe_orphan, &current),
HeaderStatus::Orphan => remove_header(&mut self.orphan, &current),
HeaderStatus::MaybeReceipts => remove_header(&mut self.maybe_receipts, &current),
HeaderStatus::Receipts => remove_header(&mut self.receipts, &current),
HeaderStatus::Ready => remove_header(&mut self.ready, &current),
HeaderStatus::Submitted => remove_header(&mut self.submitted, &current),
HeaderStatus::Synced => break,
}.expect("header has a given status; given queue has the header; qed");
log::debug!(target: "bridge", "Ethereum header {:?} is now {:?}", current, HeaderStatus::Synced);
*self.known_headers
.entry(current.0)
.or_default()
.entry(current.1)
.or_insert(HeaderStatus::Synced) = HeaderStatus::Synced;
current = header.parent_id();
}
// remember that the header is synced
log::debug!(target: "bridge", "Ethereum header {:?} is now {:?}", id, HeaderStatus::Synced);
*self.known_headers
.entry(id.0)
.or_default()
.entry(id.1)
.or_insert(HeaderStatus::Synced) = HeaderStatus::Synced;
// now let's move all descendants from maybe_orphan && orphan queues to
// maybe_receipts queue
move_header_descendants(
&mut [&mut self.maybe_orphan, &mut self.orphan],
&mut self.maybe_receipts,
&mut self.known_headers,
HeaderStatus::MaybeReceipts,
id,
);
}
/// Receive Substrate response for MaybeOrphan request.
pub fn maybe_orphan_response(&mut self, id: &HeaderId, response: bool) {
if !response {
move_header_descendants(
&mut [&mut self.maybe_orphan],
&mut self.orphan,
&mut self.known_headers,
HeaderStatus::Orphan,
&id,
);
return;
}
move_header_descendants(
&mut [&mut self.maybe_orphan, &mut self.orphan],
&mut self.maybe_receipts,
&mut self.known_headers,
HeaderStatus::MaybeReceipts,
&id,
);
}
/// Receive Substrate response for MaybeReceipts request.
pub fn maybe_receipts_response(&mut self, id: &HeaderId, response: bool) {
let (destination_status, destination_queue) = if response {
(HeaderStatus::Receipts, &mut self.receipts)
} else {
(HeaderStatus::Ready, &mut self.ready)
};
move_header(
&mut self.maybe_receipts,
destination_queue,
&mut self.known_headers,
destination_status,
&id,
|header| header,
);
}
/// Receive transactions receipts from Ethereum node.
pub fn receipts_response(&mut self, id: &HeaderId, receipts: Vec<Receipt>) {
move_header(
&mut self.receipts,
&mut self.ready,
&mut self.known_headers,
HeaderStatus::Ready,
id,
|header| header.set_receipts(receipts),
);
}
/// When header is submitted to Substrate node.
pub fn headers_submitted(&mut self, ids: Vec<HeaderId>) {
for id in ids {
move_header(
&mut self.ready,
&mut self.submitted,
&mut self.known_headers,
HeaderStatus::Submitted,
&id,
|header| header,
);
}
}
/// Prune and never accep headers before this block.
pub fn prune(&mut self, prune_border: u64) {
if prune_border <= self.prune_border {
return;
}
prune_queue(&mut self.maybe_orphan, prune_border);
prune_queue(&mut self.orphan, prune_border);
prune_queue(&mut self.maybe_receipts, prune_border);
prune_queue(&mut self.receipts, prune_border);
prune_queue(&mut self.ready, prune_border);
prune_queue(&mut self.submitted, prune_border);
prune_known_headers(&mut self.known_headers, prune_border);
self.prune_border = prune_border;
}
/// Forgets all ever known headers.
pub fn clear(&mut self) {
self.maybe_orphan.clear();
self.orphan.clear();
self.maybe_receipts.clear();
self.receipts.clear();
self.ready.clear();
self.submitted.clear();
self.known_headers.clear();
self.prune_border = 0;
}
}
/// Insert header to the queue.
fn insert_header(queue: &mut HeadersQueue, id: HeaderId, header: QueuedHeader) {
queue.entry(id.0).or_default().insert(id.1, header);
}
/// Remove header from the queue.
fn remove_header(queue: &mut HeadersQueue, id: &HeaderId) -> Option<QueuedHeader> {
let mut headers_at = match queue.entry(id.0) {
BTreeMapEntry::Occupied(headers_at) => headers_at,
BTreeMapEntry::Vacant(_) => return None,
};
let header = headers_at.get_mut().remove(&id.1);
if headers_at.get().is_empty() {
headers_at.remove();
}
header
}
/// Move header from source to destination queue.
fn move_header(
source_queue: &mut HeadersQueue,
destination_queue: &mut HeadersQueue,
known_headers: &mut KnownHeaders,
destination_status: HeaderStatus,
id: &HeaderId,
prepare: impl FnOnce(QueuedHeader) -> QueuedHeader,
) {
let header = match remove_header(source_queue, id) {
Some(header) => prepare(header),
None => return,
};
known_headers.entry(id.0).or_default().insert(id.1, destination_status);
destination_queue.entry(id.0).or_default().insert(id.1, header);
log::debug!(
target: "bridge",
"Ethereum header {:?} is now {:?}",
id,
destination_status,
);
}
/// Move all descendant headers from the source to destination queue.
fn move_header_descendants(
source_queues: &mut [&mut HeadersQueue],
destination_queue: &mut HeadersQueue,
known_headers: &mut KnownHeaders,
destination_status: HeaderStatus,
id: &HeaderId,
) {
let mut current_number = id.0 + 1;
let mut current_parents = HashSet::new();
current_parents.insert(id.1);
while !current_parents.is_empty() {
let mut next_parents = HashSet::new();
for source_queue in source_queues.iter_mut() {
let mut source_entry = match source_queue.entry(current_number) {
BTreeMapEntry::Occupied(source_entry) => source_entry,
BTreeMapEntry::Vacant(_) => continue,
};
let mut headers_to_move = Vec::new();
let children_at_number = source_entry.get().keys().cloned().collect::<Vec<_>>();
for key in children_at_number {
let entry = match source_entry.get_mut().entry(key) {
HashMapEntry::Occupied(entry) => entry,
HashMapEntry::Vacant(_) => unreachable!("iterating existing keys; qed"),
};
if current_parents.contains(&entry.get().header().parent_hash) {
let header_to_move = entry.remove();
let header_to_move_id = header_to_move.id();
known_headers.entry(header_to_move_id.0).or_default().insert(header_to_move_id.1, destination_status);
headers_to_move.push((header_to_move_id, header_to_move));
log::debug!(
target: "bridge",
"Ethereum header {:?} is now {:?}",
header_to_move_id,
destination_status,
);
}
}
if source_entry.get().is_empty() {
source_entry.remove();
}
next_parents.extend(headers_to_move.iter().map(|(id, _)| id.1));
destination_queue
.entry(current_number)
.or_default()
.extend(headers_to_move.into_iter().map(|(id, h)| (id.1, h)))
}
current_number = current_number + 1;
std::mem::swap(&mut current_parents, &mut next_parents);
}
}
/// Return oldest header from the queue.
fn oldest_header(queue: &HeadersQueue) -> Option<&QueuedHeader> {
queue.values().flat_map(|h| h.values()).next()
}
/// Return oldest headers from the queue until functor will return false.
fn oldest_headers(queue: &HeadersQueue, mut f: impl FnMut(&QueuedHeader) -> bool) -> Option<Vec<&QueuedHeader>> {
let result = queue.values()
.flat_map(|h| h.values())
.take_while(|h| f(h))
.collect::<Vec<_>>();
if result.is_empty() {
None
} else {
Some(result)
}
}
/// Forget all headers with number less than given.
fn prune_queue(queue: &mut HeadersQueue, prune_border: u64) {
*queue = queue.split_off(&prune_border);
}
/// Forget all known headers with number less than given.
fn prune_known_headers(known_headers: &mut KnownHeaders, prune_border: u64) {
let new_known_headers = known_headers.split_off(&prune_border);
for (pruned_number, pruned_headers) in &*known_headers {
for pruned_hash in pruned_headers.keys() {
log::debug!(target: "bridge", "Pruning header {:?}.", HeaderId(*pruned_number, *pruned_hash));
}
}
*known_headers = new_known_headers;
}
#[cfg(test)]
pub(crate) mod tests {
use super::*;
pub(crate) fn header(number: u64) -> QueuedHeader {
QueuedHeader::new(Header {
number: Some(number.into()),
hash: Some(hash(number)),
parent_hash: hash(number - 1),
..Default::default()
})
}
pub(crate) fn hash(number: u64) -> H256 {
H256::from_low_u64_le(number)
}
pub(crate) fn id(number: u64) -> HeaderId {
HeaderId(number, hash(number))
}
#[test]
fn total_headers_works() {
// total headers just sums up number of headers in every queue
let mut queue = QueuedHeaders::default();
queue.maybe_orphan.entry(1).or_default().insert(hash(1), Default::default());
queue.maybe_orphan.entry(1).or_default().insert(hash(2), Default::default());
queue.maybe_orphan.entry(2).or_default().insert(hash(3), Default::default());
queue.orphan.entry(3).or_default().insert(hash(4), Default::default());
queue.maybe_receipts.entry(4).or_default().insert(hash(5), Default::default());
queue.ready.entry(5).or_default().insert(hash(6), Default::default());
assert_eq!(queue.total_headers(), 6);
}
#[test]
fn best_queued_number_works() {
// initially there are headers in MaybeOrphan queue only
let mut queue = QueuedHeaders::default();
queue.maybe_orphan.entry(1).or_default().insert(hash(1), Default::default());
queue.maybe_orphan.entry(1).or_default().insert(hash(2), Default::default());
queue.maybe_orphan.entry(3).or_default().insert(hash(3), Default::default());
assert_eq!(queue.best_queued_number(), 3);
// and then there's better header in Orphan
queue.orphan.entry(10).or_default().insert(hash(10), Default::default());
assert_eq!(queue.best_queued_number(), 10);
// and then there's better header in MaybeReceipts
queue.maybe_receipts.entry(20).or_default().insert(hash(20), Default::default());
assert_eq!(queue.best_queued_number(), 20);
// and then there's better header in Ready
queue.ready.entry(30).or_default().insert(hash(30), Default::default());
assert_eq!(queue.best_queued_number(), 30);
// and then there's better header in MaybeOrphan again
queue.maybe_orphan.entry(40).or_default().insert(hash(40), Default::default());
assert_eq!(queue.best_queued_number(), 40);
}
#[test]
fn status_works() {
// all headers are unknown initially
let mut queue = QueuedHeaders::default();
assert_eq!(queue.status(&id(10)), HeaderStatus::Unknown);
// and status is read from the KnownHeaders
queue.known_headers.entry(10).or_default().insert(hash(10), HeaderStatus::Ready);
assert_eq!(queue.status(&id(10)), HeaderStatus::Ready);
}
#[test]
fn header_works() {
// initially we have oldest header #10
let mut queue = QueuedHeaders::default();
queue.maybe_orphan.entry(10).or_default().insert(hash(1), header(100));
assert_eq!(queue.header(HeaderStatus::MaybeOrphan).unwrap().header().hash.unwrap(), hash(100));
// inserting #20 changes nothing
queue.maybe_orphan.entry(20).or_default().insert(hash(1), header(101));
assert_eq!(queue.header(HeaderStatus::MaybeOrphan).unwrap().header().hash.unwrap(), hash(100));
// inserting #5 makes it oldest
queue.maybe_orphan.entry(5).or_default().insert(hash(1), header(102));
assert_eq!(queue.header(HeaderStatus::MaybeOrphan).unwrap().header().hash.unwrap(), hash(102));
}
#[test]
fn header_response_works() {
// when parent is Synced, we insert to MaybeReceipts
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::Synced);
queue.header_response(header(101).header().clone());
assert_eq!(queue.status(&id(101)), HeaderStatus::MaybeReceipts);
// when parent is Ready, we insert to MaybeReceipts
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::Ready);
queue.header_response(header(101).header().clone());
assert_eq!(queue.status(&id(101)), HeaderStatus::MaybeReceipts);
// when parent is Receipts, we insert to MaybeReceipts
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::Receipts);
queue.header_response(header(101).header().clone());
assert_eq!(queue.status(&id(101)), HeaderStatus::MaybeReceipts);
// when parent is MaybeReceipts, we insert to MaybeReceipts
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::MaybeReceipts);
queue.header_response(header(101).header().clone());
assert_eq!(queue.status(&id(101)), HeaderStatus::MaybeReceipts);
// when parent is Orphan, we insert to Orphan
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::Orphan);
queue.header_response(header(101).header().clone());
assert_eq!(queue.status(&id(101)), HeaderStatus::Orphan);
// when parent is MaybeOrphan, we insert to MaybeOrphan
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::MaybeOrphan);
queue.header_response(header(101).header().clone());
assert_eq!(queue.status(&id(101)), HeaderStatus::MaybeOrphan);
// when parent is unknown, we insert to MaybeOrphan
let mut queue = QueuedHeaders::default();
queue.header_response(header(101).header().clone());
assert_eq!(queue.status(&id(101)), HeaderStatus::MaybeOrphan);
}
#[test]
fn ancestors_are_synced_on_substrate_best_header_response() {
// let's say someone else has submitted transaction to bridge that changes
// its best block to #100. At this time we have:
// #100 in MaybeOrphan
// #99 in Orphan
// #98 in MaybeReceipts
// #97 in Receipts
// #96 in Ready
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::MaybeOrphan);
queue.maybe_orphan.entry(100).or_default().insert(hash(100), header(100));
queue.known_headers.entry(99).or_default().insert(hash(99), HeaderStatus::Orphan);
queue.orphan.entry(99).or_default().insert(hash(99), header(99));
queue.known_headers.entry(98).or_default().insert(hash(98), HeaderStatus::MaybeReceipts);
queue.maybe_receipts.entry(98).or_default().insert(hash(98), header(98));
queue.known_headers.entry(97).or_default().insert(hash(97), HeaderStatus::Receipts);
queue.receipts.entry(97).or_default().insert(hash(97), header(97));
queue.known_headers.entry(96).or_default().insert(hash(96), HeaderStatus::Ready);
queue.ready.entry(96).or_default().insert(hash(96), header(96));
queue.substrate_best_header_response(&id(100));
// then the #100 and all ancestors of #100 (#96..#99) are treated as synced
assert!(queue.maybe_orphan.is_empty());
assert!(queue.orphan.is_empty());
assert!(queue.maybe_receipts.is_empty());
assert!(queue.receipts.is_empty());
assert!(queue.ready.is_empty());
assert_eq!(queue.known_headers.len(), 5);
assert!(queue.known_headers.values().all(|s| s.values().all(|s| *s == HeaderStatus::Synced)));
}
#[test]
fn descendants_are_moved_on_substrate_best_header_response() {
// let's say someone else has submitted transaction to bridge that changes
// its best block to #100. At this time we have:
// #101 in Orphan
// #102 in MaybeOrphan
// #103 in Orphan
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(101).or_default().insert(hash(101), HeaderStatus::Orphan);
queue.orphan.entry(101).or_default().insert(hash(101), header(101));
queue.known_headers.entry(102).or_default().insert(hash(102), HeaderStatus::MaybeOrphan);
queue.maybe_orphan.entry(102).or_default().insert(hash(102), header(102));
queue.known_headers.entry(103).or_default().insert(hash(103), HeaderStatus::Orphan);
queue.orphan.entry(103).or_default().insert(hash(103), header(103));
queue.substrate_best_header_response(&id(100));
// all descendants are moved to MaybeReceipts
assert!(queue.maybe_orphan.is_empty());
assert!(queue.orphan.is_empty());
assert_eq!(queue.maybe_receipts.len(), 3);
assert_eq!(queue.known_headers[&101][&hash(101)], HeaderStatus::MaybeReceipts);
assert_eq!(queue.known_headers[&102][&hash(102)], HeaderStatus::MaybeReceipts);
assert_eq!(queue.known_headers[&103][&hash(103)], HeaderStatus::MaybeReceipts);
}
#[test]
fn positive_maybe_orphan_response_works() {
// let's say we have:
// #100 in MaybeOrphan
// #101 in Orphan
// #102 in MaybeOrphan
// and we have asked for MaybeOrphan status of #100.parent (i.e. #99)
// and the response is: YES, #99 is known to the Substrate runtime
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::MaybeOrphan);
queue.maybe_orphan.entry(100).or_default().insert(hash(100), header(100));
queue.known_headers.entry(101).or_default().insert(hash(101), HeaderStatus::Orphan);
queue.orphan.entry(101).or_default().insert(hash(101), header(101));
queue.known_headers.entry(102).or_default().insert(hash(102), HeaderStatus::MaybeOrphan);
queue.maybe_orphan.entry(102).or_default().insert(hash(102), header(102));
queue.maybe_orphan_response(&id(99), true);
// then all headers (#100..#103) are moved to the MaybeReceipts queue
assert!(queue.orphan.is_empty());
assert!(queue.maybe_orphan.is_empty());
assert_eq!(queue.maybe_receipts.len(), 3);
assert_eq!(queue.known_headers[&100][&hash(100)], HeaderStatus::MaybeReceipts);
assert_eq!(queue.known_headers[&101][&hash(101)], HeaderStatus::MaybeReceipts);
assert_eq!(queue.known_headers[&102][&hash(102)], HeaderStatus::MaybeReceipts);
}
#[test]
fn negative_maybe_orphan_response_works() {
// let's say we have:
// #100 in MaybeOrphan
// #101 in MaybeOrphan
// and we have asked for MaybeOrphan status of #100.parent (i.e. #99)
// and the response is: NO, #99 is NOT known to the Substrate runtime
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::MaybeOrphan);
queue.maybe_orphan.entry(100).or_default().insert(hash(100), header(100));
queue.known_headers.entry(101).or_default().insert(hash(101), HeaderStatus::MaybeOrphan);
queue.maybe_orphan.entry(101).or_default().insert(hash(101), header(101));
queue.maybe_orphan_response(&id(99), false);
// then all headers (#100..#101) are moved to the Orphan queue
assert!(queue.maybe_orphan.is_empty());
assert_eq!(queue.orphan.len(), 2);
assert_eq!(queue.known_headers[&100][&hash(100)], HeaderStatus::Orphan);
assert_eq!(queue.known_headers[&101][&hash(101)], HeaderStatus::Orphan);
}
#[test]
fn positive_maybe_receipts_response_works() {
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::MaybeReceipts);
queue.maybe_receipts.entry(100).or_default().insert(hash(100), header(100));
queue.maybe_receipts_response(&id(100), true);
assert!(queue.maybe_receipts.is_empty());
assert_eq!(queue.receipts.len(), 1);
assert_eq!(queue.known_headers[&100][&hash(100)], HeaderStatus::Receipts);
}
#[test]
fn negative_maybe_receipts_response_works() {
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::MaybeReceipts);
queue.maybe_receipts.entry(100).or_default().insert(hash(100), header(100));
queue.maybe_receipts_response(&id(100), false);
assert!(queue.maybe_receipts.is_empty());
assert_eq!(queue.ready.len(), 1);
assert_eq!(queue.known_headers[&100][&hash(100)], HeaderStatus::Ready);
}
#[test]
fn receipts_response_works() {
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::Receipts);
queue.receipts.entry(100).or_default().insert(hash(100), header(100));
queue.receipts_response(&id(100), Vec::new());
assert!(queue.receipts.is_empty());
assert_eq!(queue.ready.len(), 1);
assert_eq!(queue.known_headers[&100][&hash(100)], HeaderStatus::Ready);
}
#[test]
fn header_submitted_works() {
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::Ready);
queue.ready.entry(100).or_default().insert(hash(100), header(100));
queue.headers_submitted(vec![id(100)]);
assert!(queue.ready.is_empty());
assert_eq!(queue.known_headers[&100][&hash(100)], HeaderStatus::Submitted);
}
#[test]
fn prune_works() {
let mut queue = QueuedHeaders::default();
queue.known_headers.entry(104).or_default().insert(hash(104), HeaderStatus::MaybeOrphan);
queue.maybe_orphan.entry(104).or_default().insert(hash(104), header(104));
queue.known_headers.entry(103).or_default().insert(hash(103), HeaderStatus::Orphan);
queue.orphan.entry(103).or_default().insert(hash(103), header(103));
queue.known_headers.entry(102).or_default().insert(hash(102), HeaderStatus::MaybeReceipts);
queue.maybe_receipts.entry(102).or_default().insert(hash(102), header(102));
queue.known_headers.entry(101).or_default().insert(hash(101), HeaderStatus::Receipts);
queue.receipts.entry(101).or_default().insert(hash(101), header(101));
queue.known_headers.entry(100).or_default().insert(hash(100), HeaderStatus::Ready);
queue.ready.entry(100).or_default().insert(hash(100), header(100));
queue.prune(102);
assert_eq!(queue.ready.len(), 0);
assert_eq!(queue.receipts.len(), 0);
assert_eq!(queue.maybe_receipts.len(), 1);
assert_eq!(queue.orphan.len(), 1);
assert_eq!(queue.maybe_orphan.len(), 1);
assert_eq!(queue.known_headers.len(), 3);
queue.prune(110);
assert_eq!(queue.ready.len(), 0);
assert_eq!(queue.receipts.len(), 0);
assert_eq!(queue.maybe_receipts.len(), 0);
assert_eq!(queue.orphan.len(), 0);
assert_eq!(queue.maybe_orphan.len(), 0);
assert_eq!(queue.known_headers.len(), 0);
queue.header_response(header(109).header().clone());
assert_eq!(queue.known_headers.len(), 0);
queue.header_response(header(110).header().clone());
assert_eq!(queue.known_headers.len(), 1);
}
}
bridges/relays/ethereum/src/ethereum_sync.rs
+311
View File
@@ -0,0 +1,311 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
use codec::Encode;
use crate::ethereum_headers::QueuedHeaders;
use crate::ethereum_types::{HeaderId, HeaderStatus, QueuedHeader};
use crate::ethereum_sync_loop::EthereumSyncParams;
use crate::substrate_types::{into_substrate_ethereum_header, into_substrate_ethereum_receipts};
/// Ethereum headers synchronization context.
#[derive(Debug)]
pub struct HeadersSync {
/// Synchronization parameters.
params: EthereumSyncParams,
/// Best header number known to Ethereum node.
target_header_number: Option<u64>,
/// Best header known to Substrate node.
best_header: Option<HeaderId>,
/// Headers queue.
headers: QueuedHeaders,
}
impl HeadersSync {
/// Creates new Ethereum headers synchronizer.
pub fn new(params: EthereumSyncParams) -> Self {
HeadersSync {
params,
target_header_number: None,
best_header: None,
headers: Default::default(),
}
}
/// Returns true if we have synced almost all known headers.
pub fn is_almost_synced(&self) -> bool {
match self.target_header_number {
Some(target_header_number) => self.best_header
.map(|best| target_header_number.saturating_sub(best.0) < 4)
.unwrap_or(false),
None => true,
}
}
/// Returns synchronization status.
pub fn status(&self) -> (&Option<HeaderId>, &Option<u64>) {
(&self.best_header, &self.target_header_number)
}
/// Returns reference to the headers queue.
pub fn headers(&self) -> &QueuedHeaders {
&self.headers
}
/// Returns mutable reference to the headers queue.
pub fn headers_mut(&mut self) -> &mut QueuedHeaders {
&mut self.headers
}
/// Select header that needs to be downloaded from the Ethereum node.
pub fn select_new_header_to_download(&self) -> Option<u64> {
// if we haven't received best header from Ethereum node yet, there's nothing we can download
let target_header_number = self.target_header_number.clone()?;
// if we haven't received known best header from Substrate node yet, there's nothing we can download
let best_header = self.best_header.as_ref()?;
// if there's too many headers in the queue, stop downloading
let in_memory_headers = self.headers.total_headers();
if in_memory_headers >= self.params.max_future_headers_to_download {
return None;
}
// we assume that there were no reorgs if we have already downloaded best header
let best_downloaded_number = std::cmp::max(
self.headers.best_queued_number(),
best_header.0,
);
if best_downloaded_number == target_header_number {
return None;
}
// download new header
Some(best_downloaded_number + 1)
}
/// Select headers that need to be submitted to the Substrate node.
pub fn select_headers_to_submit(&self) -> Option<Vec<&QueuedHeader>> {
let headers_in_submit_status = self.headers.headers_in_status(HeaderStatus::Submitted);
let headers_to_submit_count = self.params.max_headers_in_submitted_status
.checked_sub(headers_in_submit_status)?;
let mut total_size = 0;
let mut total_headers = 0;
self.headers.headers(HeaderStatus::Ready, |header| {
if total_headers == headers_to_submit_count {
return false;
}
if total_headers == self.params.max_headers_in_single_submit {
return false;
}
let encoded_size = into_substrate_ethereum_header(header.header()).encode().len()
+ into_substrate_ethereum_receipts(header.receipts()).map(|receipts| receipts.encode().len())
.unwrap_or(0);
if total_headers != 0 && total_size + encoded_size > self.params.max_headers_size_in_single_submit {
return false;
}
total_size += encoded_size;
total_headers += 1;
true
})
}
/// Receive new target header number from the Ethereum node.
pub fn ethereum_best_header_number_response(&mut self, best_header_number: u64) {
log::debug!(target: "bridge", "Received best header number from Ethereum: {}", best_header_number);
self.target_header_number = Some(best_header_number);
}
/// Receive new best header from the Substrate node.
/// Returns true if it is different from the previous block known to us.
pub fn substrate_best_header_response(&mut self, best_header: HeaderId) -> bool {
log::debug!(target: "bridge", "Received best known header from Substrate: {:?}", best_header);
// early return if it is still the same
if self.best_header == Some(best_header) {
return false;
}
// remember that this header is now known to the Substrate runtime
self.headers.substrate_best_header_response(&best_header);
// prune ancient headers
self.headers.prune(best_header.0.saturating_sub(self.params.prune_depth));
// finally remember the best header itself
self.best_header = Some(best_header);
true
}
/// Restart synchronization.
pub fn restart(&mut self) {
self.target_header_number = None;
self.best_header = None;
self.headers.clear();
}
}
#[cfg(test)]
mod tests {
use crate::ethereum_headers::tests::{header, id};
use crate::ethereum_types::{H256, HeaderStatus};
use super::*;
fn side_hash(number: u64) -> H256 {
H256::from_low_u64_le(1000 + number)
}
#[test]
fn select_new_header_to_download_works() {
let mut eth_sync = HeadersSync::new(Default::default());
// both best && target headers are unknown
assert_eq!(eth_sync.select_new_header_to_download(), None);
// best header is known, target header is unknown
eth_sync.best_header = Some(HeaderId(0, Default::default()));
assert_eq!(eth_sync.select_new_header_to_download(), None);
// target header is known, best header is unknown
eth_sync.best_header = None;
eth_sync.target_header_number = Some(100);
assert_eq!(eth_sync.select_new_header_to_download(), None);
// when our best block has the same number as the target
eth_sync.best_header = Some(HeaderId(100, Default::default()));
assert_eq!(eth_sync.select_new_header_to_download(), None);
// when we actually need a new header
eth_sync.target_header_number = Some(101);
assert_eq!(eth_sync.select_new_header_to_download(), Some(101));
// when there are too many headers scheduled for submitting
for i in 1..1000 {
eth_sync.headers.header_response(header(i).header().clone());
}
assert_eq!(eth_sync.select_new_header_to_download(), None);
}
#[test]
fn sync_without_reorgs_works() {
let mut eth_sync = HeadersSync::new(Default::default());
eth_sync.params.max_headers_in_submitted_status = 1;
// ethereum reports best header #102
eth_sync.ethereum_best_header_number_response(102);
// substrate reports that it is at block #100
eth_sync.substrate_best_header_response(id(100));
// block #101 is downloaded first
assert_eq!(eth_sync.select_new_header_to_download(), Some(101));
eth_sync.headers.header_response(header(101).header().clone());
// now header #101 is ready to be submitted
assert_eq!(eth_sync.headers.header(HeaderStatus::MaybeReceipts), Some(&header(101)));
eth_sync.headers.maybe_receipts_response(&id(101), false);
assert_eq!(eth_sync.headers.header(HeaderStatus::Ready), Some(&header(101)));
assert_eq!(eth_sync.select_headers_to_submit(), Some(vec![&header(101)]));
// and header #102 is ready to be downloaded
assert_eq!(eth_sync.select_new_header_to_download(), Some(102));
eth_sync.headers.header_response(header(102).header().clone());
// receive submission confirmation
eth_sync.headers.headers_submitted(vec![id(101)]);
// we have nothing to submit because previous header hasn't been confirmed yet
// (and we allow max 1 submit transaction in the wild)
assert_eq!(eth_sync.headers.header(HeaderStatus::MaybeReceipts), Some(&header(102)));
eth_sync.headers.maybe_receipts_response(&id(102), false);
assert_eq!(eth_sync.headers.header(HeaderStatus::Ready), Some(&header(102)));
assert_eq!(eth_sync.select_headers_to_submit(), None);
// substrate reports that it has imported block #101
eth_sync.substrate_best_header_response(id(101));
// and we are ready to submit #102
assert_eq!(eth_sync.select_headers_to_submit(), Some(vec![&header(102)]));
eth_sync.headers.headers_submitted(vec![id(102)]);
// substrate reports that it has imported block #102
eth_sync.substrate_best_header_response(id(102));
// and we have nothing to download
assert_eq!(eth_sync.select_new_header_to_download(), None);
}
#[test]
fn sync_with_orphan_headers_work() {
let mut eth_sync = HeadersSync::new(Default::default());
// ethereum reports best header #102
eth_sync.ethereum_best_header_number_response(102);
// substrate reports that it is at block #100, but it isn't part of best chain
eth_sync.substrate_best_header_response(HeaderId(100, side_hash(100)));
// block #101 is downloaded first
assert_eq!(eth_sync.select_new_header_to_download(), Some(101));
eth_sync.headers.header_response(header(101).header().clone());
// we can't submit header #101, because its parent status is unknown
assert_eq!(eth_sync.select_headers_to_submit(), None);
// instead we are trying to determine status of its parent (#100)
assert_eq!(eth_sync.headers.header(HeaderStatus::MaybeOrphan), Some(&header(101)));
// and the status is still unknown
eth_sync.headers.maybe_orphan_response(&id(100), false);
// so we consider #101 orphaned now && will download its parent - #100
assert_eq!(eth_sync.headers.header(HeaderStatus::Orphan), Some(&header(101)));
eth_sync.headers.header_response(header(100).header().clone());
// we can't submit header #100, because its parent status is unknown
assert_eq!(eth_sync.select_headers_to_submit(), None);
// instead we are trying to determine status of its parent (#99)
assert_eq!(eth_sync.headers.header(HeaderStatus::MaybeOrphan), Some(&header(100)));
// and the status is known, so we move previously orphaned #100 and #101 to ready queue
eth_sync.headers.maybe_orphan_response(&id(99), true);
// and we are ready to submit #100
assert_eq!(eth_sync.headers.header(HeaderStatus::MaybeReceipts), Some(&header(100)));
eth_sync.headers.maybe_receipts_response(&id(100), false);
assert_eq!(eth_sync.select_headers_to_submit(), Some(vec![&header(100)]));
eth_sync.headers.headers_submitted(vec![id(100)]);
// and we are ready to submit #101
assert_eq!(eth_sync.headers.header(HeaderStatus::MaybeReceipts), Some(&header(101)));
eth_sync.headers.maybe_receipts_response(&id(101), false);
assert_eq!(eth_sync.select_headers_to_submit(), Some(vec![&header(101)]));
eth_sync.headers.headers_submitted(vec![id(101)]);
}
#[test]
fn pruning_happens_on_substrate_best_header_response() {
let mut eth_sync = HeadersSync::new(Default::default());
eth_sync.params.prune_depth = 50;
eth_sync.substrate_best_header_response(id(100));
assert_eq!(eth_sync.headers.prune_border(), 50);
}
}
bridges/relays/ethereum/src/ethereum_sync_loop.rs
+486
View File
@@ -0,0 +1,486 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
use futures::{future::FutureExt, stream::StreamExt};
use crate::ethereum_client;
use crate::ethereum_types::HeaderStatus as EthereumHeaderStatus;
use crate::substrate_client;
// TODO: when SharedClient will be available, switch to Substrate headers subscription
// (because we do not need old Substrate headers)
/// Interval (in ms) at which we check new Ethereum headers when we are synced/almost synced.
const ETHEREUM_TICK_INTERVAL_MS: u64 = 10_000;
/// Interval (in ms) at which we check new Substrate blocks.
const SUBSTRATE_TICK_INTERVAL_MS: u64 = 5_000;
/// When we submit Ethereum headers to Substrate runtime, but see no updates of best
/// Ethereum block known to Substrate runtime during STALL_SYNC_TIMEOUT_MS milliseconds,
/// we consider that our headers are rejected because there has been reorg in Substrate.
/// This reorg could invalidate our knowledge about sync process (i.e. we have asked if
/// HeaderA is known to Substrate, but then reorg happened and the answer is different
/// now) => we need to reset sync.
/// The other option is to receive **EVERY** best Substrate header and check if it is
/// direct child of previous best header. But: (1) subscription doesn't guarantee that
/// the subscriber will receive every best header (2) reorg won't always lead to sync
/// stall and restart is a heavy operation (we forget all in-memory headers).
const STALL_SYNC_TIMEOUT_MS: u64 = 30_000;
/// Delay (in milliseconds) after connection-related error happened before we'll try
/// reconnection again.
const CONNECTION_ERROR_DELAY_MS: u64 = 10_000;
/// Error type that can signal connection errors.
pub trait MaybeConnectionError {
/// Returns true if error (maybe) represents connection error.
fn is_connection_error(&self) -> bool;
}
/// Ethereum synchronization parameters.
pub struct EthereumSyncParams {
/// Ethereum RPC host.
pub eth_host: String,
/// Ethereum RPC port.
pub eth_port: u16,
/// Substrate RPC host.
pub sub_host: String,
/// Substrate RPC port.
pub sub_port: u16,
/// Substrate transactions signer.
pub sub_signer: sp_core::sr25519::Pair,
/// Maximal number of ethereum headers to pre-download.
pub max_future_headers_to_download: usize,
/// Maximal number of active (we believe) submit header transactions.
pub max_headers_in_submitted_status: usize,
/// Maximal number of headers in single submit request.
pub max_headers_in_single_submit: usize,
/// Maximal total headers size in single submit request.
pub max_headers_size_in_single_submit: usize,
/// We only may store and accept (from Ethereum node) headers that have
/// number >= than best_substrate_header.number - prune_depth.
pub prune_depth: u64,
}
impl std::fmt::Debug for EthereumSyncParams {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
f.debug_struct("EthereumSyncParams")
.field("eth_host", &self.eth_host)
.field("eth_port", &self.eth_port)
.field("sub_host", &self.sub_port)
.field("sub_port", &self.sub_port)
.field("max_future_headers_to_download", &self.max_future_headers_to_download)
.field("max_headers_in_submitted_status", &self.max_headers_in_submitted_status)
.field("max_headers_in_single_submit", &self.max_headers_in_single_submit)
.field("max_headers_size_in_single_submit", &self.max_headers_size_in_single_submit)
.field("prune_depth", &self.prune_depth)
.finish()
}
}
impl Default for EthereumSyncParams {
fn default() -> Self {
EthereumSyncParams {
eth_host: "localhost".into(),
eth_port: 8545,
sub_host: "localhost".into(),
sub_port: 9933,
sub_signer: sp_keyring::AccountKeyring::Alice.pair(),
max_future_headers_to_download: 128,
max_headers_in_submitted_status: 128,
max_headers_in_single_submit: 32,
max_headers_size_in_single_submit: 131_072,
prune_depth: 4096,
}
}
}
/// Run Ethereum headers synchronization.
pub fn run(params: EthereumSyncParams) {
let mut local_pool = futures::executor::LocalPool::new();
let mut progress_context = (std::time::Instant::now(), None, None);
local_pool.run_until(async move {
let eth_uri = format!("http://{}:{}", params.eth_host, params.eth_port);
let sub_uri = format!("http://{}:{}", params.sub_host, params.sub_port);
let sub_signer = params.sub_signer.clone();
let mut eth_sync = crate::ethereum_sync::HeadersSync::new(params);
let mut stall_countdown = None;
let mut eth_maybe_client = None;
let mut eth_best_block_number_required = false;
let eth_best_block_number_future = ethereum_client::best_block_number(
ethereum_client::client(&eth_uri)
).fuse();
let eth_new_header_future = futures::future::Fuse::terminated();
let eth_orphan_header_future = futures::future::Fuse::terminated();
let eth_receipts_future = futures::future::Fuse::terminated();
let eth_go_offline_future = futures::future::Fuse::terminated();
let eth_tick_stream = interval(ETHEREUM_TICK_INTERVAL_MS).fuse();
let mut sub_maybe_client = None;
let mut sub_best_block_required = false;
let sub_best_block_future = substrate_client::best_ethereum_block(
substrate_client::client(&sub_uri, sub_signer),
).fuse();
let sub_receipts_check_future = futures::future::Fuse::terminated();
let sub_existence_status_future = futures::future::Fuse::terminated();
let sub_submit_header_future = futures::future::Fuse::terminated();
let sub_go_offline_future = futures::future::Fuse::terminated();
let sub_tick_stream = interval(SUBSTRATE_TICK_INTERVAL_MS).fuse();
futures::pin_mut!(
eth_best_block_number_future,
eth_new_header_future,
eth_orphan_header_future,
eth_receipts_future,
eth_go_offline_future,
eth_tick_stream,
sub_best_block_future,
sub_receipts_check_future,
sub_existence_status_future,
sub_submit_header_future,
sub_go_offline_future,
sub_tick_stream
);
loop {
futures::select! {
(eth_client, eth_best_block_number) = eth_best_block_number_future => {
eth_best_block_number_required = false;
process_future_result(
&mut eth_maybe_client,
eth_client,
eth_best_block_number,
|eth_best_block_number| eth_sync.ethereum_best_header_number_response(eth_best_block_number),
&mut eth_go_offline_future,
|eth_client| delay(CONNECTION_ERROR_DELAY_MS, eth_client),
"Error retrieving best header number from Ethereum number",
);
},
(eth_client, eth_new_header) = eth_new_header_future => {
process_future_result(
&mut eth_maybe_client,
eth_client,
eth_new_header,
|eth_new_header| eth_sync.headers_mut().header_response(eth_new_header),
&mut eth_go_offline_future,
|eth_client| delay(CONNECTION_ERROR_DELAY_MS, eth_client),
"Error retrieving header from Ethereum node",
);
},
(eth_client, eth_orphan_header) = eth_orphan_header_future => {
process_future_result(
&mut eth_maybe_client,
eth_client,
eth_orphan_header,
|eth_orphan_header| eth_sync.headers_mut().header_response(eth_orphan_header),
&mut eth_go_offline_future,
|eth_client| delay(CONNECTION_ERROR_DELAY_MS, eth_client),
"Error retrieving orphan header from Ethereum node",
);
},
(eth_client, eth_receipts) = eth_receipts_future => {
process_future_result(
&mut eth_maybe_client,
eth_client,
eth_receipts,
|(header, receipts)| eth_sync.headers_mut().receipts_response(&header, receipts),
&mut eth_go_offline_future,
|eth_client| delay(CONNECTION_ERROR_DELAY_MS, eth_client),
"Error retrieving transactions receipts from Ethereum node",
);
},
eth_client = eth_go_offline_future => {
eth_maybe_client = Some(eth_client);
},
_ = eth_tick_stream.next() => {
if eth_sync.is_almost_synced() {
eth_best_block_number_required = true;
}
},
(sub_client, sub_best_block) = sub_best_block_future => {
sub_best_block_required = false;
process_future_result(
&mut sub_maybe_client,
sub_client,
sub_best_block,
|sub_best_block| {
let head_updated = eth_sync.substrate_best_header_response(sub_best_block);
match head_updated {
// IF head is updated AND there are still our transactions:
// => restart stall countdown timer
true if eth_sync.headers().headers_in_status(EthereumHeaderStatus::Submitted) != 0 =>
stall_countdown = Some(std::time::Instant::now()),
// IF head is updated AND there are no our transactions:
// => stop stall countdown timer
true => stall_countdown = None,
// IF head is not updated AND stall countdown is not yet completed
// => do nothing
false if stall_countdown
.map(|stall_countdown| std::time::Instant::now() - stall_countdown <
std::time::Duration::from_millis(STALL_SYNC_TIMEOUT_MS))
.unwrap_or(true)
=> (),
// IF head is not updated AND stall countdown has completed
// => restart sync
false => {
log::info!(
target: "bridge",
"Possible Substrate fork detected. Restarting Ethereum headers synchronization.",
);
stall_countdown = None;
eth_sync.restart();
},
}
},
&mut sub_go_offline_future,
|sub_client| delay(CONNECTION_ERROR_DELAY_MS, sub_client),
"Error retrieving best known header from Substrate node",
);
},
(sub_client, sub_existence_status) = sub_existence_status_future => {
process_future_result(
&mut sub_maybe_client,
sub_client,
sub_existence_status,
|(sub_header, sub_existence_status)| eth_sync
.headers_mut()
.maybe_orphan_response(&sub_header, sub_existence_status),
&mut sub_go_offline_future,
|sub_client| delay(CONNECTION_ERROR_DELAY_MS, sub_client),
"Error retrieving existence status from Substrate node",
);
},
(sub_client, sub_submit_header_result) = sub_submit_header_future => {
process_future_result(
&mut sub_maybe_client,
sub_client,
sub_submit_header_result,
|(_, submitted_headers)| eth_sync.headers_mut().headers_submitted(submitted_headers),
&mut sub_go_offline_future,
|sub_client| delay(CONNECTION_ERROR_DELAY_MS, sub_client),
"Error submitting headers to Substrate node",
);
},
(sub_client, sub_receipts_check_result) = sub_receipts_check_future => {
// we can minimize number of receipts_check calls by checking header
// logs bloom here, but it may give us false positives (when authorities
// source is contract, we never need any logs)
process_future_result(
&mut sub_maybe_client,
sub_client,
sub_receipts_check_result,
|(header, receipts_check_result)| eth_sync
.headers_mut()
.maybe_receipts_response(&header, receipts_check_result),
&mut sub_go_offline_future,
|sub_client| delay(CONNECTION_ERROR_DELAY_MS, sub_client),
"Error retrieving receipts requirement from Substrate node",
);
},
sub_client = sub_go_offline_future => {
sub_maybe_client = Some(sub_client);
},
_ = sub_tick_stream.next() => {
sub_best_block_required = true;
},
}
// print progress
progress_context = print_progress(progress_context, &eth_sync);
// if client is available: wait, or call Substrate RPC methods
if let Some(sub_client) = sub_maybe_client.take() {
// the priority is to:
// 1) get best block - it stops us from downloading/submitting new blocks + we call it rarely;
// 2) check transactions receipts - it stops us from downloading/submitting new blocks;
// 3) check existence - it stops us from submitting new blocks;
// 4) submit header
if sub_best_block_required {
log::debug!(target: "bridge", "Asking Substrate about best block");
sub_best_block_future.set(substrate_client::best_ethereum_block(sub_client).fuse());
} else if let Some(header) = eth_sync.headers().header(EthereumHeaderStatus::MaybeReceipts) {
log::debug!(
target: "bridge",
"Checking if header submission requires receipts: {:?}",
header.id(),
);
let header = header.clone();
sub_receipts_check_future.set(
substrate_client::ethereum_receipts_required(sub_client, header).fuse()
);
} else if let Some(header) = eth_sync.headers().header(EthereumHeaderStatus::MaybeOrphan) {
// for MaybeOrphan we actually ask for parent' header existence
let parent_id = header.parent_id();
log::debug!(
target: "bridge",
"Asking Substrate node for existence of: {:?}",
parent_id,
);
sub_existence_status_future.set(
substrate_client::ethereum_header_known(sub_client, parent_id).fuse(),
);
} else if let Some(headers) = eth_sync.select_headers_to_submit() {
let ids = match headers.len() {
1 => format!("{:?}", headers[0].id()),
2 => format!("[{:?}, {:?}]", headers[0].id(), headers[1].id()),
len => format!("[{:?} ... {:?}]", headers[0].id(), headers[len - 1].id()),
};
log::debug!(
target: "bridge",
"Submitting {} header(s) to Substrate node: {:?}",
headers.len(),
ids,
);
let headers = headers.into_iter().cloned().collect();
sub_submit_header_future.set(
substrate_client::submit_ethereum_headers(sub_client, headers).fuse(),
);
// remember that we have submitted some headers
if stall_countdown.is_none() {
stall_countdown = Some(std::time::Instant::now());
}
} else {
sub_maybe_client = Some(sub_client);
}
}
// if client is available: wait, or call Ethereum RPC methods
if let Some(eth_client) = eth_maybe_client.take() {
// the priority is to:
// 1) get best block - it stops us from downloading new blocks + we call it rarely;
// 2) check transactions receipts - it stops us from downloading/submitting new blocks;
// 3) check existence - it stops us from submitting new blocks;
// 4) submit header
if eth_best_block_number_required {
log::debug!(target: "bridge", "Asking Ethereum node about best block number");
eth_best_block_number_future.set(ethereum_client::best_block_number(eth_client).fuse());
} else if let Some(header) = eth_sync.headers().header(EthereumHeaderStatus::Receipts) {
let id = header.id();
log::debug!(
target: "bridge",
"Retrieving receipts for header: {:?}",
id,
);
eth_receipts_future.set(
ethereum_client::transactions_receipts(
eth_client,
id,
header.header().transactions.clone(),
).fuse()
);
} else if let Some(header) = eth_sync.headers().header(EthereumHeaderStatus::Orphan) {
// for Orphan we actually ask for parent' header
let parent_id = header.parent_id();
log::debug!(
target: "bridge",
"Going to download orphan header from Ethereum node: {:?}",
parent_id,
);
eth_orphan_header_future.set(
ethereum_client::header_by_hash(eth_client, parent_id.1).fuse(),
);
} else if let Some(id) = eth_sync.select_new_header_to_download() {
log::debug!(
target: "bridge",
"Going to download new header from Ethereum node: {:?}",
id,
);
eth_new_header_future.set(
ethereum_client::header_by_number(eth_client, id).fuse(),
);
} else {
eth_maybe_client = Some(eth_client);
}
}
}
});
}
fn print_progress(
progress_context: (std::time::Instant, Option<u64>, Option<u64>),
eth_sync: &crate::ethereum_sync::HeadersSync,
) -> (std::time::Instant, Option<u64>, Option<u64>) {
let (prev_time, prev_best_header, prev_target_header) = progress_context;
let now_time = std::time::Instant::now();
let (now_best_header, now_target_header) = eth_sync.status();
let need_update = now_time - prev_time > std::time::Duration::from_secs(10)
|| match (prev_best_header, now_best_header) {
(Some(prev_best_header), Some(now_best_header)) => now_best_header.0.saturating_sub(prev_best_header) > 10,
_ => false,
};
if !need_update {
return (prev_time, prev_best_header, prev_target_header);
}
log::info!(
target: "bridge",
"Synced {:?} of {:?} headers",
now_best_header.map(|id| id.0),
now_target_header,
);
(now_time, now_best_header.clone().map(|id| id.0), *now_target_header)
}
async fn delay<T>(timeout_ms: u64, retval: T) -> T {
async_std::task::sleep(std::time::Duration::from_millis(timeout_ms)).await;
retval
}
fn interval(timeout_ms: u64) -> impl futures::Stream<Item = ()> {
futures::stream::unfold((), move |_| async move { delay(timeout_ms, ()).await; Some(((), ())) })
}
fn process_future_result<TClient, TResult, TError, TGoOfflineFuture>(
maybe_client: &mut Option<TClient>,
client: TClient,
result: Result<TResult, TError>,
on_success: impl FnOnce(TResult),
go_offline_future: &mut std::pin::Pin<&mut futures::future::Fuse<TGoOfflineFuture>>,
go_offline: impl FnOnce(TClient) -> TGoOfflineFuture,
error_pattern: &'static str,
) where
TError: std::fmt::Debug + MaybeConnectionError,
TGoOfflineFuture: FutureExt,
{
match result {
Ok(result) => {
*maybe_client = Some(client);
on_success(result);
},
Err(error) => {
if error.is_connection_error() {
go_offline_future.set(go_offline(client).fuse());
} else {
*maybe_client = Some(client);
}
log::error!(target: "bridge", "{}: {:?}", error_pattern, error);
},
}
}
bridges/relays/ethereum/src/ethereum_types.rs
+110
View File
@@ -0,0 +1,110 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
pub use web3::types::{Bytes, H256, U128, U64};
/// When header is just received from the Ethereum node, we check that it has
/// both number and hash fields filled.
pub const HEADER_ID_PROOF: &'static str = "checked on retrieval; qed";
/// When receipt is just received from the Ethereum node, we check that it has
/// gas_used field filled.
pub const RECEIPT_GAS_USED_PROOF: &'static str = "checked on retrieval; qed";
/// Ethereum header type.
pub type Header = web3::types::Block<H256>;
/// Ethereum transaction receipt type.
pub type Receipt = web3::types::TransactionReceipt;
/// Ethereum header Id.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct HeaderId(pub u64, pub H256);
impl From<&Header> for HeaderId {
fn from(header: &Header) -> HeaderId {
HeaderId(header.number.expect(HEADER_ID_PROOF).as_u64(), header.hash.expect(HEADER_ID_PROOF))
}
}
/// Ethereum header synchronization status.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum HeaderStatus {
/// Header is unknown.
Unknown,
/// Header is in MaybeOrphan queue.
MaybeOrphan,
/// Header is in Orphan queue.
Orphan,
/// Header is in MaybeReceipts queue.
MaybeReceipts,
/// Header is in Receipts queue.
Receipts,
/// Header is in Ready queue.
Ready,
/// Header has been recently submitted to the Substrate runtime.
Submitted,
/// Header is known to the Substrate runtime.
Synced,
}
#[derive(Clone, Debug, Default)]
#[cfg_attr(test, derive(PartialEq))]
pub struct QueuedHeader {
header: Header,
receipts: Option<Vec<Receipt>>,
}
impl QueuedHeader {
/// Creates new queued header.
pub fn new(header: Header) -> Self {
QueuedHeader { header, receipts: None }
}
/// Returns ID of header.
pub fn id(&self) -> HeaderId {
(&self.header).into()
}
/// Returns ID of parent header.
pub fn parent_id(&self) -> HeaderId {
HeaderId(
self.header.number.expect(HEADER_ID_PROOF).as_u64() - 1,
self.header.parent_hash,
)
}
/// Returns reference to header.
pub fn header(&self) -> &Header {
&self.header
}
/// Returns reference to transactions receipts.
pub fn receipts(&self) -> &Option<Vec<Receipt>> {
&self.receipts
}
/// Extract header and receipts from self.
pub fn extract(self) -> (Header, Option<Vec<Receipt>>) {
(self.header, self.receipts)
}
/// Set associated transaction receipts.
pub fn set_receipts(mut self, receipts: Vec<Receipt>) -> Self {
self.receipts = Some(receipts);
self
}
}
bridges/relays/ethereum/src/main.rs
+102
View File
@@ -0,0 +1,102 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
#![recursion_limit="1024"]
mod ethereum_client;
mod ethereum_headers;
mod ethereum_sync;
mod ethereum_sync_loop;
mod ethereum_types;
mod substrate_client;
mod substrate_types;
use std::io::Write;
use sp_core::crypto::Pair;
fn main() {
initialize();
ethereum_sync_loop::run(match ethereum_sync_params() {
Ok(ethereum_sync_params) => ethereum_sync_params,
Err(err) => {
log::error!(target: "bridge", "Error parsing parameters: {}", err);
return;
},
});
}
fn initialize() {
let mut builder = env_logger::Builder::new();
let filters = match std::env::var("RUST_LOG") {
Ok(env_filters) => format!("bridge=info,{}", env_filters),
Err(_) => "bridge=info".into(),
};
builder.parse_filters(&filters);
builder.format(move |buf, record| {
writeln!(buf, "{}", {
let timestamp = time::strftime("%Y-%m-%d %H:%M:%S %Z", &time::now())
.expect("Time is incorrectly formatted");
if cfg!(windows) {
format!("{} {} {} {}", timestamp, record.level(), record.target(), record.args())
} else {
use ansi_term::Colour as Color;
let log_level = match record.level() {
log::Level::Error => Color::Fixed(9).bold().paint(record.level().to_string()),
log::Level::Warn => Color::Fixed(11).bold().paint(record.level().to_string()),
log::Level::Info => Color::Fixed(10).paint(record.level().to_string()),
log::Level::Debug => Color::Fixed(14).paint(record.level().to_string()),
log::Level::Trace => Color::Fixed(12).paint(record.level().to_string()),
};
format!("{} {} {} {}"
, Color::Fixed(8).bold().paint(timestamp)
, log_level
, Color::Fixed(8).paint(record.target())
, record.args())
}
})
});
builder.init();
}
fn ethereum_sync_params() -> Result<ethereum_sync_loop::EthereumSyncParams, String> {
let yaml = clap::load_yaml!("cli.yml");
let matches = clap::App::from_yaml(yaml).get_matches();
let mut eth_sync_params = ethereum_sync_loop::EthereumSyncParams::default();
if let Some(eth_host) = matches.value_of("eth-host") {
eth_sync_params.eth_host = eth_host.into();
}
if let Some(eth_port) = matches.value_of("eth-port") {
eth_sync_params.eth_port = eth_port.parse().map_err(|e| format!("{}", e))?;
}
if let Some(sub_host) = matches.value_of("sub-host") {
eth_sync_params.sub_host = sub_host.into();
}
if let Some(sub_port) = matches.value_of("sub-port") {
eth_sync_params.sub_port = sub_port.parse().map_err(|e| format!("{}", e))?;
}
if let Some(sub_signer) = matches.value_of("sub-signer") {
let sub_signer_password = matches.value_of("sub-signer-password");
eth_sync_params.sub_signer = sp_core::sr25519::Pair::from_string(sub_signer, sub_signer_password)
.map_err(|e| format!("{:?}", e))?;
}
Ok(eth_sync_params)
}
bridges/relays/ethereum/src/substrate_client.rs
+320
View File
@@ -0,0 +1,320 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
use codec::{Encode, Decode};
use jsonrpsee_core::{client::ClientError, common::Params};
use jsonrpsee_http::{HttpClient, RequestError, http_client};
use serde_json::{from_value, to_value};
use sp_core::crypto::Pair;
use sp_runtime::traits::IdentifyAccount;
use crate::ethereum_sync_loop::MaybeConnectionError;
use crate::ethereum_types::{
Bytes,
H256,
HeaderId as EthereumHeaderId,
QueuedHeader as QueuedEthereumHeader,
};
use crate::substrate_types::{
TransactionHash,
into_substrate_ethereum_header,
into_substrate_ethereum_receipts,
};
/// Substrate client type.
pub struct Client {
/// Substrate RPC client.
rpc_client: HttpClient,
/// Transactions signer.
signer: sp_core::sr25519::Pair,
/// Genesis block hash.
genesis_hash: Option<H256>,
}
/// All possible errors that can occur during interacting with Ethereum node.
#[derive(Debug)]
pub enum Error {
/// Request start failed.
StartRequestFailed(RequestError),
/// Request not found (should never occur?).
RequestNotFound,
/// Failed to receive response.
ResponseRetrievalFailed(ClientError<RequestError>),
/// Failed to parse response.
ResponseParseFailed,
}
impl MaybeConnectionError for Error {
fn is_connection_error(&self) -> bool {
match *self {
Error::StartRequestFailed(_) | Error::ResponseRetrievalFailed(_) => true,
_ => false,
}
}
}
/// Returns client that is able to call RPCs on Substrate node.
pub fn client(uri: &str, signer: sp_core::sr25519::Pair) -> Client {
Client {
rpc_client: http_client(uri),
signer,
genesis_hash: None,
}
}
/// Returns best Ethereum block that Substrate runtime knows of.
pub async fn best_ethereum_block(client: Client) -> (Client, Result<EthereumHeaderId, Error>) {
let (client, result) = call_rpc::<(u64, H256)>(
client,
"state_call",
Params::Array(vec![
to_value("EthereumHeadersApi_best_block").unwrap(),
to_value("0x").unwrap(),
]),
).await;
(client, result.map(|(num, hash)| EthereumHeaderId(num, hash)))
}
/// Returns true if transactions receipts are required for Ethereum header submission.
pub async fn ethereum_receipts_required(
client: Client,
header: QueuedEthereumHeader,
) -> (Client, Result<(EthereumHeaderId, bool), Error>) {
let id = header.id();
let header = into_substrate_ethereum_header(header.header());
let encoded_header = header.encode();
let (client, receipts_required) = call_rpc(
client,
"state_call",
Params::Array(vec![
to_value("EthereumHeadersApi_is_import_requires_receipts").unwrap(),
to_value(Bytes(encoded_header)).unwrap(),
]),
).await;
(client, receipts_required.map(|receipts_required| (id, receipts_required)))
}
/// Returns true if Ethereum header is known to Substrate runtime.
pub async fn ethereum_header_known(
client: Client,
id: EthereumHeaderId,
) -> (Client, Result<(EthereumHeaderId, bool), Error>) {
// Substrate module could prune old headers. So this fn could return false even
// if header is synced. And we'll mark corresponding Ethereum header as Orphan.
//
// But when we'll read best header from Substrate next time, we will know that
// there's a better header => this Orphan will either be marked as synced, or
// eventually pruned.
let encoded_id = id.1.encode();
let (client, is_known_block) = call_rpc(
client,
"state_call",
Params::Array(vec![
to_value("EthereumHeadersApi_is_known_block").unwrap(),
to_value(Bytes(encoded_id)).unwrap(),
]),
).await;
(client, is_known_block.map(|is_known_block| (id, is_known_block)))
}
/// Submits Ethereum header to Substrate runtime.
pub async fn submit_ethereum_headers(
client: Client,
headers: Vec<QueuedEthereumHeader>,
) -> (Client, Result<(TransactionHash, Vec<EthereumHeaderId>), Error>) {
let ids = headers.iter().map(|header| header.id()).collect();
let (client, genesis_hash) = match client.genesis_hash {
Some(genesis_hash) => (client, genesis_hash),
None => {
let (mut client, genesis_hash) = block_hash_by_number(client, 0).await;
let genesis_hash = match genesis_hash {
Ok(genesis_hash) => genesis_hash,
Err(err) => return (client, Err(err)),
};
client.genesis_hash = Some(genesis_hash);
(client, genesis_hash)
},
};
let account_id = client.signer.public().as_array_ref().clone().into();
let (client, nonce) = next_account_index(client, account_id).await;
let nonce = match nonce {
Ok(nonce) => nonce,
Err(err) => return (client, Err(err)),
};
let transaction = create_submit_transaction(
headers,
&client.signer,
nonce,
genesis_hash,
);
let encoded_transaction = transaction.encode();
let (client, transaction_hash) = call_rpc(
client,
"author_submitExtrinsic",
Params::Array(vec![
to_value(Bytes(encoded_transaction)).unwrap(),
]),
).await;
(client, transaction_hash.map(|transaction_hash| (transaction_hash, ids)))
}
/// Get Substrate block hash by its number.
async fn block_hash_by_number(client: Client, number: u64) -> (Client, Result<H256, Error>) {
call_rpc(
client,
"chain_getBlockHash",
Params::Array(vec![
to_value(number).unwrap(),
]),
).await
}
/// Get substrate account nonce.
async fn next_account_index(
client: Client,
account: node_primitives::AccountId,
) -> (Client, Result<node_primitives::Index, Error>) {
use sp_core::crypto::Ss58Codec;
let (client, index) = call_rpc_u64(
client,
"system_accountNextIndex",
Params::Array(vec![
to_value(account.to_ss58check()).unwrap(),
]),
).await;
(client, index.map(|index| index as _))
}
/// Calls RPC on Substrate node that returns Bytes.
async fn call_rpc<T: Decode>(
mut client: Client,
method: &'static str,
params: Params,
) -> (Client, Result<T, Error>) {
async fn do_call_rpc<T: Decode>(
client: &mut Client,
method: &'static str,
params: Params,
) -> Result<T, Error> {
let request_id = client
.rpc_client
.start_request(method, params)
.await
.map_err(Error::StartRequestFailed)?;
// WARN: if there'll be need for executing >1 request at a time, we should avoid
// calling request_by_id
let response = client
.rpc_client
.request_by_id(request_id)
.ok_or(Error::RequestNotFound)?
.await
.map_err(Error::ResponseRetrievalFailed)?;
let encoded_response: Bytes = from_value(response).map_err(|_| Error::ResponseParseFailed)?;
Decode::decode(&mut &encoded_response.0[..]).map_err(|_| Error::ResponseParseFailed)
}
let result = do_call_rpc(&mut client, method, params).await;
(client, result)
}
/// Calls RPC on Substrate node that returns u64.
async fn call_rpc_u64(
mut client: Client,
method: &'static str,
params: Params,
) -> (Client, Result<u64, Error>) {
async fn do_call_rpc(
client: &mut Client,
method: &'static str,
params: Params,
) -> Result<u64, Error> {
let request_id = client
.rpc_client
.start_request(method, params)
.await
.map_err(Error::StartRequestFailed)?;
// WARN: if there'll be need for executing >1 request at a time, we should avoid
// calling request_by_id
let response = client
.rpc_client
.request_by_id(request_id)
.ok_or(Error::RequestNotFound)?
.await
.map_err(Error::ResponseRetrievalFailed)?;
response.as_u64().ok_or(Error::ResponseParseFailed)
}
let result = do_call_rpc(&mut client, method, params).await;
(client, result)
}
/// Create Substrate transaction for submitting Ethereum header.
fn create_submit_transaction(
headers: Vec<QueuedEthereumHeader>,
signer: &sp_core::sr25519::Pair,
index: node_primitives::Index,
genesis_hash: H256,
) -> node_runtime::UncheckedExtrinsic {
let function = node_runtime::Call::BridgeEthPoa(
node_runtime::BridgeEthPoaCall::import_headers(
headers
.into_iter()
.map(|header| {
let (header, receipts) = header.extract();
(
into_substrate_ethereum_header(&header),
into_substrate_ethereum_receipts(&receipts),
)
})
.collect(),
),
);
let extra = |i: node_primitives::Index, f: node_primitives::Balance| {
(
frame_system::CheckVersion::<node_runtime::Runtime>::new(),
frame_system::CheckGenesis::<node_runtime::Runtime>::new(),
frame_system::CheckEra::<node_runtime::Runtime>::from(sp_runtime::generic::Era::Immortal),
frame_system::CheckNonce::<node_runtime::Runtime>::from(i),
frame_system::CheckWeight::<node_runtime::Runtime>::new(),
pallet_transaction_payment::ChargeTransactionPayment::<node_runtime::Runtime>::from(f),
Default::default(),
)
};
let raw_payload = node_runtime::SignedPayload::from_raw(
function,
extra(index, 0),
(
198, // VERSION.spec_version as u32,
genesis_hash,
genesis_hash,
(),
(),
(),
(),
),
);
let signature = raw_payload.using_encoded(|payload| signer.sign(payload));
let signer: sp_runtime::MultiSigner = signer.public().into();
let (function, extra, _) = raw_payload.deconstruct();
node_runtime::UncheckedExtrinsic::new_signed(
function,
signer.into_account().into(),
signature.into(),
extra,
)
}
bridges/relays/ethereum/src/substrate_types.rs
+68
View File
@@ -0,0 +1,68 @@
// Copyright 2019 Parity Technologies (UK) Ltd.
// This file is part of Parity-Bridge.
// Parity-Bridge is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Parity-Bridge is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Parity-Bridge. If not, see <http://www.gnu.org/licenses/>.
pub use sp_bridge_eth_poa::{
Address, Bloom, Bytes, H256, Header as SubstrateEthereumHeader,
LogEntry as SubstrateEthereumLogEntry, Receipt as SubstrateEthereumReceipt,
TransactionOutcome as SubstrateEthereumTransactionOutcome, U256,
};
pub use crate::ethereum_types::H256 as TransactionHash;
use crate::ethereum_types::{
HEADER_ID_PROOF as ETHEREUM_HEADER_ID_PROOF,
RECEIPT_GAS_USED_PROOF as ETHEREUM_RECEIPT_GAS_USED_PROOF,
Header as EthereumHeader,
Receipt as EthereumReceipt,
};
/// Convert Ethereum header into Ethereum header for Substrate.
pub fn into_substrate_ethereum_header(header: &EthereumHeader) -> SubstrateEthereumHeader {
SubstrateEthereumHeader {
parent_hash: header.parent_hash,
timestamp: header.timestamp.as_u64(),
number: header.number.expect(ETHEREUM_HEADER_ID_PROOF).as_u64(),
author: header.author,
transactions_root: header.transactions_root,
uncles_hash: header.uncles_hash,
extra_data: header.extra_data.0.clone(),
state_root: header.state_root,
receipts_root: header.receipts_root,
log_bloom: header.logs_bloom.data().into(),
gas_used: header.gas_used,
gas_limit: header.gas_limit,
difficulty: header.difficulty,
seal: header.seal_fields.iter().map(|s| s.0.clone()).collect(),
}
}
/// Convert Ethereum transactions receipts into Ethereum transactions receipts for Substrate.
pub fn into_substrate_ethereum_receipts(
receipts: &Option<Vec<EthereumReceipt>>,
) -> Option<Vec<SubstrateEthereumReceipt>> {
receipts.as_ref().map(|receipts| receipts.iter().map(|receipt| SubstrateEthereumReceipt {
gas_used: receipt.gas_used.expect(ETHEREUM_RECEIPT_GAS_USED_PROOF),
log_bloom: receipt.logs_bloom.data().into(),
logs: receipt.logs.iter().map(|log_entry| SubstrateEthereumLogEntry {
address: log_entry.address,
topics: log_entry.topics.clone(),
data: log_entry.data.0.clone(),
}).collect(),
outcome: match (receipt.status, receipt.root) {
(Some(status), None) => SubstrateEthereumTransactionOutcome::StatusCode(status.as_u64() as u8),
(None, Some(root)) => SubstrateEthereumTransactionOutcome::StateRoot(root),
_ => SubstrateEthereumTransactionOutcome::Unknown,
},
}).collect())
}