Erasure encoding availability (#345)

* Erasure encoding availability initial commit

 * Modifications to availability store to keep chunks as well as
   reconstructed blocks and extrinsics.
 * Gossip messages containig signed erasure chunks.
 * Requesting eraure chunks with polkadot-specific messages.
 * Validation of erasure chunk messages.

* Apply suggestions from code review

Co-Authored-By: Luke Schoen <ltfschoen@users.noreply.github.com>

* Fix build after a merge

* Gossip erasure chunk messages under their own topic

* erasure_chunks should use the appropriate topic

* Updates Cargo.lock

* Fixes after merge

* Removes a couple of leftover pieces of code

* Fixes simple stuff from review

* Updates erasure and storage for more flexible logic

* Changes validation and candidate receipt production.

* Adds add_erasure_chunks method

* Fixes most of the nits

* Better validate_collation and validate_receipt functions

* Fixes the tests

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Removes unwrap() calls

* Removes ErasureChunks primitive

* Removes redundant fields from ErasureChunk struct

* AvailabilityStore should store CandidateReceipt

* Changes the way chunk messages are imported and validated.

 * Availability store now stores a validator_index and n_validators for
 each relay_parent.
 * Availability store now also stores candidate receipts.
 * Removes importing chunks in the table and moves it into network
 gossip validation.
 * Validation of erasure messages id done against receipts that are
 stored in the availability store.

* Correctly compute topics for erasure messages

* Removes an unused parameter

* Refactors availability db querying into a helper

* Adds the apis described in the writeup

* Adds a runtime api to extract erasure roots form raw extrinsics.

* Adds a barebone BlockImport impl for avalability store

* Adds the implementation of the availability worker

* Fix build after the merge with master.

* Make availability store API async

* Bring back the default wasmtime feature

* Lines width

* Bump runtime version

* Formatting and dead code elimination

* some style nits (#1)

* More nits and api cleanup

* Disable wasm CI for availability-store

* Another nit

* Formatting
This commit is contained in:
Fedor Sakharov
2019-12-03 17:49:07 +03:00
committed by Robert Habermeier
parent ec54d5b1e4
commit 99d164b5e7
29 changed files with 2957 additions and 572 deletions
+7 -8
View File
@@ -53,6 +53,7 @@ use futures::{
future, Future, Stream, FutureExt, TryFutureExt, StreamExt,
compat::{Compat01As03, Future01CompatExt, Stream01CompatExt}
};
use futures01::{Future as _};
use log::{warn, error};
use client::BlockchainEvents;
use primitives::{Pair, Blake2Hasher};
@@ -216,19 +217,18 @@ pub async fn collate<R, P>(
let egress_queue_roots =
polkadot_validation::egress_roots(&mut outgoing.outgoing_messages);
let receipt = parachain::CandidateReceipt {
let info = parachain::CollationInfo {
parachain_index: local_id,
collator: key.public(),
signature,
head_data,
egress_queue_roots,
fees: 0,
head_data,
block_data_hash,
upward_messages: Vec::new(),
};
let collation = parachain::Collation {
receipt,
info,
pov: PoVBlock {
block_data,
ingress,
@@ -393,6 +393,7 @@ impl<P, E> Worker for CollationNode<P, E> where
let key = key.clone();
let parachain_context = parachain_context.clone();
let validation_network = validation_network.clone();
let inner_exit_2 = inner_exit.clone();
let work = future::lazy(move |_| {
let api = client.runtime_api();
@@ -432,9 +433,7 @@ impl<P, E> Worker for CollationNode<P, E> where
outgoing,
);
if let Err(e) = res {
warn!("Unable to broadcast local collation: {:?}", e);
}
tokio::spawn(res.select(inner_exit_2.clone()).then(|_| Ok(())));
})
});
@@ -602,7 +601,7 @@ mod tests {
let collation = futures::executor::block_on(future).unwrap().0;
// ascending order by root.
assert_eq!(collation.receipt.egress_queue_roots, vec![(a, root_a), (b, root_b)]);
assert_eq!(collation.info.egress_queue_roots, vec![(a, root_a), (b, root_b)]);
}
}