Bastian Köcher
d0c97539e4
* Fix bug and further optimizations in availability distribution - There was a bug that resulted in only getting one candidate per block as the candidates were put into the hashmap with the relay block hash as key. The solution for this is to use the candidate hash and the relay block hash as key. - We stored received/sent messages with the candidate hash and chunk index as key. The candidate hash wasn't required in this case, as the messages are already stored per candidate. * Update node/core/bitfield-signing/src/lib.rs Co-authored-by: Robert Habermeier <rphmeier@gmail.com> * Remove the reverse map * major refactor of receipts & query_live * finish refactoring remove ancestory mapping, improve relay-parent cleanup & receipts-cache cleanup, add descriptor to `PerCandidate` * rename and rewrite query_pending_availability * add a bunch of consistency tests * Add some last changes * xy * fz * Make it compile again * Fix one test * Fix logging * Remove some buggy code * Make tests work again * Move stuff around * Remove dbg * Remove state from test_harness * More refactor and new test * New test and fixes * Move metric * Remove "duplicated code" * Fix tests * New test * Change break to continue * Update node/core/av-store/src/lib.rs * Update node/core/av-store/src/lib.rs * Update node/core/bitfield-signing/src/lib.rs Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com> * update guide to match live_candidates changes * add comment * fix bitfield signing Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
2.7 KiB
Availability Distribution
Distribute availability erasure-coded chunks to validators.
After a candidate is backed, the availability of the PoV block must be confirmed by 2/3+ of all validators. Validating a candidate successfully and contributing it to being backable leads to the PoV and erasure-coding being stored in the Availability Store.
Protocol
PeerSet: Validation
Input:
- NetworkBridgeUpdateV1(update)
Output:
- NetworkBridge::SendValidationMessage(
[PeerId], message) - NetworkBridge::ReportPeer(PeerId, cost_or_benefit)
- AvailabilityStore::QueryPoV(candidate_hash, response_channel)
- AvailabilityStore::StoreChunk(candidate_hash, chunk_index, inclusion_proof, chunk_data)
Functionality
For each relay-parent in our local view update, look at all backed candidates pending availability. Distribute via gossip all erasure chunks for all candidates that we have to peers.
We define an operation live_candidates(relay_heads) -> Set<CandidateHash> which returns a set of hashes corresponding to CandidateReceipts.
This is defined as all candidates pending availability in any of those relay-chain heads or any of their last K ancestors in the same session. We assume that state is not pruned within K blocks of the chain-head. K commonly is small and is currently fixed to K=3.
We will send any erasure-chunks that correspond to candidates in live_candidates(peer_most_recent_view_update).
Likewise, we only accept and forward messages pertaining to a candidate in live_candidates(current_heads).
Each erasure chunk should be accompanied by a merkle proof that it is committed to by the erasure trie root in the candidate receipt, and this gossip system is responsible for checking such proof.
We re-attempt to send anything live to a peer upon any view update from that peer.
On our view change, for all live candidates, we will check if we have the PoV by issuing a QueryAvailabileData message and waiting for the response. If the query returns Some, we will perform the erasure-coding and distribute all messages to peers that will accept them.
If we are operating as a validator, we note our index i in the validator set and keep the ith availability chunk for any live candidate, as we receive it. We keep the chunk and its merkle proof in the Availability Store by sending a StoreChunk command. This includes chunks and proofs generated as the result of a successful QueryPoV.
The back-and-forth seems suboptimal at first glance, but drastically simplifies the pruning in the availability store, as it creates an invariant that chunks are only stored if the candidate was actually backed.