Batch vote import in dispute-distribution (#5894)

* Start work on batching in dispute-distribution.

* Guide work.

* More guide changes. Still very much WIP.

* Finish guide changes.

* Clarification

* Adjust argument about slashing.

* WIP: Add constants to receiver.

* Maintain order of disputes.

* dispute-distribuion sender Rate limit.

* Cleanup

* WIP: dispute-distribution receiver.

- [ ] Rate limiting
- [ ] Batching

* WIP: Batching.

* fmt

* Update `PeerQueues` to maintain more invariants.

* WIP: Batching.

* Small cleanup

* Batching logic.

* Some integration work.

* Finish.

Missing: Tests

* Typo.

* Docs.

* Report missing metric.

* Doc pass.

* Tests for waiting_queue.

* Speed up some crypto by 10x.

* Fix redundant import.

* Add some tracing.

* Better sender rate limit

* Some tests.

* Tests

* Add logging to rate limiter

* Update roadmap/implementers-guide/src/node/disputes/dispute-distribution.md

Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>

* Update roadmap/implementers-guide/src/node/disputes/dispute-distribution.md

Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>

* Update node/network/dispute-distribution/src/receiver/mod.rs

Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>

* Review feedback.

* Also log peer in log messages.

* Fix indentation.

* waker -> timer

* Guide improvement.

* Remove obsolete comment.

* waker -> timer

* Fix spell complaints.

* Fix Cargo.lock

Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
This commit is contained in:
Robert Klotzner
2022-10-04 18:02:05 +02:00
committed by GitHub
parent a64cc4a860
commit 938bc96a2c
17 changed files with 1772 additions and 458 deletions
@@ -19,8 +19,10 @@
use fatality::Nested;
use gum::CandidateHash;
use polkadot_node_network_protocol::{request_response::incoming, PeerId};
use polkadot_node_subsystem_util::runtime;
use polkadot_primitives::v2::AuthorityDiscoveryId;
use crate::LOG_TARGET;
@@ -35,8 +37,8 @@ pub enum Error {
#[error("Retrieving next incoming request failed.")]
IncomingRequest(#[from] incoming::Error),
#[error("Sending back response to peer {0} failed.")]
SendResponse(PeerId),
#[error("Sending back response to peers {0:#?} failed.")]
SendResponses(Vec<PeerId>),
#[error("Changing peer's ({0}) reputation failed.")]
SetPeerReputation(PeerId),
@@ -44,16 +46,29 @@ pub enum Error {
#[error("Dispute request with invalid signatures, from peer {0}.")]
InvalidSignature(PeerId),
#[error("Import of dispute got canceled for peer {0} - import failed for some reason.")]
ImportCanceled(PeerId),
#[error("Received votes from peer {0} have been completely redundant.")]
RedundantMessage(PeerId),
#[error("Import of dispute got canceled for candidate {0} - import failed for some reason.")]
ImportCanceled(CandidateHash),
#[error("Peer {0} attempted to participate in dispute and is not a validator.")]
NotAValidator(PeerId),
#[error("Force flush for batch that could not be found attempted, candidate hash: {0}")]
ForceFlushBatchDoesNotExist(CandidateHash),
// Should never happen in practice:
#[error("We needed to drop messages, because we reached limit on concurrent batches.")]
MaxBatchLimitReached,
#[error("Authority {0} sent messages at a too high rate.")]
AuthorityFlooding(AuthorityDiscoveryId),
}
pub type Result<T> = std::result::Result<T, Error>;
pub type JfyiErrorResult<T> = std::result::Result<T, JfyiError>;
pub type JfyiResult<T> = std::result::Result<T, JfyiError>;
/// Utility for eating top level errors and log them.
///