* Remove signature verification in backing.
`SignedFullStatement` now signals that the signature has already been
checked.
* Remove unused check_payload function.
* Introduced unchecked signed variants.
* Fix inclusion to use unchecked variant.
* More unchecked variants.
* Use unchecked variants in protocols.
* Start fixing statement-distribution.
* Fixup statement distribution.
* Fix inclusion.
* Fix warning.
* Fix backing properly.
* Fix bitfield distribution.
* Make crypto store optional for `RuntimeInfo`.
* Factor out utility functions.
* get_group_rotation_info
* WIP: Collator cleanup + check signatures.
* Convenience signature checking functions.
* Check signature on collator-side.
* Fix warnings.
* Fix collator side tests.
* Get rid of warnings.
* Better Signed/UncheckedSigned implementation.
Also get rid of Encode/Decode for Signed! *party*
* Get rid of dead code.
* Move Signed in its own module.
* into_checked -> try_into_checked
* Fix merge.
* Factor out runtime module into utils.
* First fatal error design.
* Better error handling infra.
* Error handling cleanup.
* Send to peers of our group first.
* Finish backing group prioritization.
* Little cleanup.
* More cleanup.
* Forgot to checkin error.rs.
* Notes.
* Runtime -> RuntimeInfo
* qed in debug assert.
* PolkaErr -> Fault.
* Simplify some Option / Result / ? operator patterns
When they identically match a combinator on those types.
Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
* adjust review comments
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Factor out runtime module into utils.
* Add maybe_authority information to `PeerConnected` event.
We already gather this information in authority discovery, so we might
as well share it with others.
This opens up an easy path to trigger validators differently from normal
nodes, e.g. for prioritization. This change has become more important
now, that we just connect to all validators and therefore just have a
long peer list without any information about those nodes.
* Test fix.
* More test coverage.
* Preserve peer order.
* Better test coverage.
* Even more test coverage.
* Add doc comment to `IndexMap`.
* Fix flaky test.
* Review remarks.
* Review remarks.
* Wip
* Increase proposer timeout.
* WIP.
* Better timeout values now that we are going to be connected to all nodes. (#2778)
* Better timeout values.
* Fix typo.
* Fix validator bandwidth.
* Fix compilation.
* Better and more consistent sizes.
Most importantly code size is now 5 Meg, which is the limit we currently
want to support in statement distribution.
* Introduce statement fetching request.
* WIP
* Statement cache retrieval logic.
* Review remarks by @rphmeier
* Fixes.
* Better requester logic.
* WIP: Handle requester messages.
* Missing dep.
* Fix request launching logic.
* Finish fetching logic.
* Sending logic.
* Redo code size calculations.
Now that max code size is compressed size.
* Update Cargo.lock (new dep)
* Get request receiver to statement distribution.
* Expose new functionality for responding to requests.
* Cleanup.
* Responder logic.
* Fixes + Cleanup.
* Cargo.lock
* Whitespace.
* Add lost copyright.
* Launch responder task.
* Typo.
* info -> warn
* Typo.
* Fix.
* Fix.
* Update comment.
* Doc fix.
* Better large statement heuristics.
* Fix tests.
* Fix network bridge tests.
* Add test for size estimate.
* Very simple tests that checks we get LargeStatement.
* Basic check, that fetching of large candidates is performed.
* More tests.
* Basic metrics for responder.
* More metrics.
* Use Encode::encoded_size().
* Some useful spans.
* Get rid of redundant metrics.
* Don't add peer on duplicate.
* Properly check hash
instead of relying on signatures alone.
* Preserve ordering + better flood protection.
* Get rid of redundant clone.
* Don't shutdown responder on failed query.
And add test for this.
* Smaller fixes.
* Quotes.
* Better queue size calculation.
* A bit saner response sizes.
* Fixes.
* use compressed blob in candidate-validation
* add some tests for compressed code blobs
* remove CompressedPoV and apply compression in collation-generation
* decompress BlockData before executing
* don't produce oversized collations
* add test for PoV decompression failure
* fix tests and clean up
* fix test
* address review and fix CI
* take this )
* tests/av-store: use future::join instead of future::select
* tests/backing: use future::join instead of future::select
* tests/provisioner: use future::join instead of future::select
* tests/av-dist: use future::join instead of future::select
* tests/av-recovery: use future::join instead of future::select
* tests/bridge: use future::join instead of future::select
* tests/collator-protocol: use future::join instead of future::select
* tests/stmt-dist: use future::join instead of future::select
* fix tests
* guide: declare one para as a collator
* add ParaId to Declare messages and clean up
* fix build
* fix the testerinos
* begin adding keystore to collator-protocol
* remove request_x_ctx
* add core_for_group
* add bump_rotation
* add some more helpers to subsystem-util
* change signing_key API to take ref
* determine current and next para assignments
* disconnect collators who are not on current or next para
* add collator peer count metric
* notes for later
* some fixes
* add data & keystore to test state
* add a test utility for answering runtime API requests
* fix existing collator tests
* add new tests
* remove sc_keystore
* update cargo lock
Co-authored-by: Andronik Ordian <write@reusable.software>
* extract database from av-store itself
* generalize approval-voting over database type
* modes (without handling) and pruning old wakeups
* rework approval importing
* add our_approval_sig to ApprovalEntry
* import assignment
* guide updates for check-full-approval changes
* some aux functions
* send messages when becoming active.
* guide: network bridge sends view updates only when done syncing
* network bridge: send view updates only when done syncing
* tests for new network-bridge behavior
* add a test for updating approval entry with sig
* fix some warnings
* test load-all-blocks
* instantiate new parachains DB
* fix network-bridge empty view updates
* tweak
* fix wasm build, i think
* Update node/core/approval-voting/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* add some versioning to parachains_db
* warnings
* fix merge changes
* remove versioning again
Co-authored-by: Andronik Ordian <write@reusable.software>
* gossip: do not issue a connection request if we are not a validator
* guide updates
* use all relevant authorities when issuing a request
* use AuthorityDiscoveryApi instead
* update comments to the status quo
* guide: don't request availability data from ourselves
* add QueryAllChunks message
* implement QueryAllChunks
* remove unused relay_parent from StoreChunk
* test QueryAllChunks
* fast paths make short roads
* test early exit behavior
* guide updates
* keep interactions alive until receivers drop
* retry indefinitely
* cancel approval tasks on finality
* use swap_remove instead of remove
* availability-distribution: Retry on fail on next block.
Retry failed fetches on next block when still pending availability.
* Update node/network/availability-distribution/src/requester/fetch_task/mod.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Fix existing tests.
* Add test for trying all validators.
* Add test for testing retries.
Co-authored-by: Andronik Ordian <write@reusable.software>
* Remove stuff out of the runtime that does not belong there.
There might be more, but it is a start.
* White space fixes.
* Fix tests.
* Leave whitespace in ui tests alone.
* Add back zstd for no reason.
* Fix browser wasm (hopefully)
* overseer: pass messages directly between subsystems
* test that message is held on to
* Update node/overseer/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* give every subsystem an unbounded sender too
* remove metered_channel::name
1. we don't provide good names
2. these names are never used anywhere
* unused mut
* remove unnecessary &mut
* subsystem unbounded_send
* remove unused MaybeTimer
We have channel size metrics that serve the same purpose better now and the implementation of message timing was pretty ugly.
* remove comment
* split up senders and receivers
* update metrics
* fix tests
* fix test subsystem context
* use SubsystemSender in jobs system now
* refactor of awful jobs code
* expose public `run` on JobSubsystem
* update candidate backing to new jobs & use unbounded
* bitfield signing
* candidate-selection
* provisioner
* approval voting: send unbounded for assignment/approvals
* async not needed
* begin bridge split
* split up network tasks into background worker
* port over network bridge
* Update node/network/bridge/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* rename ValidationWorkerNotifications
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
* approval-distribution: limit the amount of packets on unify
* guide: fix a typo
* compilation fix
* grammar
* Update roadmap/implementers-guide/src/node/approval/approval-distribution.md
Co-authored-by: David <dvdplm@gmail.com>
* more grammar
* propagate only local assignments/approvals after a certain depth
* increase the threshold
* guides update
Co-authored-by: David <dvdplm@gmail.com>
Andronik Ordian