* Move NewGossipTopology -> SessionGridTopology outside as this implementation is shared
* Add method to return peers difference between topologies
* Implement basic grid topology usage for the bitfield distribution
* Fix tests
* Oops, fix tests
* Add some tests for random routing
* Add a unit test for topology distribution
* Store the current and the previous topology to match sessions boundaries
* Update tests
* Update node/network/bitfield-distribution/src/lib.rs
Co-authored-by: Andronik <write@reusable.software>
* Update node/network/protocol/src/grid_topology.rs
Co-authored-by: Andronik <write@reusable.software>
* Update node/network/bitfield-distribution/src/lib.rs
Co-authored-by: Andronik <write@reusable.software>
* Add some debug
* Fix tests as HashSet order is undefined
Co-authored-by: Andronik <write@reusable.software>
* explicitly tag network requests with version
* fmt
* make PeerSet more aware of versioning
* some generalization of the network bridge to support upgrades
* walk back some renaming
* walk back some version stuff
* extract version from fallback
* remove V1 from NetworkBridgeUpdate
* add accidentally-removed timer
* implement focusing for versioned messages
* fmt
* fix up network bridge & tests
* remove inaccurate version check in bridge
* remove some TODO [now]s
* fix fallout in statement distribution
* fmt
* fallout in gossip-support
* fix fallout in collator-protocol
* fix fallout in bitfield-distribution
* fix fallout in approval-distribution
* fmt
* use never!
* fmt
* gossip-support: be explicit about dimensions
* some guide updates
* update network-bridge to distinguish x and y dimensions
* get everything to compile
* beginnings
* some TODOs
* polkadot runtime: use relevant_authorities
* make gossip topologies per-session
* better formatting
* gossip support: use current session validators
* expand in comment
* adjust tests and fix index bug
* add past/present/future connection test and clean up code
* fmt
* network bridge: updated types
* update protocols to new gossip topology message
* guide updates
* add session to BlockApprovalMeta
* add session to block info
* refactor knowledge and remove most unify logic
* start replacing gossip_peers with new SessionTopologies
* add routing information to message state
* add some utilities to SessionTopology
* implement new gossip topology logic
* re-implement unify_with_peer
* distribute assignments according to topology
* finish grid topology implementation
* refactor network bridge slightly
* issue connection requests on all past/present/future
* fmt
* address grumbles
* tighten invariants in unify_with_peer
* implement random propagation
* refactor: extract required routing adjustment logic
* some block-age logic
* aggressively propagate messages when finality is slow
* overhaul aggression system to have 3 levels
* add aggression metrics
* remove aggression L3
* reduce random circulation
* remove PeerData
* get approval tests compiling
* use btree_map in known_by to make deterministic
* Revert "use btree_map in known_by to make deterministic"
This reverts commit 330d65343a7bb6fe4dd0f24bd8dbc15c0cbdbd9d.
* test XY grid propagation
* remove stray println
* test unshared dimension propagation
* add random gossip check
* test unify_with_peer better
* test sending after getting gossip topology
* test L1 aggression on originator
* test L1 aggression for non-originators
* test non-originator aggression L2
* fnt
* ~spellcheck
* fix statement-distribution tests
* fix flaky test
* fix metrics typo
* re-send periodically
* test resending
* typo
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* add more metrics about apd messages
* add back unify_with_peer logs
* make Resend an enum
* be more explicit when resending
* fmt
* fix error
* add a TODO for refactoring
* remove debug metrics
* add some guide stuff
* fmt
* update runtime API in test-runtim
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* remove v0 primitives from polkadot-primitives
* first pass: remove v0
* fix fallout in erasure-coding
* remove v1 primitives, consolidate to v2
* the great import update
* update runtime_api_impl_v1 to v2 as well
* guide: add `Version` request for runtime API
* add version query to runtime API
* reintroduce OldV1SessionInfo in a limited way
This commit implements the last major piece of #3211: the subsystem that
tracks PVFs that require voting, issues pre-check requests to
candidate-validation and makes sure that the votes are submitted to the
chain.
This commit hooks up the API provided by #4457 to the runtime API
subsystem. In a following PR this API will be consumed by the PVF
pre-checking subsystem.
Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
As part of #3211 we will need to add a couple of runtime APIs. That
change is coming in a following PR.
The runtime API for pre-checking will be introduced with a version bump
for `ParachainHost`. This commit prepares the ground for that change,
by introducing a error condition that signals that the given API is not
supported.
* Mostly notes.
* Better error messages.
* Introduce Fatal/NonFatal + drop back channel participation
- Fatal/NonFatal - in order to make it easier to use utility functions.
- We drop the back channel in dispute participation as it won't be
needed any more.
* Better error messages.
* Utility function for receiving `CandidateEvent`s.
* Ordering module typechecks.
* cargo fmt
* Prepare spam slots module.
* Implement SpamSlots mechanism.
* Implement queues.
* cargo fmt
* Participation.
* Participation taking shape.
* Finish participation.
* cargo fmt
* Cleanup.
* WIP: Cleanup + Integration.
* Make `RollingSessionWindow` initialized by default.
* Make approval voting typecheck.
* Get rid of lazy_static & fix approval voting tests
* Move `SessionWindowSize` to node primitives.
* Implement dispute coordinator initialization.
* cargo fmt
* Make queues return error instead of boolean.
* Initialized: WIP
* Introduce chain api for getting finalized block.
* Fix ordering to only prune candidates on finalized events.
* Pruning of old sessions in spam slots.
* New import logic.
* Make everything typecheck.
* Fix warnings.
* Get rid of obsolete dispute-participation.
* Fixes.
* Add back accidentelly deleted Cargo.lock
* Deliver disputes in an ordered fashion.
* Add module docs for errors
* Use type synonym.
* hidden docs.
* Fix overseer tests.
* Ordering provider taking `CandidateReceipt`.
... To be kicked on one next commit.
* Fix ordering to use relay_parent
as included block is not unique per candidate.
* Add comment in ordering.rs.
* Take care of duplicate entries in queues.
* Better spam slots.
* Review remarks + docs.
* Fix db tests.
* Participation tests.
* Also scrape votes on first leaf for good measure.
* Make tests typecheck.
* Spelling.
* Only participate in actual disputes, not on every import.
* Don't account backing votes to spam slots.
* Fix more tests.
* Don't participate if we don't have keys.
* Fix tests, typos and warnings.
* Fix merge error.
* Spelling fixes.
* Add missing docs.
* Queue tests.
* More tests.
* Add metrics + don't short circuit import.
* Basic test for ordering provider.
* Import fix.
* Remove dead link.
* One more dead link.
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
* Substrate companion: Authority discovery multiple peer ids
Authority discovery before had a fixed mapping from `PeerId` to
`AuthorityId`. This wasn't correct, as a `PeerId` can actually map to
multiple `AuthorityId`s. The linked Substrate pr fixes this.
https://github.com/paritytech/substrate/pull/10259
* Update node/network/availability-distribution/src/requester/mod.rs
* Update node/network/collator-protocol/src/validator_side/mod.rs
* Update node/network/statement-distribution/src/tests.rs
* Update guide
* Adapt to Substrate pr
* Update Substrate
* Introduce new Runtime API endpoint
`persisted_validation_data_with_code_hash` that will be used
by the candidate validation subsystem in order to decrease amount
of runtime API requests.
* Node-side part of new runtime API request
* Define code hash getter via macro
* Rename new endpoint to `assumed_validation_data`
* Docs for runtime API impl of new endpoint
* AssumedValidationData specialized request function
* fmt
* dummy: impl another runtime API
* query the on chain disputes, and inform self
* make use of the refactor
* minro
* SPLIT ME
* write dispute values
* wip
* impl for all runtimes
* chore: fmt
* [] -> get
* fixup mock runtime
* fixup
* fixup discovery for overseer init
* chore: fmt
* spellcheck
* rename imported_on_chain_disputes -> on_chain_votes
* reduction
* make it mockable
* rename and refactor
* don't query on chain info if it's not needed
* yikes
* fmt
* fix test
* minimal fix for existing tests
* attempt to fetch the session info from the rolling window before falling back
* moved
* comments
* comments
* test for backing votes
* rename
* Update runtime/polkadot/src/lib.rs
* chore: spellcheck + dict
* chore: fmt
* fixup cache size
* add warning
* logging, rationale, less defense
* introduce new unchecked, that still checks in debug builds
* fix
* draft alt approach
* fix unused imports
* include the session
* Update node/core/dispute-coordinator/src/real/mod.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* provide where possible
* expand comment
* fixin
* fixup
* ValidityVote <-> ValidityAttestation <-> CompactStatement has a 1:1 representation
* mark TODO
* Update primitives/src/v1/mod.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* address review comments
* update docs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* pvf: make execution timeout configurable
* guide: add timeouts to candidate validation params
* add timeouts to candidate validation messages
* fmt
* port backing to use the backing pvf timeout
* port approval-voting to use the execution timeout
* port dispute participation to use the correct timeout
* fmt
* address grumbles & test failure
* bump version and spec_version
* bump transaction version
* Bump of the default value in CI
* bump tx version for westend and kusama
Co-authored-by: Wilfried Kopp <wilfried@parity.io>
* Attempt to add log stats to gossip-support.
* WIP: Keep track of connected validators.
* Clarify metric.
* WIP: Make gossip support report connectivity.
* WIP: Fixing tests.
* Fix network bridge + integrate in overseer.
* Consistent naming.
* Fix logic error
* cargo fmt
* Pretty logs.
* cargo fmt
* Use `Delay` to trigger periodic checks.
* fmt
* Fix warning for authority set size of 1.
* More correct ratio report if there are no resolved validators.
* Prettier rendering of empty set.
* Fix typo.
* Another typo.
* Don't check on every leaf update.
* Make compatible with older rustc.
* Fix tests.
* Demote warning.
Chevdor