* remove real-overseer
* overseer: only activate leaves which support parachains
* integrate HeadSupportsParachains into service
* remove unneeded line
* Remove stuff out of the runtime that does not belong there.
There might be more, but it is a start.
* White space fixes.
* Fix tests.
* Leave whitespace in ui tests alone.
* Add back zstd for no reason.
* Fix browser wasm (hopefully)
* drop in tracing to replace log
* add structured logging to trace messages
* add structured logging to debug messages
* add structured logging to info messages
* add structured logging to warn messages
* add structured logging to error messages
* normalize spacing and Display vs Debug
* add instrumentation to the various 'fn run'
* use explicit tracing module throughout
* fix availability distribution test
* don't double-print errors
* remove further redundancy from logs
* fix test errors
* fix more test errors
* remove unused kv_log_macro
* fix unused variable
* add tracing spans to collation generation
* add tracing spans to av-store
* add tracing spans to backing
* add tracing spans to bitfield-signing
* add tracing spans to candidate-selection
* add tracing spans to candidate-validation
* add tracing spans to chain-api
* add tracing spans to provisioner
* add tracing spans to runtime-api
* add tracing spans to availability-distribution
* add tracing spans to bitfield-distribution
* add tracing spans to network-bridge
* add tracing spans to collator-protocol
* add tracing spans to pov-distribution
* add tracing spans to statement-distribution
* add tracing spans to overseer
* cleanup
* Make `AllSubsystems` usage easier in tests
This makes the usage of `AllSubsystems` easier in tests by introducing
new methods.
- `dummy` initializes `AllSubsystems` with all systems set to dummy
- `replace_*` to replace any subsystem
Besides that this pr adds a `ForwardSubsystem` that is also useful for
tests. This subsystem will forward all incoming messages to the given channel.
* Update node/overseer/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update node/subsystem/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update node/subsystem/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Move ForwardSubsystem and add a test
* Break some lines
Co-authored-by: Andronik Ordian <write@reusable.software>
* service-new: cosmetic changes
* overseer: draft of prometheus metrics
* metrics: update active_leaves metrics
* metrics: extract into functions
* metrics: resolve XXX
* metrics: it's ugly, but it works
* Bump Substrate
* metrics: move a bunch of code around
* Bumb substrate again
* metrics: fix a warning
* fix a warning in runtime
* metrics: statements signed
* metrics: statements impl RegisterMetrics
* metrics: refactor Metrics trait
* metrics: add Metrics assoc type to JobTrait
* metrics: move Metrics trait to util
* metrics: fix overseer
* metrics: fix backing
* metrics: fix candidate validation
* metrics: derive Default
* metrics: docs
* metrics: add stubs for other subsystems
* metrics: add more stubs and fix compilation
* metrics: fix doctest
* metrics: move to subsystem
* metrics: fix candidate validation
* metrics: bitfield signing
* metrics: av store
* metrics: chain API
* metrics: runtime API
* metrics: stub for avad
* metrics: candidates seconded
* metrics: ok I gave up
* metrics: provisioner
* metrics: remove a clone by requiring Metrics: Sync
* metrics: YAGNI
* metrics: remove another TODO
* metrics: for later
* metrics: add parachain_ prefix
* metrics: s/signed_statement/signed_statements
* utils: add a comment for job metrics
* metrics: address review comments
* metrics: oops
* metrics: make sure to save files before commit 😅
* use _total suffix for requests metrics
Co-authored-by: Max Inden <mail@max-inden.de>
* metrics: add tests for overseer
* update Cargo.lock
* overseer: add a test for CollationGeneration
* collation-generation: impl metrics
* collation-generation: use kebab-case for name
* collation-generation: add a constructor
Co-authored-by: Gav Wood <gavin@parity.io>
Co-authored-by: Ashley Ruglys <ashley.ruglys@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
* update networking types
* port over overseer-protocol message types
* Add the collation protocol to network bridge
* message sending
* stub for ConnectToValidators
* add some helper traits and methods to protocol types
* add collator protocol message
* leaves-updating
* peer connection and disconnection
* add utilities for dispatching multiple events
* implement message handling
* add an observedrole enum with equality and no sentry nodes
* derive partial-eq on network bridge event
* add PartialEq impls for network message types
* add Into implementation for observedrole
* port over existing network bridge tests
* add some more tests
* port bitfield distribution
* port over bitfield distribution tests
* add codec indices
* port PoV distribution
* port over PoV distribution tests
* port over statement distribution
* port over statement distribution tests
* update overseer and service-new
* address review comments
* port availability distribution
* port over availability distribution tests
* propagate chain_api subsystem through various locations
* add ChainApi to AllMessages
Co-authored-by: Peter Goodspeed-Niklaus <peter.r.goodspeedniklaus@gmail.com>
* update guide to reduce confusion and TODOs
* work from previous bitfield signing effort
There were large merge issues with the old bitfield signing PR, so
we're just copying all the work from that onto this and restarting.
Much of the existing work will be discarded because we now have better
tools available, but that's fine.
* start rewriting bitfield signing in terms of the util module
* implement construct_availability_bitvec
It's not an ideal implementation--we can make it much more concurrent--
but at least it compiles.
* implement the unimplemented portions of bitfield signing
* get core availability concurrently, not sequentially
* use sp-std instead of std for a parachain item
* resolve type inference failure caused by multiple From impls
* handle bitfield signing subsystem & Allmessages variant in overseer
* fix more multi-From inference issues
* more concisely handle overflow
Co-authored-by: Andronik Ordian <write@reusable.software>
* Revert "resolve type inference failure caused by multiple From impls"
This reverts commit 7fc77805de5e5074a1b01037f8d4e3919e03e0e1.
* Revert "fix more multi-From inference issues"
This reverts commit f14ffe589e20d664d8a900ed62f68b6fb844a514.
* impl From<i32> for ParaId
* handle another instance of AllSubsystems
* improve consistency when returning existing options
Co-authored-by: Andronik Ordian <write@reusable.software>
* create a v1 primitives module
* Improve guide on availability types
* punctuate
* new parachains runtime uses new primitives
* tests of new runtime now use new primitives
* add ErasureChunk to guide
* export erasure chunk from v1 primitives
* subsystem crate uses v1 primitives
* node-primitives uses new v1 primitives
* port overseer to new primitives
* new-proposer uses v1 primitives (no ParachainHost anymore)
* fix no-std compilation for primitives
* service-new uses v1 primitives
* network-bridge uses new primitives
* statement distribution uses v1 primitives
* PoV distribution uses v1 primitives; add PoV::hash fn
* move parachain to v0
* remove inclusion_inherent module and place into v1
* remove everything from primitives crate root
* remove some unused old types from v0 primitives
* point everything else at primitives::v0
* squanch some warns up
* add RuntimeDebug import to no-std as well
* port over statement-table and validation
* fix final errors in validation and node-primitives
* add dummy Ord impl to committed candidate receipt
* guide: update CandidateValidationMessage
* add primitive for validationoutputs
* expand CandidateValidationMessage further
* bikeshed
* add some impls to omitted-validation-data and available-data
* expand CandidateValidationMessage
* make erasure-coding generic over v1/v0
* update usages of erasure-coding
* implement commitments.hash()
* use Arc<Pov> for CandidateValidation
* improve new erasure-coding method names
* fix up candidate backing
* update docs a bit
* fix most tests and add short-circuiting to make_pov_available
* fix remainder of candidate backing tests
* squanching warns
* squanch it up
* some fallout
* overseer fallout
* free from polkadot-test-service hell
* overseer: introduce a utility typemap
* it's ugly but it compiles
* move DummySubsystem to subsystem crate
* fix tests fallout
* use a struct for all subsystems
* more tests fallout
* add missing pov_distribution subsystem
* remove unused imports and bounds
* fix minimal-example
* Updates guide for CandidateBacking
* Move assignment types to primitives
* Initial implementation.
* More functionality
* use assert_matches
* Changes to report misbehaviors
* Some fixes after a review
* Remove a blank line
* Update guide and some types
* Adds run_job function
* Some comments and refactorings
* Fix review
* Remove warnings
* Use summary in kicking off validation
* Parallelize requests
* Validation provides local and global validation params
* Test issued validity tracking
* Nits from review
* network bridge skeleton
* move some primitives around and add debug impls
* protocol registration glue & abstract network interface
* add send_msgs to subsystemctx
* select logic
* transform different events into actions and handle
* implement remaining network bridge state machine
* start test skeleton
* make network methods asynchronous
* extract subsystem out to subsystem crate
* port over overseer to subsystem context trait
* fix minimal example
* fix overseer doc test
* update network-bridge crate
* write a subsystem test-helpers crate
* write a network test helper for network-bridge
* set up (broken) view test
* Revamp network to be more async-friendly and not require Sync
* fix spacing
* fix test compilation
* insert side-channel for actions
* Add some more message types to AllMessages
* introduce a test harness
* add some tests
* ensure service compiles and passes tests
* fix typo
* fix service-new compilation
* Subsystem test helpers send messages synchronously
* remove smelly action inspector
* remove superfluous let binding
* fix warnings
* Update node/network/bridge/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* fix compilation
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* introduce polkadot-node-primitives
* guide: change statement distribution message types
* guide: remove variant from `CandidateSelectionMessage`
* add a few more message types
* add TODOs
* Almost all messages
* NewBackedCandidate notification
* Formatting
* Use AttestedCandidate as BackedCandidate
* Update node/primitives/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Fix the tests
* Bring in types from #1242
* Adds network bridge messages
* More message types from doc
* use fn pointer type
* Fixes from the review
* Add missing Runtime subsystem message
* rename to CandidateValidationMessage and fix tests
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Robert Habermeier