This commit hooks up the API provided by #4457 to the runtime API
subsystem. In a following PR this API will be consumed by the PVF
pre-checking subsystem.
Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
As part of #3211 we will need to add a couple of runtime APIs. That
change is coming in a following PR.
The runtime API for pre-checking will be introduced with a version bump
for `ParachainHost`. This commit prepares the ground for that change,
by introducing a error condition that signals that the given API is not
supported.
* Companion PR for removing Prometheus metrics prefix
* Was missing some metrics
* Fix missing renames
* Fix test
* Fixes
* Update test
* Update Substrate
* Second time
* remove prefix from intergration test for zombienet
* update zombienet image
* Update Substrate
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
* remove Default from CandidateHash
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* chore: fmt
* remove backed candidate default
* Partial migration away from CandidateReceipt::default
* Remove more CandidateReceipt defaults
* fmt
* Mostly remove CommittedCandidateReceipt default usage
* Remove CommittedCandidateReceipt
* Remove more Defaults from polakdot primitives v1 + fmt
* Remove more Default from polkadot primites v1
* WIP trying to get overseer example + tests to compile
* feat: add primitives test helpers
* reduce deps of helper
* update primitive helpers
* make candidate validation compile
* fixup cargo lock
* make av-store compile
* fixup disputes coordinator tests
* test: fixup backing
* test: fixup approval voting
* fixup bitfield signing
* test: fixup runtime-api
* test: fixup availability dist
* foxi[ pverseer test]
* remove some Defaults, remove bounds from `dummy`
All `fn dummy` in primitives need to be removed anyways.
This aids in the transition.
* it's a test helper, so always use std
* test: fixup parachains runtime tests
Excluding benches.
* fix keyring
* fix paras runtime properly, no more default
* Remove fn dummy() usage from approval voting
* Move TestCandidateBuilder out of av store to test helpers
* Make candidate validation tests pass
* Make most dispute coirdinator tests pass
* Make provisioner tests work
* Make availability recovery tests work with test helpers
* Update polkadot-collator-protocol tests
* Update statement distribution tests
* Update polkadot overseer examples and tests
* Derive default for validation code so we don't break unrelated things
* Make para runtime test pass (no bench)
* Some more work
* chore: cargo fmt
* cargo fix
* avoid some Default::default
* fixup dispute coordinator test
* remove unused crate deps
* remove Default::default wherever possible, replace by dummy_* for the most part
* chore: cargo fmt
* Remove some warnings
* Remove CommittedCandidateReceipt dummy
* Remove CandidateReceipt dummy
* Remove CandidateDescriptor dummy
* Remove commented out code
* Fix para runtime tests
* chore: nightly
* Some updates to the builder
* Dynamically adjust mock head data size
* Make dispute cooridinator tests work
* Fix test candidate_backing_reorders_votes work
* +nightly-2021-10-29 fmt
* Spelling and remove a default use in builder
* Various clean up
* More small updates
* fmt
* More small updates
* Doc comments for test helpers
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
* Update lib.rs
* review comments
* fix warnings
* fix test by using correct candidate receipt relay parent
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>
* Introduce new Runtime API endpoint
`persisted_validation_data_with_code_hash` that will be used
by the candidate validation subsystem in order to decrease amount
of runtime API requests.
* Node-side part of new runtime API request
* Define code hash getter via macro
* Rename new endpoint to `assumed_validation_data`
* Docs for runtime API impl of new endpoint
* AssumedValidationData specialized request function
* fmt
* dummy: impl another runtime API
* query the on chain disputes, and inform self
* make use of the refactor
* minro
* SPLIT ME
* write dispute values
* wip
* impl for all runtimes
* chore: fmt
* [] -> get
* fixup mock runtime
* fixup
* fixup discovery for overseer init
* chore: fmt
* spellcheck
* rename imported_on_chain_disputes -> on_chain_votes
* reduction
* make it mockable
* rename and refactor
* don't query on chain info if it's not needed
* yikes
* fmt
* fix test
* minimal fix for existing tests
* attempt to fetch the session info from the rolling window before falling back
* moved
* comments
* comments
* test for backing votes
* rename
* Update runtime/polkadot/src/lib.rs
* chore: spellcheck + dict
* chore: fmt
* fixup cache size
* add warning
* logging, rationale, less defense
* introduce new unchecked, that still checks in debug builds
* fix
* draft alt approach
* fix unused imports
* include the session
* Update node/core/dispute-coordinator/src/real/mod.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* provide where possible
* expand comment
* fixin
* fixup
* ValidityVote <-> ValidityAttestation <-> CompactStatement has a 1:1 representation
* mark TODO
* Update primitives/src/v1/mod.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* address review comments
* update docs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* CI: add spellcheck
* revert me
* CI: explicit command for spellchecker
* spellcheck: edit misspells
* CI: run spellcheck on diff
* spellcheck: edits
* spellcheck: edit misspells
* spellcheck: add rules
* spellcheck: mv configs
* spellcheck: more edits
* spellcheck: chore
* spellcheck: one more thing
* spellcheck: and another one
* spellcheck: seems like it doesn't get to an end
* spellcheck: new words after rebase
* spellcheck: new words appearing out of nowhere
* chore
* review edits
* more review edits
* more edits
* wonky behavior
* wonky behavior 2
* wonky behavior 3
* change git behavior
* spellcheck: another bunch of new edits
* spellcheck: new words are koming out of nowhere
* CI: finding the master
* CI: fetching master implicitly
* CI: undebug
* new errors
* a bunch of new edits
* and some more
* Update node/core/approval-voting/src/approval_db/v1/mod.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update xcm/xcm-executor/src/assets.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* Suggestions from the code review
* CI: scan only changed files
Co-authored-by: Andronik Ordian <write@reusable.software>
* code stored in para + modify CandidateDescriptor.
* WIP: digest + some more impl
* validation_code_hash in payload + check in inclusion
* check in client + refator
* tests
* fix encoding indices
* remove old todos
* fix test
* fix test
* add test
* fetch validation code inside collation-generation from the relay-chain
* HashMismatch -> PoVHashMismatch + miscompilation
* refactor, store hash when needed
* storage rename: more specific but slightly too verbose
* do not hash on candidate validation, fetch hash instead
* better test
* fix test
* guide updates
* don't panic in runtime
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Add a config field to babe epochs
* Fix test
* Add BABE_GENESIS_EPOCH_CONFIG consts
* Use PrimaryAndSecondaryVRFSlots and remove newlines
* Make epoch_configs Some
* Fix tests
* Fix test service tests
* Add a BabeEpochConfigMigrations OnRuntimeUpgrade
* Apply suggestions
* Use PrimaryAndSecondaryPlainSlots in kusama
* Remove migration from test runtime and rococo
* Add HasPalletPrefix
* Rename to BabePalletPrefix and change BabeApi -> Babe
* "Update Substrate"
* Update substrate
* Resolve parantheses errors
Co-authored-by: parity-processbot <>
* skeleton
* skeleton aux-schema module
* start approval types
* start aux schema with aux store
* doc
* finish basic types
* start approval types
* doc
* finish basic types
* write out schema types
* add debug and codec impls to approval types
* add debug and codec impls to approval types
also add some key computation
* add debug and codec impls to approval types
* getters for block and candidate entries
* grumbles
* remove unused AssignmentId
* load_decode utility
* implement DB clearing
* function for adding new block entry to aux store
* start `canonicalize` implementation
* more skeleton
* finish implementing canonicalize
* tag TODO
* implement a test AuxStore
* add allow(unused)
* basic loading and deleting test
* block_entry test function
* add a test for `add_block_entry`
* ensure range is exclusive at end
* test clear()
* test that add_block sets children
* add a test for canonicalize
* extract Pre-digest from header
* utilities for extracting RelayVRFStory from the header-chain
* add approval voting message types
* approval distribution message type
* subsystem skeleton
* state struct
* add futures-timer
* prepare service for babe slot duration
* more skeleton
* better integrate AuxStore
* RelayVRF -> RelayVRFStory
* canonicalize
* implement some tick functionality
* guide: tweaks
* check_approval
* more tweaks and helpers
* guide: add core index to candidate event
* primitives: add core index to candidate event
* runtime: add core index to candidate events
* head handling (session window)
* implement `determine_new_blocks`
* add TODO
* change error type on functions
* compute RelayVRFModulo assignments
* compute RelayVRFDelay assignments
* fix delay tranche calc
* assignment checking
* pluralize
* some dummy code for fetching assignments
* guide: add babe epoch runtime API
* implement a current_epoch() runtime API
* compute assignments
* candidate events get backing group
* import blocks and assignments into DB
* push block approval meta
* add message types, no overseer integration yet
* notify approval distribution of new blocks
* refactor import into separate functions
* impl tranches_to_approve
* guide: improve function signatures
* guide: remove Tick from ApprovalEntry
* trigger and broadcast assignment
* most of approval launching
* remove byteorder crate
* load blocks back to finality, except on startup
* check unchecked assignments
* add claimed core to approval voting message
* fix checks
* assign only to backing group
* remove import_checked_assignment from guide
* newline
* import assignments
* abstract out a bit
* check and import approvals
* check full approvals from assignment import too
* comment
* create a Transaction utility
* must_use
* use transaction in `check_full_approvals`
* wire up wakeups
* add Ord to CandidateHash
* wakeup refactoring
* return candidate info from add_block_entry
* schedule wakeups
* background task: do candidate validation
* forward candidate validation requests
* issue approval votes when requested
* clean up a couple TODOs
* fix up session caching
* clean up last unimplemented!() items
* fix remaining warnings
* remove TODO
* implement handle_approved_ancestor
* update Cargo.lock
* fix runtime API tests
* guide: cleanup assignment checking
* use claimed candidate index instead of core
* extract time to a trait
* tests module
* write a mock clock for testing
* allow swapping out the clock
* make abstract over assignment criteria
* add some skeleton tests and simplify params
* fix backing group check
* do backing group check inside check_assignment_cert
* write some empty test functions to implement
* add a test for non-backing
* test that produced checks pass
* some empty test ideas
* runtime/inclusion: remove outdated TODO
* fix compilation
* av-store: fix tests
* dummy cert
* criteria tests
* move `TestStore` to main tests file
* fix unused warning
* test harness beginnings
* resolve slots renaming fallout
* more compilation fixes
* wip: extract pure data into a separate module
* wip: extract pure data into a separate module
* move types completely to v1
* add persisted_entries
* add conversion trait impls
* clean up some warnings
* extract import logic to own module
* schedule wakeups
* experiment with Actions
* uncomment approval-checking
* separate module for approval checking utilities
* port more code to use actions
* get approval pipeline using actions
* all logic is uncommented
* main loop processes actions
* all loop logic uncommented
* separate function for handling actions
* remove last unimplemented item
* clean up warnings
* State gives read-only access to underlying DB
* tests for approval checking
* tests for approval criteria
* skeleton test module for import
* list of import tests to do
* some test glue code
* test reject bad assignment
* test slot too far in future
* test reject assignment with unknown candidate
* remove loads_blocks tests
* determine_new_blocks back to finalized & harness
* more coverage for determining new blocks
* make `imported_block_info` have less reliance on State
* candidate_info tests
* tests for session caching
* remove println
* extricate DB and main TestStores
* rewrite approval checking logic to counteract early delays
* move state out of function
* update approval-checking tests
* tweak wakeups & scheduling logic
* rename check_full_approvals
* test that assignment import updates candidate
* some approval import tests
* some tests for check_and_apply_approval
* add 'full' qualifier to avoid confusion
* extract should-trigger logic to separate function
* some tests for all triggering
* tests for when we trigger assignments
* test wakeups
* add block utilities for testing
* some more tests for approval updates
* approved_ancestor tests
* new action type for launch approval
* process-wakeup tests
* clean up some warnings
* fix in_future test
* approval checking tests
* tighten up too-far-in-future
* special-case genesis when caching sessions
* fix bitfield len
Co-authored-by: Andronik Ordian <write@reusable.software>
* collation-generation: use persisted validation data
* node: remote FullValidationData API
* runtime: remove FullValidationData API
* backing tests: use persisted validation data
* FullCandidateReceipt: use persisted validation data
This is not a big change since this type is not used anywhere
* Remove ValidationData and TransientValidationData
Also update the guide
* refactor View to include finalized_number
* guide: update the NetworkBridge on BlockFinalized
* av-store: fix the tests
* actually fix tests
* grumbles
* ignore macro doctest
* use Hash::repeat_bytes more consistently
* broadcast empty leaves updates as well
* fix issuing view updates on empty leaves updates
* Add an upper number of maximum parallel runtime api requests
Instead of spawning all runtime api requests in the background and using
all wasm instances. This pr adds a maximum number of parallel requests.
* Update node/core/runtime-api/src/lib.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Review feedback
* Increase instances
* Add warning
* Update node/core/runtime-api/src/lib.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
This pr changes how the runtime api subsystem processes runtime api
requests. Instead of answering all of them in the subsystem task and
thus, making all requests sequential, we now answer them in a background
task. This enables us to serve multiple requests at once.
* guide: fix formatting for SessionInfo module
* primitives: SessionInfo type
* punt on approval keys
* ah, revert the type alias
* session info runtime module skeleton
* update the guide
* runtime/configuration: sync with the guide
* runtime/configuration: setters for newly added fields
* runtime/configuration: set codec indexes
* runtime/configuration: update test
* primitives: fix SessionInfo definition
* runtime/session_info: initial impl
* runtime/session_info: use initializer for session handling (wip)
* runtime/session_info: mock authority discovery trait
* guide: update the initializer's order
* runtime/session_info: tests skeleton
* runtime/session_info: store n_delay_tranches in Configuration
* runtime/session_info: punt on approval keys
* runtime/session_info: add some basic tests
* Update primitives/src/v1.rs
* small fixes
* remove codec index annotation on structs
* fix off-by-one error
* validator_discovery: accept a session index
* runtime: replace validator_discovery api with session_info
* Update runtime/parachains/src/session_info.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* runtime/session_info: add a comment about missing entries
* runtime/session_info: define the keys
* util: expose connect_to_past_session_validators
* util: allow session_info requests for jobs
* runtime-api: add mock test for session_info
* collator-protocol: add session_index to test state
* util: fix error message for runtime error
* fix compilation
* fix tests after merge with master
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* use snake_case for log targets
* remove unused continue
* validator_discovery: when disconnecting, use all addresses
* validator_discovery: simplify request revokation
* fix a typo
* reexport prometheus-super for ease of use of other subsystems
* add some prometheus timers for collation generation subsystem
* add timing metrics to av-store
* add metrics to candidate backing
* add timing metric to bitfield signing
* add timing metrics to candidate selection
* add timing metrics to candidate-validation
* add timing metrics to chain-api
* add timing metrics to provisioner
* add timing metrics to runtime-api
* add timing metrics to availability-distribution
* add timing metrics to bitfield-distribution
* add timing metrics to collator protocol: collator side
* add timing metrics to collator protocol: validator side
* fix candidate validation test failures
* add timing metrics to pov distribution
* add timing metrics to statement-distribution
* use substrate_prometheus_endpoint prometheus reexport instead of prometheus_super
* don't include JOB_DELAY in bitfield-signing metrics
* give adder-collator ability to easily export its genesis-state and validation code
* wip: adder-collator pushbutton script
* don't attempt to register the adder-collator automatically
Instead, get these values with
```sh
target/release/adder-collator export-genesis-state
target/release/adder-collator export-genesis-wasm
```
And then register the parachain on https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944#/explorer
To collect prometheus data, after running the script, create `prometheus.yml` per the instructions
at https://www.notion.so/paritytechnologies/Setting-up-Prometheus-locally-835cb3a9df7541a781c381006252b5ff
and then run:
```sh
docker run -v `pwd`/prometheus.yml:/etc/prometheus/prometheus.yml:z --network host prom/prometheus
```
Demonstrates that data makes it across to prometheus, though it is likely to be useful in the future
to tweak the buckets.
* Update parachain/test-parachains/adder/collator/src/cli.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* use the grandpa-pause parameter
* skip metrics in tracing instrumentation
* remove unnecessary grandpa_pause cli param
Co-authored-by: Andronik Ordian <write@reusable.software>
* drop in tracing to replace log
* add structured logging to trace messages
* add structured logging to debug messages
* add structured logging to info messages
* add structured logging to warn messages
* add structured logging to error messages
* normalize spacing and Display vs Debug
* add instrumentation to the various 'fn run'
* use explicit tracing module throughout
* fix availability distribution test
* don't double-print errors
* remove further redundancy from logs
* fix test errors
* fix more test errors
* remove unused kv_log_macro
* fix unused variable
* add tracing spans to collation generation
* add tracing spans to av-store
* add tracing spans to backing
* add tracing spans to bitfield-signing
* add tracing spans to candidate-selection
* add tracing spans to candidate-validation
* add tracing spans to chain-api
* add tracing spans to provisioner
* add tracing spans to runtime-api
* add tracing spans to availability-distribution
* add tracing spans to bitfield-distribution
* add tracing spans to network-bridge
* add tracing spans to collator-protocol
* add tracing spans to pov-distribution
* add tracing spans to statement-distribution
* add tracing spans to overseer
* cleanup
* HRMP: Update the impl guide
* HRMP: Incorporate the channel notifications into the guide
* HRMP: Renaming in the impl guide
* HRMP: Constrain the maximum number of HRMP messages per candidate
This commit addresses the HRMP part of https://github.com/paritytech/polkadot/issues/1869
* XCM: Introduce HRMP related message types
* HRMP: Data structures and plumbing
* HRMP: Configuration
* HRMP: Data layout
* HRMP: Acceptance & Enactment
* HRMP: Test base logic
* Update adder collator
* HRMP: Runtime API for accessing inbound messages
Also, removing some redundant fully-qualified names.
* HRMP: Add diagnostic logging in acceptance criteria
* HRMP: Additional tests
* Self-review fixes
* save test refactorings for the next time
* Missed a return statement.
* a formatting blip
* Add missing logic for appending HRMP digests
* Remove the channel contents vectors which became empty
* Tighten HRMP channel digests invariants.
* Apply suggestions from code review
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Remove a note about sorting for channel id
* Add missing rustdocs to the configuration
* Clarify and update the invariant for HrmpChannelDigests
* Make the onboarding invariant less sloppy
Namely, introduce `Paras::is_valid_para` (in fact, it already is present
in the implementation) and hook up the invariant to that.
Note that this says "within a session" because I don't want to make it
super strict on the session boundary. The logic on the session boundary
should be extremely careful.
* Make `CandidateCheckContext` use T::BlockNumber for hrmp_watermark
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* fix: ensure candidate validation gets code based on occupied core assumption
* guide: runtime API for historical validation code
* add historical runtime API
* integrate into runtime API subsystem
* remove blocked TODO
* fix service build: enable notifications protocol only under real overseer
* Update node/subsystem/src/messages.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* fix compilation
Co-authored-by: Robert Habermeier <robert@Roberts-MacBook-Pro.local>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* DMP: data structures and plumbing
* DMP: Implement DMP logic in the router module
DMP: Integrate DMP parts into the inclusion module
* DMP: Introduce the max size limit for the size of a downward message
* DMP: Runtime API for accessing inbound messages
* OCD
Small clean ups
* DMP: fix the naming of the error
* DMP: add caution about a non-existent recipient
* start working on building the real overseer
Unfortunately, this fails to compile right now due to an upstream
failure to compile which is probably brought on by a recent upgrade
to rustc v1.47.
* fill in AllSubsystems internal constructors
* replace fn make_metrics with Metrics::attempt_to_register
* update to account for #1740
* remove Metrics::register, rename Metrics::attempt_to_register
* add 'static bounds to real_overseer type params
* pass authority_discovery and network_service to real_overseer
It's not straightforwardly obvious that this is the best way to handle
the case when there is no authority discovery service, but it seems
to be the best option available at the moment.
* select a proper database configuration for the availability store db
* use subdirectory for av-store database path
* apply Basti's patch which avoids needing to parameterize everything on Block
* simplify path extraction
* get all tests to compile
* Fix Prometheus double-registry error
for debugging purposes, added this to node/subsystem-util/src/lib.rs:472-476:
```rust
Some(registry) => Self::try_register(registry).map_err(|err| {
eprintln!("PrometheusError calling {}::register: {:?}", std::any::type_name::<Self>(), err);
err
}),
```
That pointed out where the registration was failing, which led to
this fix. The test still doesn't pass, but it now fails in a new
and different way!
* authorities must have authority discovery, but not necessarily overseer handlers
* fix broken SpawnedSubsystem impls
detailed logging determined that using the `Box::new` style of
future generation, the `self.run` method was never being called,
leading to dropped receivers / closed senders for those subsystems,
causing the overseer to shut down immediately.
This is not the final fix needed to get things working properly,
but it's a good start.
* use prometheus properly
Prometheus lets us register simple counters, which aren't very
interesting. It also allows us to register CounterVecs, which are.
With a CounterVec, you can provide a set of labels, which can
later be used to filter the counts.
We were using them wrong, though. This pattern was repeated in a
variety of places in the code:
```rust
// panics with an cardinality mismatch
let my_counter = register(CounterVec::new(opts, &["succeeded", "failed"])?, registry)?;
my_counter.with_label_values(&["succeeded"]).inc()
```
The problem is that the labels provided in the constructor are not
the set of legal values which can be annotated, but a set of individual
label names which can have individual, arbitrary values.
This commit fixes that.
* get av-store subsystem to actually run properly and not die on first signal
* typo fix: incomming -> incoming
* don't disable authority discovery in test nodes
* Fix rococo-v1 missing session keys
* Update node/core/av-store/Cargo.toml
* try dummying out av-store on non-full-nodes
* overseer and subsystems are required only for full nodes
* Reduce the amount of warnings on browser target
* Fix two more warnings
* InclusionInherent should actually have an Inherent module on rococo
* Ancestry: don't return genesis' parent hash
* Update Cargo.lock
* fix broken test
* update test script: specify chainspec as script argument
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update node/service/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* node/service/src/lib: Return error via ? operator
* post-merge blues
* add is_collator flag
* prevent occasional av-store test panic
* simplify fix; expand application
* run authority_discovery in Role::Discover when collating
* distinguish between proposer closed channel errors
* add IsCollator enum, remove is_collator CLI flag
* improve formatting
* remove nop loop
* Fix some stuff
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Robert Habermeier <robert@Roberts-MBP.lan1>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
* annoying whitespaces
* update guide
Add `CheckValidationOutputs` runtime api and also change the
candidate-validation stuff
* promote ValidationOutputs to global primitives
i.e. move it from node specific primitives to global v1 primitives. This
will be needed when we share it later in the runtime inclusion module
* refactor acceptance checks in the inclusion module
factor out the common code to share it during the block inclusion and
for the forthcoming CheckValidationOutputs runtime api.
Also note that the acceptance criteria was updated to incorporate checks
that exist now in candidate-validation
* plumb the runtime api outside
* extract validation_data from ValidationOutputs
* use runtime-api to check validation outputs
apart from that refactor, update docs and tidy a bit
* Update the maxium code size
This is to fix a test that performs an upgrade.
* stupid, but it compiles
* redo
* cleanup
* add ValidatorDiscovery to msgs
* sketch network bridge code
* ConnectToAuthorities instead of validators
* more stuff
* cleanup
* more stuff
* complete ConnectToAuthoritiesState
* Update node/network/bridge/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Collator protocol subsystem (#1659)
* WIP
* The initial implementation of the collator side.
* Improve comments
* Multiple collation requests
* Add more tests and comments to validator side
* Add comments, remove dead code
* Apply suggestions from code review
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Fix build after suggested changes
* Also connect to the next validator group
* Remove a Future impl and move TimeoutExt to util
* Minor nits
* Fix build
* Change FetchCollations back to FetchCollation
* Try this
* Final fixes
* Fix build
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* handle multiple in-flight connection requests
* handle cancelled requests
* Update node/core/runtime-api/src/lib.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* redo it again
* more stuff
* redo it again
* update comments
* workaround Future is not Send
* fix trailing spaces
* clarify comments
* bridge: fix compilation in tests
* update more comments
* small fixes
* port collator protocol to new validator discovery api
* collator tests compile
* collator tests pass
* do not revoke a request when the stream receiver is closed
* make revoking opt-in
* fix is_fulfilled
* handle request revokation in collator
* tests
* wait for validator connections asyncronously
* fix compilation
* relabel my todos
* apply Fedor's patch
* resolve reconnection TODO
* resolve revoking TODO
* resolve channel capacity TODO
* resolve peer cloning TODO
* resolve peer disconnected TODO
* resolve PeerSet TODO
* wip tests
* more tests
* resolve Arc TODO
* rename pending to non_revoked
* one more test
* extract utility function into util crate
* fix compilation in tests
* Apply suggestions from code review
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
* revert pin_project removal
* fix while let loop
* Revert "revert pin_project removal"
This reverts commit ae7f529d8de982ef66c3007dd1ff74c6ddce80d2.
* fix compilation
* Update node/subsystem/src/messages.rs
* docs on pub items
* guide updates
* remove a TODO
* small guide update
* fix a typo
* link to the issue
* validator discovery: on_request docs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* update primitives
* correct parent_head field
* make hrmp field pub
* refactor validation data: runtime
* refactor validation data: messages
* add arguments to full_validation_data runtime API
* port runtime API
* mostly port over candidate validation
* remove some parameters from ValidationParams
* guide: update candidate validation
* update candidate outputs
* update ValidationOutputs in primitives
* port over candidate validation
* add a new test for no-transient behavior
* update util runtime API wrappers
* candidate backing
* fix missing imports
* change some fields of validation data around
* runtime API impl
* update candidate validation
* fix backing tests
* grumbles from review
* fix av-store tests
* fix some more crates
* fix provisioner tests
* fix availability distribution tests
* port collation-generation to new validation data
* fix overseer tests
* Update roadmap/implementers-guide/src/node/utility/candidate-validation.md
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* service-new: cosmetic changes
* overseer: draft of prometheus metrics
* metrics: update active_leaves metrics
* metrics: extract into functions
* metrics: resolve XXX
* metrics: it's ugly, but it works
* Bump Substrate
* metrics: move a bunch of code around
* Bumb substrate again
* metrics: fix a warning
* fix a warning in runtime
* metrics: statements signed
* metrics: statements impl RegisterMetrics
* metrics: refactor Metrics trait
* metrics: add Metrics assoc type to JobTrait
* metrics: move Metrics trait to util
* metrics: fix overseer
* metrics: fix backing
* metrics: fix candidate validation
* metrics: derive Default
* metrics: docs
* metrics: add stubs for other subsystems
* metrics: add more stubs and fix compilation
* metrics: fix doctest
* metrics: move to subsystem
* metrics: fix candidate validation
* metrics: bitfield signing
* metrics: av store
* metrics: chain API
* metrics: runtime API
* metrics: stub for avad
* metrics: candidates seconded
* metrics: ok I gave up
* metrics: provisioner
* metrics: remove a clone by requiring Metrics: Sync
* metrics: YAGNI
* metrics: remove another TODO
* metrics: for later
* metrics: add parachain_ prefix
* metrics: s/signed_statement/signed_statements
* utils: add a comment for job metrics
* metrics: address review comments
* metrics: oops
* metrics: make sure to save files before commit 😅
* use _total suffix for requests metrics
Co-authored-by: Max Inden <mail@max-inden.de>
* metrics: add tests for overseer
* update Cargo.lock
* overseer: add a test for CollationGeneration
* collation-generation: impl metrics
* collation-generation: use kebab-case for name
* collation-generation: add a constructor
Co-authored-by: Gav Wood <gavin@parity.io>
Co-authored-by: Ashley Ruglys <ashley.ruglys@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
* type defaults for ParachainHost
* add ValidationCode message
* implement core loop of runtime API subsystem
* subsystem trait implementation for runtime API subsystem
* implement a mock runtime API
* some tests that ensure requests are forwarded to runtime API correctly
* fix dependency grumbles
* improve RuntimeApiError API
Sergei Shulepov