* update primitives
* correct parent_head field
* make hrmp field pub
* refactor validation data: runtime
* refactor validation data: messages
* add arguments to full_validation_data runtime API
* port runtime API
* mostly port over candidate validation
* remove some parameters from ValidationParams
* guide: update candidate validation
* update candidate outputs
* update ValidationOutputs in primitives
* port over candidate validation
* add a new test for no-transient behavior
* update util runtime API wrappers
* candidate backing
* fix missing imports
* change some fields of validation data around
* runtime API impl
* update candidate validation
* fix backing tests
* grumbles from review
* fix av-store tests
* fix some more crates
* fix provisioner tests
* fix availability distribution tests
* port collation-generation to new validation data
* fix overseer tests
* Update roadmap/implementers-guide/src/node/utility/candidate-validation.md
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* rename GlobalValidationSchedule to GlobalValidationData
* guide: update candidate descriptor to contain validation data hash
* guide: add note in inclusion module about checking validation data hash
* primitives: update CandidateDescriptor to contain new hash
* fix payload computation
* add helpers for computing validation data to runtime modules
* guide: note routines
* inclusion: check validation data hash and fix local_validation_data bug
* add a case to candidate_checks and improve that test substantially
* bump versions
* address review comments
* add a test for including code upgrade
* bump kusama version
* bump westend & polkadot versions
* create a v1 primitives module
* Improve guide on availability types
* punctuate
* new parachains runtime uses new primitives
* tests of new runtime now use new primitives
* add ErasureChunk to guide
* export erasure chunk from v1 primitives
* subsystem crate uses v1 primitives
* node-primitives uses new v1 primitives
* port overseer to new primitives
* new-proposer uses v1 primitives (no ParachainHost anymore)
* fix no-std compilation for primitives
* service-new uses v1 primitives
* network-bridge uses new primitives
* statement distribution uses v1 primitives
* PoV distribution uses v1 primitives; add PoV::hash fn
* move parachain to v0
* remove inclusion_inherent module and place into v1
* remove everything from primitives crate root
* remove some unused old types from v0 primitives
* point everything else at primitives::v0
* squanch some warns up
* add RuntimeDebug import to no-std as well
* port over statement-table and validation
* fix final errors in validation and node-primitives
* add dummy Ord impl to committed candidate receipt
* guide: update CandidateValidationMessage
* add primitive for validationoutputs
* expand CandidateValidationMessage further
* bikeshed
* add some impls to omitted-validation-data and available-data
* expand CandidateValidationMessage
* make erasure-coding generic over v1/v0
* update usages of erasure-coding
* implement commitments.hash()
* use Arc<Pov> for CandidateValidation
* improve new erasure-coding method names
* fix up candidate backing
* update docs a bit
* fix most tests and add short-circuiting to make_pov_available
* fix remainder of candidate backing tests
* squanching warns
* squanch it up
* some fallout
* overseer fallout
* free from polkadot-test-service hell
* Downward messages, the front-end.
* Move types around to make them accessible from Parachains
* Fix compilation
* Fix branch
* Make it compile for Cumulus
* Update the branch names
* Add default generic parameter
* Implement `Partialeq`
* Move upward messages into the `ValidationResult`
* Support disabling of the runtime api
* Update branch
* Adds support for handling downward messages
* Implement sending XCMP messages as up/downward messages
* service: update to latest ServiceBuilder changes
* Make it compile
* Initial commit
Forked at: ef2aa428d7
Parent branch: origin/master
* Update substrate branch to cecton-update-polkadot-substrate
* Update substrate & polkadot to cumulus-branch
* Reset branch
* Update primitives/src/parachain.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Update runtime/common/src/parachains.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Update runtime/common/src/parachains.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Minor fixes
* Fix wasm build
Co-authored-by: Gav Wood <gavin@parity.io>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* add signed wrapper, typedef SignedStatement
* typedef SignedAvailabilityBitfield
* implement Signed wrapper
This is strictly an addition as of this commit; nothing is yet
changed in existing behavior.
* inline getters, remove review comment
* move EncodeAs, Signed from node::primitives to primitives::parachain
* Refactor SignedAvailabilityBitfield to use Signed
* don't double-encode real payload
This isn't an ideal solution, because it depends on the
implementation details of how SCALE encodes tuples, but OTOH
that behavior seems unlikely to change anytime soon.
* fix build errors
* cause the runtime to build properly with the new changes
Not sure why cargo check didn't catch this earlier; oh well.
* fix runtime tests and separate SignedStatement from SignedFullStatement
* better explain why CompactStatement exists
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
This pr ensures that the table router stays alive for the lifetime of
the validation instance. This is required to ensure that the node
responds onto gossip messages for the particular relay chain round.
Before, the table router was only kept alive for relay chain nodes that
were assigned to a Parachain, however the lifetime was also relative
short. This lead to bugs where a relay chain node did not include
PoVBlock's, because it did not receive them (rejected them on receive,
because it was not listening on the particular round).
* upgrade primitives to allow changing validation function
* set up storage schema for old parachains code
* fix compilation errors
* fix test compilation
* add some tests for past code meta
* most of the runtime logic for code upgrades
* implement old-code pruning
* add a couple tests
* clean up remaining TODOs
* add a whole bunch of tests for runtime functionality
* remove unused function
* fix runtime compilation
* extract some primitives to parachain crate
* add validation-code upgrades to validation params and result
* extend validation params with code upgrade fields
* provide maximums to validation params
* port test-parachains
* add a code-upgrader test-parachain and tests
* fix collator tests
* move test-parachains to own folder to work around compilation errors
* fix test compilation
* update the Cargo.lock
* fix parachains tests
* remove dbg! invocation
* use new pool in code-upgrader
* bump lockfile
* link TODO to issue
* Ensure that table router is always build
This pr ensures that the table router is always build, aka the future is
resolved. This is important, as the table router internally spawns tasks
to handle gossip messages. Handling gossip messages is not only required
on parachain validators, but also on relay chain validators to receive collations.
Tests are added to ensure that the assumptions hold.
* Fix compilation
* Switch to closures
* Remove empty line
* Revert "Remove empty line"
This reverts commit 0d4aaba1780aec1c8d61e1d5dcf7768918af02d9.
* Revert "Switch to closures"
This reverts commit d128c4ecc02c911552a3bfd2142b5a4f7b1338ba.
* Hybrid approach
* Rename test
* Make trait crate local
* Adds a SigningContext type
* Bump spec versions
* Fixes requested changes
* Bump ParachainHost api_version and guard signing_context call
* Improve error message
* If there is no signing_context api use default value
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* move protocol.rs into subfolder
* add trait for mocking network behavior
* add a mock version of network ops
* remove some redundant parameters from service messages
* ensure fetching erasure chunks automatically cancels
* introduce dummy ProvideRuntimeApi
* abstract over gossip somewhat
* add mock gossip handler
* skeleton test
* remove dependence of shared table on router
* remove worker dependence on its own sender
* test shutdown
* add tests
* test that gossip streams are cleaned up correctly
* refactor worker out into its own struct and reduce bound on executor
* remove reliance of tests on global thread pool
* Flag to force kusama runtime
* Chainspecs for kusama
* Polkadot config for westend
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* network/src/legacy/gossip: Wrap GossipEngine in Arc Mutex & lock it on use
`GossipEngine` in itself has no need to be Send and Sync, given that it
does not rely on separately spawned background tasks anymore.
`RegisteredMessageValidator` needs to be `Send` and `Sync` due to the
inherited trait bounds from implementing `GossipService`. In addition
`RegisteredMessageValidator` derives `Clone`. Thereby `GossipEngine`
needs to be wrapped in an `Arc` and `Mutex` to keep the status quo.
* Needed fixes.
* Fixes
* Fixed build
* Fixed build w benchmarking CLI
* Fixed building tests
* Added --dev shortcut
Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
* expunge legacy code from polkadot-network
* mostly rip out old legacy protocol from service
* ensure validation work is spawned by incoming messages
* decouple availabliity store from network logic; clean up data flow
* av_store: test helpers and use futures-abort
* update polkadot-validation to pass n_validators when submitting chunks
* fallible erasure-chunk fetching
* implement `ErasureNetworking` for new network prot
* API for registering availability store in network
* fully integrate new network service into service
* fix validation tests
* scaffolding for porting collator over to new network
* track connected validators' peer IDs and distribute collators' collations
* helper in network for fetching all checked statements
* fix adder-collator
* actually register notifications protocol
* Update service/src/lib.rs
* merge with master
* encode the candidate statement as only the hash
* refactor CandidateReceipt and CollationInfo
* introduce an abridged candidate receipt type
* erasure coding stores candidate receipt
* store omitted data instead and introduce AvailableData type
* refactor availability-store schema
* tweak schema and APIs a bit more
* get availability-store tests passing
* accept AbridgedCandidateReceipt in `set_heads`
* change statement type in primitives to be hash-only
* fix parachains runtime tests
* fix bad merge
* rewrite validation pipeline
* remove evaluation module
* use abridged candidate hash as canonical
* statement table uses abridged candidate receipts
* kill availability_store::Data struct
* port shared table to new validation pipelines
* extract full validation pipeline to helper
* remove old validation pipeline from collation module
* polkadot-validation compiles
* polkadot-validation tests compile
* make local collation available in validation service
* port legacy network code
* polkadot-network fully ported
* network: ensure fresh statement is propagated
* remove pov_block_hash from LocalValidationData
* remove candidate_hash field from AttestedCandidate and update runtime
* port runtimes to new ParachainHost definition
* port over polkadot-collator
* fix test compilation
* better fix
* remove unrelated validation work dispatch fix
* address grumbles
* fix equality check
* WIP
* WIp
* Mostly get tests to compile
* Fix adder collator
* Remove more stuff
* Revert some changes to av store
* Fix av store tests
* Nitpicks
* Restore some things
* Small changes
* Remvoe unused error variants
* extract all network code to legacy submodule
* update references to legacy proto
* skeleton of futures-based protocol
* refactor skeleton to use background task
* rename communication_for to build_table_router
* implement internal message types for validation network
* basic ParachainNetwork and TableRouter implementations
* add some module docs
* remove exit-future from validation
* hack: adapt legacy protocol to lack of exit-future
* generalize RegisteredMessageValidator somewhat
* instantiate and teardown table routers
* clean up RouterInner drop logic
* implement most of the statement import loop
* implement statement loop in async/await
* remove unneeded TODO
* most of the collation skeleton
* send session keys and validator roles
* also send role after status
* use config in startup
* point TODO to issue
* fix test compilation
* Be specific about the `BitVec` generic arguments
Currently we use the default generic arguments for `BitVec`. This means
we use `BigEndian` and `u8`. These default values are not stable, with
`0.17` of the `BitVec` crate this changes. To make sure we don't break
anything in the future, make sure we explictly set the generics.
* Update `spec_version`
* runtime: candidate receipt must pass parent head
* construct parachain candidates using correct parent_head
* validate that the parent header is correct in candidate receipt
* fix test fallout
* bump runtime versions
* add some more docs about statement import
* instantiate environment async
* move attestation service into subfolder
* refactor validation service architecture somewhat
* remove dependence on validation service in proposer
* fix a bunch of warnings
* improve docs
* introduce a builder for the validation service
* extract block production to its own file
* integrate new API into service
* address review grumbles
* point to in-progress Substrate branch
* instantiate environment async
* Fix futures
* Bump runtime
* Fix collation tests
* point to polkadot-master again
* point to polkadot-master again
* update deps
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Bernhard Schuster