* Don't wait for dispute coordinator
in backing and approval-voting - we are single threaded there, so this
is blocking everything.
* Add missing import.
* Don't warn on dropped receiver.
* remove v0 primitives from polkadot-primitives
* first pass: remove v0
* fix fallout in erasure-coding
* remove v1 primitives, consolidate to v2
* the great import update
* update runtime_api_impl_v1 to v2 as well
* guide: add `Version` request for runtime API
* add version query to runtime API
* reintroduce OldV1SessionInfo in a limited way
* First step in implementing https://github.com/paritytech/polkadot/issues/4386
This PR:
- Reduces MAX_UNSHARED_UPLOAD_TIME to 150ms
- Increases timeout on collation fetching to 1200ms
- Reduces limit on needed backing votes in the runtime
This PR does not yet reduce the number of needed backing votes on the
node as this can only be meaningfully enacted once the changed limit in
the runtime is live.
* Fix tests.
* Guide updates.
* Review remarks.
* Bump minimum required backing votes to 2 in runtime.
* Make sure node side code won't make runtime vomit.
* cargo +nightly fmt
* Companion PR for removing Prometheus metrics prefix
* Was missing some metrics
* Fix missing renames
* Fix test
* Fixes
* Update test
* Update Substrate
* Second time
* remove prefix from intergration test for zombienet
* update zombienet image
* Update Substrate
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
* remove Default from CandidateHash
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* chore: fmt
* remove backed candidate default
* Partial migration away from CandidateReceipt::default
* Remove more CandidateReceipt defaults
* fmt
* Mostly remove CommittedCandidateReceipt default usage
* Remove CommittedCandidateReceipt
* Remove more Defaults from polakdot primitives v1 + fmt
* Remove more Default from polkadot primites v1
* WIP trying to get overseer example + tests to compile
* feat: add primitives test helpers
* reduce deps of helper
* update primitive helpers
* make candidate validation compile
* fixup cargo lock
* make av-store compile
* fixup disputes coordinator tests
* test: fixup backing
* test: fixup approval voting
* fixup bitfield signing
* test: fixup runtime-api
* test: fixup availability dist
* foxi[ pverseer test]
* remove some Defaults, remove bounds from `dummy`
All `fn dummy` in primitives need to be removed anyways.
This aids in the transition.
* it's a test helper, so always use std
* test: fixup parachains runtime tests
Excluding benches.
* fix keyring
* fix paras runtime properly, no more default
* Remove fn dummy() usage from approval voting
* Move TestCandidateBuilder out of av store to test helpers
* Make candidate validation tests pass
* Make most dispute coirdinator tests pass
* Make provisioner tests work
* Make availability recovery tests work with test helpers
* Update polkadot-collator-protocol tests
* Update statement distribution tests
* Update polkadot overseer examples and tests
* Derive default for validation code so we don't break unrelated things
* Make para runtime test pass (no bench)
* Some more work
* chore: cargo fmt
* cargo fix
* avoid some Default::default
* fixup dispute coordinator test
* remove unused crate deps
* remove Default::default wherever possible, replace by dummy_* for the most part
* chore: cargo fmt
* Remove some warnings
* Remove CommittedCandidateReceipt dummy
* Remove CandidateReceipt dummy
* Remove CandidateDescriptor dummy
* Remove commented out code
* Fix para runtime tests
* chore: nightly
* Some updates to the builder
* Dynamically adjust mock head data size
* Make dispute cooridinator tests work
* Fix test candidate_backing_reorders_votes work
* +nightly-2021-10-29 fmt
* Spelling and remove a default use in builder
* Various clean up
* More small updates
* fmt
* More small updates
* Doc comments for test helpers
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
* Update lib.rs
* review comments
* fix warnings
* fix test by using correct candidate receipt relay parent
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>
* pvf: make execution timeout configurable
* guide: add timeouts to candidate validation params
* add timeouts to candidate validation messages
* fmt
* port backing to use the backing pvf timeout
* port approval-voting to use the execution timeout
* port dispute participation to use the correct timeout
* fmt
* address grumbles & test failure
* CI: add spellcheck
* revert me
* CI: explicit command for spellchecker
* spellcheck: edit misspells
* CI: run spellcheck on diff
* spellcheck: edits
* spellcheck: edit misspells
* spellcheck: add rules
* spellcheck: mv configs
* spellcheck: more edits
* spellcheck: chore
* spellcheck: one more thing
* spellcheck: and another one
* spellcheck: seems like it doesn't get to an end
* spellcheck: new words after rebase
* spellcheck: new words appearing out of nowhere
* chore
* review edits
* more review edits
* more edits
* wonky behavior
* wonky behavior 2
* wonky behavior 3
* change git behavior
* spellcheck: another bunch of new edits
* spellcheck: new words are koming out of nowhere
* CI: finding the master
* CI: fetching master implicitly
* CI: undebug
* new errors
* a bunch of new edits
* and some more
* Update node/core/approval-voting/src/approval_db/v1/mod.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update xcm/xcm-executor/src/assets.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* Suggestions from the code review
* CI: scan only changed files
Co-authored-by: Andronik Ordian <write@reusable.software>
* Create validator_side module
* Subsume Candidate Selection
* Add test to ensure candidate backing logic is correct
* Ensure secondings are adequately cleaned up and address test flakyness
* Address Feedback
* guide: reversion safety
* guide: manage reversion safety in subsystems
* add leaf status to ActivatedLeaf
* add an LRU-cache to overseer for staleness detection
* update ActivatedLeaf usages in tests to contain status field
* add variant where missed accidentally
* add some helpers to LeafStatus
* address grumbles
* Remove signature verification in backing.
`SignedFullStatement` now signals that the signature has already been
checked.
* Remove unused check_payload function.
* Introduced unchecked signed variants.
* Fix inclusion to use unchecked variant.
* More unchecked variants.
* Use unchecked variants in protocols.
* Start fixing statement-distribution.
* Fixup statement distribution.
* Fix inclusion.
* Fix warning.
* Fix backing properly.
* Fix bitfield distribution.
* Make crypto store optional for `RuntimeInfo`.
* Factor out utility functions.
* get_group_rotation_info
* WIP: Collator cleanup + check signatures.
* Convenience signature checking functions.
* Check signature on collator-side.
* Fix warnings.
* Fix collator side tests.
* Get rid of warnings.
* Better Signed/UncheckedSigned implementation.
Also get rid of Encode/Decode for Signed! *party*
* Get rid of dead code.
* Move Signed in its own module.
* into_checked -> try_into_checked
* Fix merge.
with the MaxValidators parameter set, this meant that only validators participating in parachain consensus would be able to author relay-chain blocks containing backed candidates
* tests/av-store: use future::join instead of future::select
* tests/backing: use future::join instead of future::select
* tests/provisioner: use future::join instead of future::select
* tests/av-dist: use future::join instead of future::select
* tests/av-recovery: use future::join instead of future::select
* tests/bridge: use future::join instead of future::select
* tests/collator-protocol: use future::join instead of future::select
* tests/stmt-dist: use future::join instead of future::select
* fix tests
* Remove stuff out of the runtime that does not belong there.
There might be more, but it is a start.
* White space fixes.
* Fix tests.
* Leave whitespace in ui tests alone.
* Add back zstd for no reason.
* Fix browser wasm (hopefully)
* overseer: pass messages directly between subsystems
* test that message is held on to
* Update node/overseer/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* give every subsystem an unbounded sender too
* remove metered_channel::name
1. we don't provide good names
2. these names are never used anywhere
* unused mut
* remove unnecessary &mut
* subsystem unbounded_send
* remove unused MaybeTimer
We have channel size metrics that serve the same purpose better now and the implementation of message timing was pretty ugly.
* remove comment
* split up senders and receivers
* update metrics
* fix tests
* fix test subsystem context
* use SubsystemSender in jobs system now
* refactor of awful jobs code
* expose public `run` on JobSubsystem
* update candidate backing to new jobs & use unbounded
* bitfield signing
* candidate-selection
* provisioner
* approval voting: send unbounded for assignment/approvals
* async not needed
* begin bridge split
* split up network tasks into background worker
* port over network bridge
* Update node/network/bridge/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* rename ValidationWorkerNotifications
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
* Indentation fix.
* Prepare request-response for PoV fetching.
* Drop old PoV distribution.
* WIP: Fetch PoV directly from backing.
* Backing compiles.
* Runtime access and connection management for PoV distribution.
* Get rid of seemingly dead code.
* Implement PoV fetching.
Backing does not yet use it.
* Don't send `ConnectToValidators` for empty list.
* Even better - no need to check over and over again.
* PoV fetching implemented.
+ Typechecks
+ Should work
Missing:
- Guide
- Tests
- Do fallback fetching in case fetching from seconding validator fails.
* Check PoV hash upon reception.
* Implement retry of PoV fetching in backing.
* Avoid pointless validation spawning.
* Add jaeger span to pov requesting.
* Add back tracing.
* Review remarks.
* Whitespace.
* Whitespace again.
* Cleanup + fix tests.
* Log to log target in overseer.
* Fix more tests.
* Don't fail if group cannot be found.
* Simple test for PoV fetcher.
* Handle missing group membership better.
* Add test for retry functionality.
* Fix flaky test.
* Spaces again.
* Guide updates.
* Spaces.
* add number to `ActivatedLeavesUpdate`
* update subsystem util and overseer
* use new ActivatedLeaf everywhere
* sort view
* sorted and limited view in network bridge
* use live block hash only if it's newer
* grumples
* add a magic number to backing statements encoded
* fix fallout in statement table
* fix some fallout in backing
* add magic to approval votes
* remove last references to Candidate variant
* update size-hint
* WIP
* availability distribution, still very wip.
Work on the requesting side of things.
* Some docs on what I intend to do.
* Checkpoint of session cache implementation
as I will likely replace it with something smarter.
* More work, mostly on cache
and getting things to type check.
* Only derive MallocSizeOf and Debug for std.
* availability-distribution: Cache feature complete.
* Sketch out logic in `FetchTask` for actual fetching.
- Compile fixes.
- Cleanup.
* Format cleanup.
* More format fixes.
* Almost feature complete `fetch_task`.
Missing:
- Check for cancel
- Actual querying of peer ids.
* Finish FetchTask so far.
* Directly use AuthorityDiscoveryId in protocol and cache.
* Resolve `AuthorityDiscoveryId` on sending requests.
* Rework fetch_task
- also make it impossible to check the wrong chunk index.
- Export needed function in validator_discovery.
* From<u32> implementation for `ValidatorIndex`.
* Fixes and more integration work.
* Make session cache proper lru cache.
* Use proper lru cache.
* Requester finished.
* ProtocolState -> Requester
Also make sure to not fetch our own chunk.
* Cleanup + fixes.
* Remove unused functions
- FetchTask::is_finished
- SessionCache::fetch_session_info
* availability-distribution responding side.
* Cleanup + Fixes.
* More fixes.
* More fixes.
adder-collator is running!
* Some docs.
* Docs.
* Fix reporting of bad guys.
* Fix tests
* Make all tests compile.
* Fix test.
* Cleanup + get rid of some warnings.
* state -> requester
* Mostly doc fixes.
* Fix test suite.
* Get rid of now redundant message types.
* WIP
* Rob's review remarks.
* Fix test suite.
* core.relay_parent -> leaf for session request.
* Style fix.
* Decrease request timeout.
* Cleanup obsolete errors.
* Metrics + don't fail on non fatal errors.
* requester.rs -> requester/mod.rs
* Panic on invalid BadValidator report.
* Fix indentation.
* Use typed default timeout constant.
* Make channel size 0, as each sender gets one slot anyways.
* Fix incorrect metrics initialization.
* Fix build after merge.
* More fixes.
* Hopefully valid metrics names.
* Better metrics names.
* Some tests that already work.
* Slightly better docs.
* Some more tests.
* Fix network bridge test.
* Notify collators about seconded collation
This pr adds functionality to inform a collator that its collation was
seconded by a parachain validator. Before this signed statement was only
gossiped over the validation substream. Now, we explicitly send the
seconded statement to the collator after it was validated successfully.
Besides that it changes the `CollatorFn` to return an optional result
sender that is informed when the build collation was seconded by a
parachain validator.
* Add test
* Make sure we only send `Seconded` statements
* Make sure we only receive valid statements
* Review feedback
Robert Klotzner