* tests/av-store: use future::join instead of future::select
* tests/backing: use future::join instead of future::select
* tests/provisioner: use future::join instead of future::select
* tests/av-dist: use future::join instead of future::select
* tests/av-recovery: use future::join instead of future::select
* tests/bridge: use future::join instead of future::select
* tests/collator-protocol: use future::join instead of future::select
* tests/stmt-dist: use future::join instead of future::select
* fix tests
* guide: declare one para as a collator
* add ParaId to Declare messages and clean up
* fix build
* fix the testerinos
* begin adding keystore to collator-protocol
* remove request_x_ctx
* add core_for_group
* add bump_rotation
* add some more helpers to subsystem-util
* change signing_key API to take ref
* determine current and next para assignments
* disconnect collators who are not on current or next para
* add collator peer count metric
* notes for later
* some fixes
* add data & keystore to test state
* add a test utility for answering runtime API requests
* fix existing collator tests
* add new tests
* remove sc_keystore
* update cargo lock
Co-authored-by: Andronik Ordian <write@reusable.software>
* code stored in para + modify CandidateDescriptor.
* WIP: digest + some more impl
* validation_code_hash in payload + check in inclusion
* check in client + refator
* tests
* fix encoding indices
* remove old todos
* fix test
* fix test
* add test
* fetch validation code inside collation-generation from the relay-chain
* HashMismatch -> PoVHashMismatch + miscompilation
* refactor, store hash when needed
* storage rename: more specific but slightly too verbose
* do not hash on candidate validation, fetch hash instead
* better test
* fix test
* guide updates
* don't panic in runtime
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* extract database from av-store itself
* generalize approval-voting over database type
* modes (without handling) and pruning old wakeups
* rework approval importing
* add our_approval_sig to ApprovalEntry
* import assignment
* guide updates for check-full-approval changes
* some aux functions
* send messages when becoming active.
* guide: network bridge sends view updates only when done syncing
* network bridge: send view updates only when done syncing
* tests for new network-bridge behavior
* add a test for updating approval entry with sig
* fix some warnings
* test load-all-blocks
* instantiate new parachains DB
* fix network-bridge empty view updates
* tweak
* fix wasm build, i think
* Update node/core/approval-voting/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* add some versioning to parachains_db
* warnings
* fix merge changes
* remove versioning again
Co-authored-by: Andronik Ordian <write@reusable.software>
* dispute types
* add Debug to dispute primitives in std and InherentData
* use ParachainsInherentData on node-side
* change inclusion_inherent to paras_inherent
* RuntimeDebug
* add type parameter to PersistedValidationData users
* fix test client
* spaces
* fix collation-generation test
* fix provisioner tests
* remove references to inclusion inherent
* guide: don't request availability data from ourselves
* add QueryAllChunks message
* implement QueryAllChunks
* remove unused relay_parent from StoreChunk
* test QueryAllChunks
* fast paths make short roads
* test early exit behavior
* guide updates
* keep interactions alive until receivers drop
* retry indefinitely
* cancel approval tasks on finality
* use swap_remove instead of remove
* Remove stuff out of the runtime that does not belong there.
There might be more, but it is a start.
* White space fixes.
* Fix tests.
* Leave whitespace in ui tests alone.
* Add back zstd for no reason.
* Fix browser wasm (hopefully)
* overseer: pass messages directly between subsystems
* test that message is held on to
* Update node/overseer/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* give every subsystem an unbounded sender too
* remove metered_channel::name
1. we don't provide good names
2. these names are never used anywhere
* unused mut
* remove unnecessary &mut
* subsystem unbounded_send
* remove unused MaybeTimer
We have channel size metrics that serve the same purpose better now and the implementation of message timing was pretty ugly.
* remove comment
* split up senders and receivers
* update metrics
* fix tests
* fix test subsystem context
* use SubsystemSender in jobs system now
* refactor of awful jobs code
* expose public `run` on JobSubsystem
* update candidate backing to new jobs & use unbounded
* bitfield signing
* candidate-selection
* provisioner
* approval voting: send unbounded for assignment/approvals
* async not needed
* begin bridge split
* split up network tasks into background worker
* port over network bridge
* Update node/network/bridge/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* rename ValidationWorkerNotifications
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
* Indentation fix.
* Prepare request-response for PoV fetching.
* Drop old PoV distribution.
* WIP: Fetch PoV directly from backing.
* Backing compiles.
* Runtime access and connection management for PoV distribution.
* Get rid of seemingly dead code.
* Implement PoV fetching.
Backing does not yet use it.
* Don't send `ConnectToValidators` for empty list.
* Even better - no need to check over and over again.
* PoV fetching implemented.
+ Typechecks
+ Should work
Missing:
- Guide
- Tests
- Do fallback fetching in case fetching from seconding validator fails.
* Check PoV hash upon reception.
* Implement retry of PoV fetching in backing.
* Avoid pointless validation spawning.
* Add jaeger span to pov requesting.
* Add back tracing.
* Review remarks.
* Whitespace.
* Whitespace again.
* Cleanup + fix tests.
* Log to log target in overseer.
* Fix more tests.
* Don't fail if group cannot be found.
* Simple test for PoV fetcher.
* Handle missing group membership better.
* Add test for retry functionality.
* Fix flaky test.
* Spaces again.
* Guide updates.
* Spaces.
* add number to `ActivatedLeavesUpdate`
* update subsystem util and overseer
* use new ActivatedLeaf everywhere
* sort view
* sorted and limited view in network bridge
* use live block hash only if it's newer
* grumples
* Add a config field to babe epochs
* Fix test
* Add BABE_GENESIS_EPOCH_CONFIG consts
* Use PrimaryAndSecondaryVRFSlots and remove newlines
* Make epoch_configs Some
* Fix tests
* Fix test service tests
* Add a BabeEpochConfigMigrations OnRuntimeUpgrade
* Apply suggestions
* Use PrimaryAndSecondaryPlainSlots in kusama
* Remove migration from test runtime and rococo
* Add HasPalletPrefix
* Rename to BabePalletPrefix and change BabeApi -> Babe
* "Update Substrate"
* Update substrate
* Resolve parantheses errors
Co-authored-by: parity-processbot <>
* add a magic number to backing statements encoded
* fix fallout in statement table
* fix some fallout in backing
* add magic to approval votes
* remove last references to Candidate variant
* update size-hint
* WIP
* availability distribution, still very wip.
Work on the requesting side of things.
* Some docs on what I intend to do.
* Checkpoint of session cache implementation
as I will likely replace it with something smarter.
* More work, mostly on cache
and getting things to type check.
* Only derive MallocSizeOf and Debug for std.
* availability-distribution: Cache feature complete.
* Sketch out logic in `FetchTask` for actual fetching.
- Compile fixes.
- Cleanup.
* Format cleanup.
* More format fixes.
* Almost feature complete `fetch_task`.
Missing:
- Check for cancel
- Actual querying of peer ids.
* Finish FetchTask so far.
* Directly use AuthorityDiscoveryId in protocol and cache.
* Resolve `AuthorityDiscoveryId` on sending requests.
* Rework fetch_task
- also make it impossible to check the wrong chunk index.
- Export needed function in validator_discovery.
* From<u32> implementation for `ValidatorIndex`.
* Fixes and more integration work.
* Make session cache proper lru cache.
* Use proper lru cache.
* Requester finished.
* ProtocolState -> Requester
Also make sure to not fetch our own chunk.
* Cleanup + fixes.
* Remove unused functions
- FetchTask::is_finished
- SessionCache::fetch_session_info
* availability-distribution responding side.
* Cleanup + Fixes.
* More fixes.
* More fixes.
adder-collator is running!
* Some docs.
* Docs.
* Fix reporting of bad guys.
* Fix tests
* Make all tests compile.
* Fix test.
* Cleanup + get rid of some warnings.
* state -> requester
* Mostly doc fixes.
* Fix test suite.
* Get rid of now redundant message types.
* WIP
* Rob's review remarks.
* Fix test suite.
* core.relay_parent -> leaf for session request.
* Style fix.
* Decrease request timeout.
* Cleanup obsolete errors.
* Metrics + don't fail on non fatal errors.
* requester.rs -> requester/mod.rs
* Panic on invalid BadValidator report.
* Fix indentation.
* Use typed default timeout constant.
* Make channel size 0, as each sender gets one slot anyways.
* Fix incorrect metrics initialization.
* Fix build after merge.
* More fixes.
* Hopefully valid metrics names.
* Better metrics names.
* Some tests that already work.
* Slightly better docs.
* Some more tests.
* Fix network bridge test.
* add tracing to approval voting
* notify if session info is not working
* add dispute period to chain specs
* propagate genesis session to parachains runtime
* use `on_genesis_session`
* protect against zero cores in computation
* tweak voting rule to be based off of best and add logs
* genesis configuration should use VRF slots only
* swallow more keystore errors
* add some docs
* make validation-worker args non-optional and update clap
* better tracing for bitfield signing and provisioner
* pass amount of bits in bitfields to inclusion instead of recomputing
* debug -> warn for some logs
* better tracing for availability recovery
* a little av-store tracing
* bridge: forward availability recovery messages
* add missing try_from impl
* some more tracing
* improve approval distribution tracing
* guide: hold onto pending approval messages until NewBlocks
* Hold onto pending approval messages until NewBlocks
* guide: adjust comment
* process all actions for one wakeup at a time
* vec
* fix network bridge test
* replace randomness-collective-flip with Babe
* remove PairNotFound
* Use KeyValueDB in approval-voting
* use KVDB instead of AuxStore
* add rocksdb to cargo toml
* add a Config struct
* create new DB in service
* fix dep for regular node
* make optional
* post merge fix
Co-authored-by: Andronik Ordian <write@reusable.software>
* guide changes for a fast-path requesting from backing validators
* add backing group to availability recovery message
* add new phase to interaction
* typos
* add full data messages
* handle new network messages
* dispatch full data requests
* cleanup
* check chunk index
* test for invalid recovery
* tests
* Typos.
* fix some grumbles
* be more explicit about error handling and control flow
* fast-path param
* use with_chunks_only in Service
Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
* insta-approval for low-node testnets
* fix
* handle 0 needed_approvals and add some logs
* downgrade logs to debug, per block
* fix a warning
* more useful logs
* test
* finish test 🎉
* not so fast
* the test passes, but is it enough?
Robert Habermeier