* Simplify some Option / Result / ? operator patterns
When they identically match a combinator on those types.
Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
* adjust review comments
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* tests/av-store: use future::join instead of future::select
* tests/backing: use future::join instead of future::select
* tests/provisioner: use future::join instead of future::select
* tests/av-dist: use future::join instead of future::select
* tests/av-recovery: use future::join instead of future::select
* tests/bridge: use future::join instead of future::select
* tests/collator-protocol: use future::join instead of future::select
* tests/stmt-dist: use future::join instead of future::select
* fix tests
* guide: declare one para as a collator
* add ParaId to Declare messages and clean up
* fix build
* fix the testerinos
* begin adding keystore to collator-protocol
* remove request_x_ctx
* add core_for_group
* add bump_rotation
* add some more helpers to subsystem-util
* change signing_key API to take ref
* determine current and next para assignments
* disconnect collators who are not on current or next para
* add collator peer count metric
* notes for later
* some fixes
* add data & keystore to test state
* add a test utility for answering runtime API requests
* fix existing collator tests
* add new tests
* remove sc_keystore
* update cargo lock
Co-authored-by: Andronik Ordian <write@reusable.software>
* guide: don't request availability data from ourselves
* add QueryAllChunks message
* implement QueryAllChunks
* remove unused relay_parent from StoreChunk
* test QueryAllChunks
* fast paths make short roads
* test early exit behavior
* guide updates
* keep interactions alive until receivers drop
* retry indefinitely
* cancel approval tasks on finality
* use swap_remove instead of remove
* Remove stuff out of the runtime that does not belong there.
There might be more, but it is a start.
* White space fixes.
* Fix tests.
* Leave whitespace in ui tests alone.
* Add back zstd for no reason.
* Fix browser wasm (hopefully)
* add number to `ActivatedLeavesUpdate`
* update subsystem util and overseer
* use new ActivatedLeaf everywhere
* sort view
* sorted and limited view in network bridge
* use live block hash only if it's newer
* grumples
* WIP
* availability distribution, still very wip.
Work on the requesting side of things.
* Some docs on what I intend to do.
* Checkpoint of session cache implementation
as I will likely replace it with something smarter.
* More work, mostly on cache
and getting things to type check.
* Only derive MallocSizeOf and Debug for std.
* availability-distribution: Cache feature complete.
* Sketch out logic in `FetchTask` for actual fetching.
- Compile fixes.
- Cleanup.
* Format cleanup.
* More format fixes.
* Almost feature complete `fetch_task`.
Missing:
- Check for cancel
- Actual querying of peer ids.
* Finish FetchTask so far.
* Directly use AuthorityDiscoveryId in protocol and cache.
* Resolve `AuthorityDiscoveryId` on sending requests.
* Rework fetch_task
- also make it impossible to check the wrong chunk index.
- Export needed function in validator_discovery.
* From<u32> implementation for `ValidatorIndex`.
* Fixes and more integration work.
* Make session cache proper lru cache.
* Use proper lru cache.
* Requester finished.
* ProtocolState -> Requester
Also make sure to not fetch our own chunk.
* Cleanup + fixes.
* Remove unused functions
- FetchTask::is_finished
- SessionCache::fetch_session_info
* availability-distribution responding side.
* Cleanup + Fixes.
* More fixes.
* More fixes.
adder-collator is running!
* Some docs.
* Docs.
* Fix reporting of bad guys.
* Fix tests
* Make all tests compile.
* Fix test.
* Cleanup + get rid of some warnings.
* state -> requester
* Mostly doc fixes.
* Fix test suite.
* Get rid of now redundant message types.
* WIP
* Rob's review remarks.
* Fix test suite.
* core.relay_parent -> leaf for session request.
* Style fix.
* Decrease request timeout.
* Cleanup obsolete errors.
* Metrics + don't fail on non fatal errors.
* requester.rs -> requester/mod.rs
* Panic on invalid BadValidator report.
* Fix indentation.
* Use typed default timeout constant.
* Make channel size 0, as each sender gets one slot anyways.
* Fix incorrect metrics initialization.
* Fix build after merge.
* More fixes.
* Hopefully valid metrics names.
* Better metrics names.
* Some tests that already work.
* Slightly better docs.
* Some more tests.
* Fix network bridge test.
* add tracing to approval voting
* notify if session info is not working
* add dispute period to chain specs
* propagate genesis session to parachains runtime
* use `on_genesis_session`
* protect against zero cores in computation
* tweak voting rule to be based off of best and add logs
* genesis configuration should use VRF slots only
* swallow more keystore errors
* add some docs
* make validation-worker args non-optional and update clap
* better tracing for bitfield signing and provisioner
* pass amount of bits in bitfields to inclusion instead of recomputing
* debug -> warn for some logs
* better tracing for availability recovery
* a little av-store tracing
* bridge: forward availability recovery messages
* add missing try_from impl
* some more tracing
* improve approval distribution tracing
* guide: hold onto pending approval messages until NewBlocks
* Hold onto pending approval messages until NewBlocks
* guide: adjust comment
* process all actions for one wakeup at a time
* vec
* fix network bridge test
* replace randomness-collective-flip with Babe
* remove PairNotFound
* guide changes for a fast-path requesting from backing validators
* add backing group to availability recovery message
* add new phase to interaction
* typos
* add full data messages
* handle new network messages
* dispatch full data requests
* cleanup
* check chunk index
* test for invalid recovery
* tests
* Typos.
* fix some grumbles
* be more explicit about error handling and control flow
* fast-path param
* use with_chunks_only in Service
Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
* refactor/reputation: unify the values used
* chore/rep: rename Annoy* to Cost*, make duplicate message Cost*Repeated
* fix/reputation: lost and found, convert at the boundary to substrate
* refactor/rep: move conversion to base reputation one level down, left conversions
* fix/rep: order of magnitude adjustments
Thanks pierre!
* remove spaces
* chore/rep: give rationale for order of magnitude
* refactor/rep: move UnifiedReputationChange to separate file
* fix/rep: order of magnitudes correction
* validator_discovery: pass PeerSet to the request
* validator_discovery: track PeerSet of connected peers
* validator_discovery: fix tests
* validator_discovery: fix long line
* some fixes
* some validator_discovery logs
* log validator discovery request
* Also connect to validators on `DistributePoV`.
* validator_discovery: store the whole state per peer_set
* bump spec versions in kusama, polkadot and westend
* Correcting doc.
* validator_discovery: bump channel capacity
* pov-distribution: some cleanup
* this should fix the test, but it does not
* I just got some brain damage while fixing this
Why are you even reading this???
* wrap long line
* address some review nits
Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
* Move NetworkBridgeEvent to subsystem::messages.
It is not protocol related at all, it is in fact only part of the
subsystem communication as it gets wrapped into messages of each
subsystem.
* Request/response infrastructure is taking shape.
WIP: Does not compile.
* Multiplexer variant not supported by Rusts type system.
* request_response::request type checks.
* Cleanup.
* Minor fixes for request_response.
* Implement request sending + move multiplexer.
Request multiplexer is moved to bridge, as there the implementation is
more straight forward as we can specialize on `AllMessages` for the
multiplexing target.
Sending of requests is mostly complete, apart from a few `From`
instances. Receiving is also almost done, initializtion needs to be
fixed and the multiplexer needs to be invoked.
* Remove obsolete multiplexer.
* Initialize bridge with multiplexer.
* Finish generic request sending/receiving.
Subsystems are now able to receive and send requests and responses via
the overseer.
* Doc update.
* Fixes.
* Link issue for not yet implemented code.
* Fixes suggested by @ordian - thanks!
- start encoding at 0
- don't crash on zero protocols
- don't panic on not yet implemented request handling
* Update node/network/protocol/src/request_response/v1.rs
Use index 0 instead of 1.
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update node/network/protocol/src/request_response.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Fix existing tests.
* Better avoidance of division by zoro errors.
* Doc fixes.
* send_request -> start_request.
* Fix missing renamings.
* Update substrate.
* Pass TryConnect instead of true.
* Actually import `IfDisconnected`.
* Fix wrong import.
* Update node/network/bridge/src/lib.rs
typo
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update node/network/bridge/src/multiplexer.rs
Remove redundant import.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Stop doing tracing from within `From` instance.
Thanks for the catch @tomaka!
* Get rid of redundant import.
* Formatting cleanup.
* Fix tests.
* Add link to issue.
* Clarify comments some more.
* Fix tests.
* Formatting fix.
* tabs
* Fix link
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Use map_err.
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Improvements inspired by suggestions by @drahnr.
- Channel size is now determined by function.
- Explicitely scope NetworkService::start_request.
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* PVD: `block_number`->`relay_parent_number`
* ValidationParams: `relay_chain_height`->`relay_parent_number`
* Expose DMQ MQC hash as a well-known-key
This way the relay storage merkle proofs will be able to obtain the DMQ
MQC hash and we will be able to remove the it from the
PersistedValidationData struct.
* PersistedValidationData: Remove HRMP MQC heads
* PersistedValidationData: Remove `dmq_mqc_head`
* Expose the HRMP ingress channel index as a well-known-key
This way a parachain (PVF and collator) can find all the parachains that
have an outbound channel to the given one. That allows in turn to find
all the inbound channels for the given para.
Having access to that allows the parachain to get the same information
as the hrmp_mqc_heads now provide.
* Rename `relay_storage_root` to `relay_parent_storage_root`
* Adds message types
* Add code skeleton
* Adds subsystem code.
* Adds a first test
* Adds interaction result to availability_lru
* Use LruCache instead of a HashMap
* Whitespaces to tabs
* Do not ignore errors
* Change error type
* Add a timeout to chunk requests
* Add custom errors and log them
* Adds replace_availability_recovery method
* recovery_threshold computed by erasure crate
* change core to std
* adds docs to error type
* Adds a test for invalid reconstruction
* refactors interaction run into multiple methods
* Cleanup AwaitedChunks
* Even more fixes
* Test that recovery with wrong root is an error
* Break to launch another requests
* Styling fixes
* Add SessionIndex to API
* Proper relay parents for MakeRequest
* Remove validator_discovery and use message
* Remove a stream on exhaustion
* On cleanup free the request streams
* Fix merge and refactor
François Garillot