* Improve logging to make debugging parachains easier
This pr should make debugging parachains easier, by printing more
information about the validation process.
* 🤦
* moare
* Convert to debug
* Add warning when dropping inclusion inherent
* improve warning when dropping inclusion inherent
- use the actual runtime `warn!` macro
- do not speculate about why the error occurred
- show the actual error returned
* BROKEN: add note if the current block matches the session start block
This doesn't compile for a variety of reasons; this commit demonstrates
what the issues are. I don't think this is an essential feature, so I'm
stopping work on it here instead of pushing through it.
* Revert "BROKEN: add note if the current block matches the session start block"
This reverts commit eeb79ab7708ec94224fa60c9e1a46e12c18d52fd.
* ensure a runtime logger exists before attempting to log a warning to it
* Adds message types
* Add code skeleton
* Adds subsystem code.
* Adds a first test
* Adds interaction result to availability_lru
* Use LruCache instead of a HashMap
* Whitespaces to tabs
* Do not ignore errors
* Change error type
* Add a timeout to chunk requests
* Add custom errors and log them
* Adds replace_availability_recovery method
* recovery_threshold computed by erasure crate
* change core to std
* adds docs to error type
* Adds a test for invalid reconstruction
* refactors interaction run into multiple methods
* Cleanup AwaitedChunks
* Even more fixes
* Test that recovery with wrong root is an error
* Break to launch another requests
* Styling fixes
* Add SessionIndex to API
* Proper relay parents for MakeRequest
* Remove validator_discovery and use message
* Remove a stream on exhaustion
* On cleanup free the request streams
* Fix merge and refactor
* More doc fixes.
* Minor refactorings in the process of #2177
By having everything peer set related depend directly on the enum the
code becomes more clear and it is also straight forward to add more
peersets/protocols as the compiler will complain if you forget to
implement parts of it.
* Add peer set infos on startup properly
For feature real_overseer.
+ Fixes from review. Thanks @coriolinus and @ordian!
* More structure in network-bridge
Some changes, which would have helped me in groking the code faster.
Entry points/public types more to the top. Factored out implementation
in their own files, to clear up the top-level view.
* Get rid of local ProtocolName type definition.
Does not add much at this level.
* Fix tests + import cleanup.
* Make spaces tabs.
* Clarify what correct parameters to send_message are
* Be more less vague in docs of send_message.
* Apply suggestions from code review
Extend copyright on new files to 2021 as well.
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Andronik Ordian <write@reusable.software>
* scheduler: handle re-scheduling around finalization correctly
* also make sure parathreads get cleaned
* run scheduling in finalization
* Remove stray println!
* Update the schedule call site in inclusion inherent
* Clarify subtlety around SessionStartBlock
* Remove double semi-colon
* reschedule prior to `availability_cores` and in on-initialize
* improve docs
* fix line
* more doc reformat
* remove unneeded call
* avoid unnecessary scheduling on initialize
* split `clear` and `schedule
* Update runtime/parachains/src/scheduler.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Drive by fixes
The visibility modifiers are remnants of the previous structure where
HRMP wasn't a standalone module, by rather a submodule of the router
module.
* Add Currency assoc type to Config
This would allow us to reserve balance for deposits. This commit also
integrates the HRMP module in rococo, test-runtime and mocks to use the
balances pallet.
* Fix a bug that doesn't increment the age
In case the request is not confirmed, the age would be incremented but
not persisted.
* Fix cleaning the indexes
Before that change, the cleaning of the channel indexes was wrong, because it
naively removed entire rows that was pertaining to the para we delete.
This approach is flawed because it doesn't account for the rows that are
pertaining to other paras that contain the outgoing one.
This clearly violates the invariant imposed on the indexes, that all
the index rows must contain alive paras, but apart from that it also
lead to the situation where ingress index would contain the a different
set of channels that an egress have.
* Reserve currency for opening the channels
Note the ugly `unique_saturated_into` calls. The reason for them is the
currency trait accepts and defines the `Balance` associated type and the
deposit values are coming from the `HostConfiguration` where they are
defined using the `Balance`.
I figured that parameterising `HostConfiguration` would be annoying. On
the other hand, I don't expect these `unique_saturated_into` calls to
give us problems since it seems to be a reasonable assumption that this
module will be instantiated within a runtime where the Currency provided
will have a Balance that matches the one used in the configuration.
* Tests: Adapt `run_to_block` so that it submits a proper config
* Tests: exercise the deposit logic
* CI: add rococo build and conteinerization
* CI: overhaul rules
* CI: two new conditional jobs to build rococo, CI overhaul, npm security fix
* CI: remove web-wasm publishing
* Rename crowdfund -> crowdloan
* allow contribution on behalf of another user
* starting some benchmarks
* optimization: use append api
* Use on_initialize instead of on_finalize
* More benchmarks
* try to implement partial child storage removal
* partial dissolve test
* onboard benchmark
* begin retirement
* on_initialize
* remove _ { }
* Revert "allow contribution on behalf of another user"
This reverts commit b7dd7d1ec751495cee3ddb57133a13c390b020e5.
* finish undo of "allow contribution on behalf of another user"
* Allow any user to trigger withdraw on closed crowdloan
* use transfer instead of withdraw/create pattern
* unused warning
* Update runtime/common/src/crowdloan.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* dont need to assign to empty variable
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* subsystems have an unbounded channel to the overseer
* Update node/overseer/src/lib.rs
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* bump Cargo.lock
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
* Do not send empty view updates to peers
It happened that we send empty view updates to our peers, because we
only updated our finalized block. This could lead to situations where we
overwhelmed sub systems with too many messages. On Rococo this lead to
constant restarts of our nodes, because some node apparently was
finalizing a lot of blocks.
To prevent this, the pr is doing the following:
1. If a peer sends us an empty view, we report this peer and decrease it
reputation.
2. We ensure that we only send a view update when the `heads` changed
and not only the `finalized_number`.
3. We do not send empty `ActiveLeavesUpdates` from the overseer, as this
makes no sense to send these empty updates. If some subsystem is relying
on the finalized block, it needs to listen for the overseer signal.
* Update node/network/bridge/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Don't work if they're are no added heads
* Fix test
* Ahhh
* More fixes
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Fuse receive stream in Context
* Revert "Fuse receive stream in Context"
This reverts commit ddd26fa98f0ca1afbc22064e93010e4193a058b2.
* Exit on node shutdown from av-store loop
* Filter only context error
* Store all chunks and in a single transaction
* Adds chunks LRU to store
* Add pruning records metrics
* Use honest cache instead of LRU
* Remove unnecessary optional cache
* Fix review nits that are fixable
* Companion PR for refactoring priority groups
* Fix non reserved node
* Try fix tests
* Missing import
* Fix warning
* Change protocols order
* Fix test
* Renames
* Update syn dependency to make it compile again after merging master
* "Update Substrate"
Co-authored-by: parity-processbot <>
* add timing setup to OverseerSubsystemContext
* figure out how to initialize the rng
* attach a timer to a portion of the messages traveling to the Overseer
This timer only exists / logs a fraction of the time (configurable
by `MESSAGE_TIMER_METRIC_CAPTURE_RATE`). When it exists, it tracks
the span between the `OverSubsystemContext` receiving the message
and its receipt in `Overseer::run`.
* propagate message timing to the start of route_message
This should be more accurate; it ensures that the timer runs
at least as long as that function. As `route_message` is async,
it may not actually run for some time after it is called (or ever).
* fix failing test
* rand_chacha apparently implicitly has getrandom feature
* change rng initialization
The previous impl using `from_entropy` depends on the `getrandom`
crate, which uses the system entropy source, and which does not
work on `wasm32-unknown-unknown` because it wants to fall back to
a JS implementation which we can't assume exists.
This impl depends only on `rand::thread_rng`, which has no documentation
stating that it's similarly limited.
* remove randomness in favor of a simpler 1 of N procedure
This deserves a bit of explanation, as the motivating issue explicitly
requested randomness. In short, it's hard to get randomness to compile
for `wasm32-unknown-unknown` because that is explicitly intended to be
as deterministic as practical. Additionally, even though it would never
be used for consensus purposes, it still felt offputting to intentionally
introduce randomness into a node's operations. Except, it wasn't really
random, either: it was a deterministic PRNG varying only in its state,
and getting the state to work right for that target would have required
initializing from a constant.
Given that it was a deterministic sequence anyway, it seemed much simpler
and more explicit to simply select one of each N messages instead of
attempting any kind of realistic randomness.
* reinstate randomness for better statistical properties
This partially reverts commit 0ab8594c328b3f9ce1f696fe405556d4000630e9.
`oorandom` is much lighter than the previous `rand`-based implementation,
which makes this easier to work with.
This implementation gives each subsystem and each child RNG a distinct
increment, which should ensure they produce distinct streams of values.
Bastian Köcher