* Remove incorrect proof about Jemalloc
The truth is that Jemalloc is not always the default allocator. This is
only true for the polkadot binary.
* Fmt
* Rephrase
* Substrate Companion for rust 1.54
* CI: run with a staging CI image
* Revert "CI: run with a staging CI image"
This reverts commit f3b51da0a4c0ffa43b576af5482a2be256948870.
* CI: debug, again
* compilation fixes
* tabify
* update Substrate
* Update .gitlab-ci.yml
* CI: add ubuntu-based image with npm
Co-authored-by: Denis P <denis.pisarev@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: parity-processbot <>
* Ensure that `para_head` hash matches the actual head
This ensures that the `para_head` hash in the `CandidateDescriptor`
matches the actual `head` in the candidate commitments.
* Enable the code for `no_std`
* Formatting
* Return `relay_parent` as result of collation seconded signal as well
Before we only returned the seconded statement. However, to verify the
statement in a future proof way, we also need the relay parent that was
used as a context to sign the statement.
* FMT
* Change pallet naming in test-runtime
This is required to make the `well_know_keys::ACTIVE_CONFIG` match the
`ActiveConfig` key.
* Use correct name
* 🤦
* Only send one collation per relay parent at a time to validators
This changes the way we are sending collations to validators. Before we
answered every collation request immediatley. Now we only answer one
pov request at a time per relay parent. This should bring down the
bandwidth requirements and should help parachains to include bigger
blocks more easily.
* Guide updates
* Review feedback.
* Use wasm-builder from git
This brings new features like compressed runtimes out of the box.
* chore: update wasm builder dep. from 3.0.0 to master
* Fix tests
* Update node/core/pvf/tests/it/main.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: chevdor <chevdor@gmail.com>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Only fetch one collation at a time per relay parent
Before a validator would fetch all collations that were advertised to
him. This pr changes the behavior to always just fetch one collation at
a time. If fetching fails, the validator will start fetching one of the
other collations.
* Use enum to be more explicit
* Review comments
* polkadot-service: Make native runtime configurable
This pull requests adds support for configuring the native runtimes used
by polkadot-service. While this whole pr doesn't change that much for
polkadot, besides not having the light-node enabled for the default
polkadot binary. However, downstream projects (parachains) will have a
much better compile time. In cumulus for example the `cargo test --all
--release` is about 4m faster to compile.
* Fixes
* Fix
* Enable rococo-native
* Fix light client
* 🤦
* Fixes
* Introduce polkadot code substitute
This introduces a code substitute for the on-chain wasm of Polkadot from
block
`0x86aa36a140dfc449c30dbce16ce0fea33d5c3786766baa764e33f336841b9e29`
on wards. The underlying problem was that there exists some
miscompilation by the rust compiler in the wasm runtime that results in
state mismatches between the native/wasm runtime of the
the 0.8.30 release. This resulted in blocks being authored by the native
runtime not being importable by nodes using the wasm runtime. The
on-chain wasm is replaced by a wasm build from the 0.8.30 using the
rustc nightly from 1.03.2021.
* update the wasm substitute blob
* update Substrate
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: parity-processbot <>
Currently we will take all leaves and give that to the overseer on
startup, but this is a bad idea when the finality is lagging for
example. There can be many of unfinalized leaves, we don't even need to
look at anymore. To solve this, the pr adds a maximum of 4 leaves we
forward to the overseer and the pr also checks that we only pass uncles
of the best block.
* Lol
* Add the features
* Remove some more runtime logger init calls
* Make companion check work
* Revert "Make companion check work"
This reverts commit a255c798076466c0fa20a4db713fc712772c2b4d.
* Update Substrate
Currently the first last timestamp is `0` and that leads to the first
slot being `0` which is reserved for genesis. There is some debug assert
in BABE that complains about this in Cumulus :D
* Track the collation status in the collator protocol
This pr changes the collator protocol to track the status of a
collation. This is mainly used to log the status of a collation when it
is removed to inform the user if a collation maybe never reached a validator.
* Update node/network/collator-protocol/src/collator_side.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Moare logging
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
As futures most of the time are catching panics and we don't check this,
it could happen that we have some statement that isn't correct but the
test succeeds successfully.
* Notify collators about seconded collation
This pr adds functionality to inform a collator that its collation was
seconded by a parachain validator. Before this signed statement was only
gossiped over the validation substream. Now, we explicitly send the
seconded statement to the collator after it was validated successfully.
Besides that it changes the `CollatorFn` to return an optional result
sender that is informed when the build collation was seconded by a
parachain validator.
* Add test
* Make sure we only send `Seconded` statements
* Make sure we only receive valid statements
* Review feedback
We only checked all validity votes, ignoring invalid votes. So, the
condidition could not hold. Besides fixing the panic, I removed some old
cruft that isn't required anymore.
* Compress the PoV block before sending it over the network
This pr changes the way we send PoV blocks over the network. We now
compress the PoV block before it is send over the network. This should
reduce the size significant for PoVs which contain the runtime WASM for
example.
* Preallocate 1KB
* Try something..
* Switch to zstd and some renamings
* Make compression/decompression fail in browsers
* Use some sane maximum value
* Update roadmap/implementers-guide/src/types/network.md
Co-authored-by: Andronik Ordian <write@reusable.software>
* Fix and add test
* add
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Improve logging to make debugging parachains easier
This pr should make debugging parachains easier, by printing more
information about the validation process.
* 🤦
* moare
* Convert to debug
* Do not send empty view updates to peers
It happened that we send empty view updates to our peers, because we
only updated our finalized block. This could lead to situations where we
overwhelmed sub systems with too many messages. On Rococo this lead to
constant restarts of our nodes, because some node apparently was
finalizing a lot of blocks.
To prevent this, the pr is doing the following:
1. If a peer sends us an empty view, we report this peer and decrease it
reputation.
2. We ensure that we only send a view update when the `heads` changed
and not only the `finalized_number`.
3. We do not send empty `ActiveLeavesUpdates` from the overseer, as this
makes no sense to send these empty updates. If some subsystem is relying
on the finalized block, it needs to listen for the overseer signal.
* Update node/network/bridge/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Don't work if they're are no added heads
* Fix test
* Ahhh
* More fixes
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* Add one Jaeger span per relay parent
This adds one Jaeger span per relay parent, instead of always creating
new spans per relay parent. This should improve the UI view, because
subsystems are now grouped below one common span.
* Fix doc tests
* Replace `PerLeaveSpan` to `PerLeafSpan`
* More renaming
* Moare
* Update node/subsystem/src/lib.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Skip the spans
* Increase `spec_version`
Co-authored-by: Andronik Ordian <write@reusable.software>
This pr adds support to change the session length of a Rococo chain at
genesis. This is rather useful because Rococo has a session length of
1 hour, while on rococo-local you will now get 1 minute. This improves
the dev experience, because a parachain is only going live at the
start of a new session.
* Fix bug and further optimizations in availability distribution
- There was a bug that resulted in only getting one candidate per block
as the candidates were put into the hashmap with the relay block hash as
key. The solution for this is to use the candidate hash and the relay
block hash as key.
- We stored received/sent messages with the candidate hash and chunk
index as key. The candidate hash wasn't required in this case, as the
messages are already stored per candidate.
* Update node/core/bitfield-signing/src/lib.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Remove the reverse map
* major refactor of receipts & query_live
* finish refactoring
remove ancestory mapping,
improve relay-parent cleanup & receipts-cache cleanup,
add descriptor to `PerCandidate`
* rename and rewrite query_pending_availability
* add a bunch of consistency tests
* Add some last changes
* xy
* fz
* Make it compile again
* Fix one test
* Fix logging
* Remove some buggy code
* Make tests work again
* Move stuff around
* Remove dbg
* Remove state from test_harness
* More refactor and new test
* New test and fixes
* Move metric
* Remove "duplicated code"
* Fix tests
* New test
* Change break to continue
* Update node/core/av-store/src/lib.rs
* Update node/core/av-store/src/lib.rs
* Update node/core/bitfield-signing/src/lib.rs
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
* update guide to match live_candidates changes
* add comment
* fix bitfield signing
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
* Adds consistency checks for the `HostConfiguration`
Besides that it fixes the chain specs to make the consistency checks happy.
* Update runtime/parachains/src/configuration.rs
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Review feedback and test fixes etc
* Update
* More
* I'm an idiot
* Fix tests...
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Do not spam when we can not send a message to a job
There are legal reasons why a job ended. If a job failed, the error is
logged. So, we don't need to log an error when we can not send a message
to a job.
* Review feedback
* Rework `ConnectionsRequests`
Instead of implementing the `Stream` trait, this struct now provides a
function `next()`. This enables us to encode into the type system that
it will always return a value or block indefinitely.
* Review feedback
* Add an upper number of maximum parallel runtime api requests
Instead of spawning all runtime api requests in the background and using
all wasm instances. This pr adds a maximum number of parallel requests.
* Update node/core/runtime-api/src/lib.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Review feedback
* Increase instances
* Add warning
* Update node/core/runtime-api/src/lib.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Make it possible for the adder collator to calculate any state
This is very useful for when wanting to have multiple running or when
wanting to restart the collator.
* Update parachain/test-parachains/adder/collator/src/lib.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Simplify subsystem jobs
This pr simplifies the subsystem jobs interface. Instead of requiring an
extra message that is used to signal that a job should be ended, a job
now ends when the receiver returns `None`. Besides that it changes the
interface to enforce that messages to a job provide a relay parent.
* Drop ToJobTrait
* Remove FromJob
We always convert this message to FromJobCommand anyway.
This pr changes how the runtime api subsystem processes runtime api
requests. Instead of answering all of them in the subsystem task and
thus, making all requests sequential, we now answer them in a background
task. This enables us to serve multiple requests at once.
* Some code cleanup in overseer
- Switches to select! in the overseer run loop to be more fair about
message processing between the different sources.
- Added a check to only send `ActiveLeaves` if the update actually
contains any data.
* Move the check
* Restore old behavior
* Simplify message sending and signal sending to subsystems
* Update node/subsystem/src/lib.rs
* Do not send messages twice in bitfield distribution
This removes a bug which resulted in sending bitfield messages multiple
times by not checking if we already relayed them. Besides that it also
adds an optimization to not relay a message to a peer that send us
this message.
* Review comments
* Break some lines
* Improve collator side of the collator-protocol
This pr improves the collator-protocol implementation of the collator
side. Besides cleaning up code and rewriting it, the following changed:
- Before on `PeerViewChange` we send an advertisment to every peer, now
this only happens for validators.
- It also adds a check that we send an advertisment message only once
for a connected peer.
- If the same validator was part of the current and next group, we
requested to be connected to this validator two times. This is also
fixed now.
- Instead of having only one connection request, we now are being able
to store multiple of them. This is required as we can have multiple
active leafs at any point of time.
* Switch to common `ConnectionRequests`
* Update node/network/collator-protocol/src/collator_side.rs
* Revert "Restrict collating_using_adder_collator test on feature real-overseer (#1947)"
This reverts commit 75da335d66.
* Revert "Revert "Restrict collating_using_adder_collator test on feature real-overseer (#1947)""
This reverts commit 4e92cdfb1046b0e3da7627297050ae71ae3dd6e3.
* Test something
* Adds integration test based on adder collator
This adds an integration test for parachains that uses the adder
collator. The test will start two relay chain nodes and one collator and
waits until 4 blocks are build and enacted by the parachain.
* Make sure the integration test is run in CI
* Fix wasm compilation
* Update parachain/test-parachains/adder/collator/src/lib.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
* Update cli/src/command.rs
Co-authored-by: Sergei Shulepov <sergei@parity.io>
We need to distribute the PoV after we have seconded it. Other nodes
that will receive our `Secondded` statement and want to validate the
candidate another time will request this PoV from us.
Bastian Köcher