Commit Graph

15873 Commits

Author SHA1 Message Date
Bernhard Schuster 49c6aa9a76 feat/jaeger: more spans, more stages (#2477)
* feat/jaeger: more spans, more stages

Stage numbers are still arbitrarily picked.

* feat/jaeger: additional spans

* chore/spellcheck: improve the dictionary

* fix/jaeger JaegerSpan -> jaeger::Span
2021-02-19 14:19:43 +00:00
Bastian Köcher 20527aec0e Fix warning in rustdoc job (#8159)
* Fix warning in rustdoc job

* More fixes

* Remove `build-rust-doc` job

Remove this job until upstream is fixed:
https://github.com/rust-lang/rust/issues/82284

* CI: temp. remove of the publishing job, no use of it w/o build

Co-authored-by: Denis P <denis.pisarev@parity.io>
2021-02-19 15:04:23 +01:00
Shawn Tabrizi a5defa7c7f Session Delayed Para Changes / Actions Queue (#2406)
* initial implementation of lifecycles and upgrades

* clean up a bit

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* do not include transitioning paras in identifier

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* collapse onboarding state

* starting actions queue

* consolidate actions queue

* schedule para initialize result

* more actions queue for upgrade/downgrade

* clean up with fully implemented actions queue

* fix tests

* fix scheduler tests

* fix hrmp tests

* fix test

* doc fixes

* fix hrmp test w/ valid para

* Update paras.md

* fix paras registrar

* Update propose_parachain.rs

* fix merge

* Introduce "shared" module

* fix rococo build

* fix up and use shared

* guide updates

* add shared config to common tests

* add shared to test-runtime

* remove println

* fix note

Co-authored-by: Gavin Wood <gavin@parity.io>
2021-02-18 21:20:18 -06:00
Bastian Köcher cbe35b9bd4 Remove OnSlot associated type (#8156)
Currently we always use a boxed future everywhere anyway. This also
enables us to use a boxed `SlotWorker` (which is required for Cumulus).
2021-02-19 00:15:26 +01:00
Joshy Orndorff 2c771e2cae Minor fixes and clarifications (#333) 2021-02-18 18:55:43 +01:00
Pierre Krieger aa95196be3 Extract transactions handling from protocol.rs (#8110)
* Extract transactions handling from protocol.rs

* Oops, boolean

* Do this better

* Update client/network/src/transactions.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* [WIP] Fix handshake

* Finish handshake change

* Bugfix

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2021-02-18 17:04:23 +01:00
Robert Habermeier 006602eff2 Replace AuxStore with custom RocksDB (#2471)
* Use KeyValueDB in approval-voting

* use KVDB instead of AuxStore

* add rocksdb to cargo toml

* add a Config struct

* create new DB in service

* fix dep for regular node

* make optional

* post merge fix

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-02-18 09:24:46 -06:00
Bastian Köcher 7fe3b32aa9 Update Substrate & Polkadot again (#332) 2021-02-18 15:40:53 +01:00
Bernhard Schuster 85489ceb36 [jaeger] unify all used tags, introduce builder pattern, additional… (#2473)
* feat/jaeger: unify all used tags, introduce builder pattern, additional candidate annotations

* chores

* fixes, incomplete fn rename

* another fix

* more fixes

* silly doctests
2021-02-18 15:07:17 +01:00
Bastian Köcher b2021030cc Make it clear in CLI that paritydb is experimental (#8152)
* Make it clear in CLI that paritydb is experimental

Sadly this is a breaking change for the CLI.

* Update client/cli/src/params/database_params.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-02-18 14:58:01 +01:00
Albrecht 0b6fbe7315 Benchmark macro: Allow multiple bounds in where (#8116) 2021-02-18 12:47:33 +01:00
Pierre Krieger afc4b9bb95 Unbreak browser test CI (#8149) 2021-02-18 12:43:13 +01:00
Robert Habermeier fb0106a00e gate approval-checking logic (#2470) 2021-02-18 08:05:13 +01:00
Robert Habermeier b7aac51341 A fast-path for requesting AvailableData from backing validators (#2453)
* guide changes for a fast-path requesting from backing validators

* add backing group to availability recovery message

* add new phase to interaction

* typos

* add full data messages

* handle new network messages

* dispatch full data requests

* cleanup

* check chunk index

* test for invalid recovery

* tests

* Typos.

* fix some grumbles

* be more explicit about error handling and control flow

* fast-path param

* use with_chunks_only in Service

Co-authored-by: Robert Klotzner <robert.klotzner@gmx.at>
2021-02-17 13:51:50 -06:00
Robert Habermeier 4a5e5f13ae Full chunk messages for availability-recovery (#2466)
* add full data messages

* handle new messages

* flip condition
2021-02-17 19:01:39 +00:00
Bastian Köcher 64691eefba Track the collation status in the collator protocol (#2465)
* Track the collation status in the collator protocol

This pr changes the collator protocol to track the status of a
collation. This is mainly used to log the status of a collation when it
is removed to inform the user if a collation maybe never reached a validator.

* Update node/network/collator-protocol/src/collator_side.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Moare logging

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2021-02-17 18:26:30 +00:00
Robert Habermeier 59e2a810bb remove unused RequestBlockAuthorshipData (#2455) 2021-02-17 11:54:51 -06:00
Andronik Ordian 4004217059 make approval voting work on a small testnet (#2421)
* insta-approval for low-node testnets

* fix

* handle 0 needed_approvals and add some logs

* downgrade logs to debug, per block

* fix a warning

* more useful logs

* test

* finish test 🎉

* not so fast

* the test passes, but is it enough?
2021-02-17 11:35:26 -06:00
Andronik Ordian 03a8566cba fix earliest_stored_session (#2461) 2021-02-17 10:43:15 -06:00
Robert Habermeier 9e525e296d Allow at most one candidate with a code upgrade in a block (#2456)
* guide: max one candidate with upgrade per block

* max one candidate with upgrade per block

* ignore on runtime level too
2021-02-17 10:42:06 -06:00
Bernhard Schuster 1e2161258b refactor/reputation: unify the values used (#2462)
* refactor/reputation: unify the values used

* chore/rep: rename Annoy* to Cost*, make duplicate message Cost*Repeated

* fix/reputation: lost and found, convert at the boundary to substrate

* refactor/rep: move conversion to base reputation one level down, left conversions

* fix/rep: order of magnitude adjustments

Thanks pierre!

* remove spaces

* chore/rep: give rationale for order of magnitude

* refactor/rep: move UnifiedReputationChange to separate file

* fix/rep: order of magnitudes correction
2021-02-17 17:18:13 +01:00
Martin Pugh 62c5896592 [CI] Fix check-tx-version Gitlab job (#2449)
* set -e for tx job

* dont set -e while we investigate

* fetch latest release polkadot

* add jq to node:15 image

* Add logic for timing out if nodes never start listening

* remove trap
2021-02-17 16:25:44 +01:00
Cecile Tonglet 0771537a0b Polkadot companion for #8043 (#2382)
* WIP

* WIP

* WIP

* Revert branch change

* CLEANUP

* "Update Substrate"

* Fix build

* Revert "Fix build"

This reverts commit 2dcc3f2a14d28febb84c2c35d682243bc03523a8.

* cargo update -p sp-io

Co-authored-by: parity-processbot <>
2021-02-17 10:12:34 +00:00
Cecile Tonglet 27274c42cf Fix telemetry span not entering properly attempt 3 (#8043)
* Fix tracing tests (#8022)

* Fix tracing tests

The tests were not working properly.

1. Some test was setting a global subscriber, this could lead to racy
conditions with other tests.

2. A logging test called `process::exit` which is completly wrong.

* Update client/tracing/src/lib.rs

Co-authored-by: David <dvdplm@gmail.com>

* Review comments

Co-authored-by: David <dvdplm@gmail.com>

* Fix tracing spans are not being forwarded to spawned task (#8009)

* Fix tracing spans are not being forwarded to spawned task

There is a bug that tracing spans are not forwarded to spawned task. The
problem was that only the telemetry span was forwarded. The solution to
this is to use the tracing provided `in_current_span` to capture the
current active span and pass the telemetry span explictely. We will now
always enter the span when the future is polled. This is essentially the
same strategy as tracing is doing with its `Instrumented`, but now
extended for our use case with having multiple spans active.

* More tests

* Proper test for telemetry and prefix span

* WIP

* Fix test (need to create & enter the span at the same time)

* WIP

* Remove telemtry_span from sc_service config

* CLEANUP

* Update comment

* Incorrect indent

* More meaningful name

* Dedent

* Naming XD

* Attempt to make a more complete test

* lint

* Missing licenses

* Remove user data

* CLEANUP

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* CLEANUP

* Apply suggestion

* Update bin/node/cli/tests/telemetry.rs

Co-authored-by: David <dvdplm@gmail.com>

* Wrapping lines

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
2021-02-17 07:44:25 +00:00
Bastian Köcher 27c7347b9b Make sure the adder collator fails when something is wrong (#2450)
As futures most of the time are catching panics and we don't check this,
it could happen that we have some statement that isn't correct but the
test succeeds successfully.
2021-02-16 19:27:50 +00:00
Robert Habermeier a8d3aca13d Disputes High-level rewrite & Disputes runtime (#2424)
* REVERT: comment out graphviz

* rewrite most of protocol-disputes

* write about conclusion and  chain selection

* tie back in overview

* basic disputes module

* guide: InclusionInherent -> ParaInherent

* language

* add ParaInherentData type

* plug parainherentdata into provisioner

* provide_multi_dispute

* tweak

* inclusion pipeline logic for disputes

* be clearer about signature checking

* reject backing of disputed blocks

* some type rejigging

* known-disputes runtime API

* wire up inclusion

* Revert "REVERT: comment out graphviz"

This reverts commit 66203e362f7872cb413d258f74634a0aad70302b.

* timeouts

* include in initialization order

* address grumbles
2021-02-16 18:50:14 +00:00
Guillaume Thiolliere 8734cf62b2 companion: Babe remove inherent (#2438) 2021-02-16 19:43:38 +01:00
Andronik Ordian b28e1b4e74 fix approval import tests and some bugs (#2452)
* tests: use future::join

* fix panic in cache_session_info_for_head

* fix test assertion

* fix infinite loop in determine_new_blocks

* fix ordering in determine_new_blocks

* fix expected ancestry in tests
2021-02-16 19:08:35 +01:00
Guillaume Thiolliere f35a27cca0 Remove inherent in pallet-babe (#8124) 2021-02-16 19:03:59 +01:00
Guillaume Thiolliere f49aae65a8 fix deprecated usage of panic (#8134) 2021-02-16 19:02:53 +01:00
Bastian Köcher 8dac4f75ad Update Substrate & Polkadot (#330) 2021-02-16 17:42:08 +01:00
Gavin Wood ed9c08dd01 Move dust collection hook to outside of account mutate (#8087)
* Move dust collection hook to outside of account mutate

* Fix dust cleanup in nested mutates.

* Fixes

* Fixes

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* dust removal reentrancy test case integration (#8133)

* dust removal reentrancy test case integration

* Update frame/balances/src/tests_reentrancy.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/balances/src/tests_reentrancy.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/balances/src/tests_reentrancy.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/balances/src/tests_reentrancy.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/balances/src/tests_reentrancy.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* dust removal reentrancy test case integration | removed dependency on tests.rs

* dust removal reentrancy test case integration | formatt correction

* dust removal reentrancy test case integration | formatt correction

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: RK <r.raajey@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2021-02-16 14:26:53 +01:00
Sergei Shulepov 30cba2537b Clean up sizes for a workspace (#2445) 2021-02-16 13:07:35 +01:00
Bastian Köcher d3f9c7db38 Parachain Consensus abstractions (#329)
* Move consensus to consensus-common crate

* Move the parachain consensus out of the collator

* Add first relay chain consensus stuff

* Remove some warnings

* Fix more stuff

* Fix collator test

* Change `ParachainConsensus` to take a mutable self

* Make everything compile

* Feedback
2021-02-16 12:45:30 +01:00
Peter Goodspeed-Niklaus 44d5aba80d Create a macro which automates creation of benchmark test suites. (#8104)
* Create a macro which automates creation of benchmark test suites.

* bump impl_version

* allow unused on test_bench_by_name

* use proper doctest ignore attribute

* Explicitly hand the Module to the test suite

Much better practice than depending on it showing up implicitly in
the namespace.

* explicitly import what we need into `mod tests`

* bench_module is `ident` not `tt`

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* allow end users to specify arguments for new_test_ext

This turned out to be surprisingly easy. On reflection, it turns out
that of course the compiler can't eagerly evaluate the function call,
but needs to paste it in everywhere desired.

* enable explicitly specifying the path to the benchmarks invocation

also enable optional trailing commas

* Revert "bump impl_version"

This reverts commit 0209e4de33fd43873f8cfc6875815d0fd6151e63.

* list failing benchmark tests and the errors which caused the failure

* harden benchmark tests against internal panics

* suppress warning about ignored profiles

unfortunately, setting the profile here doesn't do anything; we'd
need to set it in every leaf package anyway. However, as this was
just making the default explicit anyway, I think it's safe enough
to remove entirely.

* impl_benchmark_test_suite for assets

* impl_benchmark_test_suite for balances

* impl_benchmark_test_suite for bounties

* impl_benchmark_test_suite for Collective

* impl_benchmark_test_suite for Contracts

* impl_benchmark_test_suite for Democracy

* don't impl_benchmark_test_suite for Elections-Phragmen

* impl_benchmark_test_suite for Identity

Note that Identity tests currently fail. They failed in an identical
way before this change, so as far as I'm concerned, the status quo is
good enough for now.

* impl_benchmark_test_suite for ImOnline

* impl_benchmark_test_suite for indices

For this crate also, the test suite fails identically with and without
this change, so we can say that this change is not the cause of the
tests' failure to compile.

* impl_benchmark_test_suite for lottery

* impl_benchmark_test_suite for merkle-mountain-range

* impl_benchmark_test_suite for Multisig

These tests fail identically with and without the change, so the change
seems unlikely to be the origin of the failures.

* impl_benchmark_test_suite for offences

* impl_benchmark_test_suite for Proxy

Fails identically with and without this change.

* impl_benchmark_test_suite for scheduler

* impl_benchmark_test_suite for session

It turns out to be important to be able to exclude items marked
`#[extra]` sometimes. Who knew?

* impl_benchmark_test_suite for staking

* impl_benchmark_test_suite for system

* impl_benchmark_test_suite for timestamp

* impl_benchmark_test_suite for tips

* impl_benchmark_test_suite for treasury

* impl_benchmark_test_suite for utility

Note that benchmark tests fail identically before and after this change.

* impl_benchmark_test_suite for vesting

* fix wrong module name in impl_benchmark_test_suite in Offences

* address line length nits

* enable optional keyword argument: exec_name

Took a _lot_ of macro-wrangling to get the functionality that I want,
but now you have the option to pass in

```rust
impl_benchmark_test_suite!(
	Elections,
	crate::tests::ExtBuilder::default().desired_members(13).desired_runners_up(7),
	crate::tests::Test,
	exec_name = build_and_execute,
);
```

and have it expand out properly. A selected fragment of the expansion:

```rust
        fn test_benchmarks() {
            crate::tests::ExtBuilder::default()
                .desired_members(13)
                .desired_runners_up(7)
                .build_and_execute(|| {
```

* get rid of dead code

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-02-16 10:01:20 +01:00
Robert Habermeier 4f21cc7e5c Integrate Approval Voting into Overseer / Service / GRANDPA (#2412)
* integrate approval voting into overseer

* expose public API and make keystore arc

* integrate overseer in service

* guide: `ApprovedAncestor` returns block number

* return block number along with hash from ApprovedAncestor

* introduce a voting rule for reporting on approval checking

* integrate the delay voting rule

* Rococo configuration

* fix compilation and add slack

* fix web-wasm build

* tweak parameterization

* migrate voting rules to asycn

* remove hack comment
2021-02-15 22:37:13 -06:00
Sergei Shulepov 5c68e6f9cc Mitigation of SIGBUS (#2440)
* Update shared-memory to new version & refactor

This two are combined in a single commit because the new version of
shared-memory doesn't provide the used functionality anymore.

Therefore in order to update the version of this crate we implement the
functionality that we need by ourselves, providing a cleaner API along
the way.

* Significantly decrease the required memory for a workspace

For some reason it was allocating an entire GiB of memory. I suspect
this has something to do with the current memory size limit of a PVF
execution environment (the prior name suggests that). However, we don't
need so much memory anywhere near that amount.

In fact, we could reduce the allocated size even more, but that maybe
for the next time.

* Unlink shmem just after opening

That will make sure that we don't leak the shmem accidentally.

* Do not compile workspace mod for androind and wasm

* Address some review comments

* Fix the test runner

* Fix missed +1 for the attached flag

* Use .expect rather than .unwrap

* Add a rustdoc for the workspace module

* fixup! Use .expect rather than .unwrap

* Add some doc comments to pub members

* Warn on error removing shm_unlink

* Change the alignment implementation

* Fix the comment nit
2021-02-15 14:40:26 -06:00
André Silva 4503346cea node: migrate grandpa voting rule to async api (#2422)
* node: migrate grandpa voting rule to async api

* node: fix grandpa voting rule tests

* "Update Substrate"

* update Substrate

Co-authored-by: parity-processbot <>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2021-02-15 19:07:36 +00:00
André Silva 24739d1ab0 grandpa: make the VotingRule API async (#8101)
* grandpa: make the VotingRule api async

* grandpa: add docs to VotingRuleResult

* grandpa: formatting

* grandpa: use async blocks

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* grandpa: expose VotingRuleResult

* grandpa: revert some broken changes to async syntax

* grandpa: use finality-grandpa v0.14.0

* grandpa: bump impl_version

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-02-15 18:28:04 +00:00
Shawn Tabrizi eb0159f288 Initializer + Paras Clean Up Messages When Offboarding (#2413)
* initial hack in

* finish up

* use notification to pass outgoing paras

* move outgoing paras from notifications

* missing comma

* update guides

* clean up
2021-02-15 17:48:25 +00:00
Pierre Krieger 11894bc21a Update Grafana dashboards (#8127) 2021-02-15 16:47:57 +00:00
Bastian Köcher 22f85c809b Polkadot companion for 8114 (#2437)
* Polkadot companion for 8114

https://github.com/paritytech/substrate/pull/8114

* "Update Substrate"

Co-authored-by: parity-processbot <>
2021-02-15 16:05:46 +01:00
Denis Pisarev 403d9aab10 CI: temp. allow cargo deny to fail (#8122) 2021-02-15 13:15:58 +01:00
Bastian Köcher 33f9becf41 Simplify runtime api error handling (#8114)
* Ahh

* Work work work

* Fix all the compilation errors

* Fix test

* More fixes...
2021-02-15 11:55:40 +00:00
Shawn Tabrizi b5e692104c Add code blocks to doc diagrams (#8118) 2021-02-15 12:10:30 +01:00
Bastian Köcher 5f5df0485a New genesis (#327)
* Update Substrate & Polkadot

* Update chainspecs

* Update again to fix test
2021-02-15 11:20:36 +01:00
Pierre Krieger dc45c2d2f0 Companion for substrate#8079 (#2408)
* Companion for substrate#8079

* More changes

* "Update Substrate"

Co-authored-by: parity-processbot <>
2021-02-15 10:10:58 +01:00
Bastian Köcher 4beb4a4a83 Bump if-watch to 0.1.8 to fix panic (#2436) 2021-02-15 09:54:45 +01:00
Pierre Krieger f6273f7914 Remove all code related to sentry nodes (#8079)
* Remove all code related to sentry nodes

* More fixing
2021-02-15 08:45:03 +00:00
Bastian Köcher 4975521d48 Notify collators about seconded collation (#2430)
* Notify collators about seconded collation

This pr adds functionality to inform a collator that its collation was
seconded by a parachain validator. Before this signed statement was only
gossiped over the validation substream. Now, we explicitly send the
seconded statement to the collator after it was validated successfully.

Besides that it changes the `CollatorFn` to return an optional result
sender that is informed when the build collation was seconded by a
parachain validator.

* Add test

* Make sure we only send `Seconded` statements

* Make sure we only receive valid statements

* Review feedback
2021-02-14 17:36:04 +01:00