Commit Graph

1802 Commits

Author SHA1 Message Date
Andronik Ordian 7a2e1ef6c1 gossip: do not try to connect if we are not validators (#2786)
* gossip: do not issue a connection request if we are not a validator

* guide updates

* use all relevant authorities when issuing a request

* use AuthorityDiscoveryApi instead

* update comments to the status quo
2021-04-01 18:11:43 +02:00
Robert Habermeier 5da762e728 Avoid querying the local validator in availability recovery (#2792)
* guide: don't request availability data from ourselves

* add QueryAllChunks message

* implement QueryAllChunks

* remove unused relay_parent from StoreChunk

* test QueryAllChunks

* fast paths make short roads

* test early exit behavior
2021-04-01 15:57:41 +02:00
Gavin Wood a960e2ff6d Companion for OnSetCode #8496 (#2794)
* Make build with OnSetCode

* Fixes

* update Substrate

Co-authored-by: parity-processbot <>
2021-04-01 15:19:57 +02:00
Tomasz Drwięga 5b77a89874 Merkle Mountain Range & BEEFY integration (#2101)
* Switch branch.

* Implement basic MMR leaf.

* Revert "Switch branch."

This reverts commit 7f4d41c67f27ca560c53fc63fd3bd06ac182403c.

* Bump substrate.

* Integrate BEEFY.

Bump all.

Fix missing imports.

* Use beefy pallet to get authorities.

* Bump BEEFY repo.

* Use next authority set instead of the current one.

* Start BEEFY service.

* Fix BEEFY start up.

* Cache BEEFY authority set.

* Add BEEFY ValidatorSetId to MMR

* Fix code.

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Review grumbles.

* Update beefy repo.

* Work-around missing protocol.

* Revert "Work-around missing protocol."

This reverts commit 0a6257a8bccc1c67e966898cdedc408c6469ffd6.

* Add beefy peers set config.

* Expose storage of BEEFY.

* Uncompress BEEFY keys for merkle tree.

* Update ordering.

* Switch to branch.

* Bump deps.

* Switch to custom beefy.

* Add MMR RuntimeApi and custom rpc.

* Add set length details.

* Fix compilation.

* Expose MmrLeaf storage.

* Expose MmrLeaf storage.

* Don't use session handler, and rather compute & cache beefy details on call.

* Don't use session handler, and rather compute & cache beefy details on call.

* Fixes.

* Update Cargo.lock.

* Switch back to master.

* Update lockfile.

* Fix xcm print issue.

* Cargo.lock.

* Use master branch.

* Remove extra dep.

* Fix tests.

* Update Cargo.lock

* Add BEEFY & MMR to westend.

* Implement session keys migration.

* Update testnet script.

* start BEEFY for all node types

* Update Cargo.lock

* fix Cargo.toml

* resolve another merge conflict

* add Westend BEEFY keys

* Apply suggestions from code review

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Update BEEFY.

* Add Rococo BEEFY keys

* resolve merge issue

* fix pallet indices

* fix Westend OldSessionKey

* remove unused imports in Westend runtime

* Fix compilation for Westend.

* address review

* start BEEFY gadget conditionally

* address review again

* fix typo

* remove duplicate

* remove another duplicate

* well

* add missing stuff

* cleanup Cargo.toml files

- revert unnecessary changes
- add missing /std dependencies
- remove unused dependencies

* runtime: remove unused structs from rococo runtime

* node: cleanup service

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: adoerr <0xad@gmx.net>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-04-01 12:50:04 +02:00
Robert Habermeier ccfabaa0c6 Node-side subsystems for Disputes (#2566)
* dispute subsystem files

* rename

* fix linkcheck

* flesh out section README

* coordinator schema

* DisputeCoordinatorMessage

* stub & coordinator protocol

* dispute coordinator

* add some more message fields

* move links to bottom

* dispute participation

* Cleen It Up !

* runtime: store candidate receipts in dispute state

yeah, this is a little heavier. why are you reading this?

* Revert "runtime: store candidate receipts in dispute state"

This reverts commit 51c10bfd4d866e287e6bd88f317ed57ed987eaee.

* add dispute availability statement type and prepare for availability

* add 'spam slots' to disputes runtmie

* return Spam Slots info from runtime

* rework `ImportStatement` to `ImportStatements`

* some more methods for dispute coordinator

* candidates-included runtime API

* algo for providing disputes to runtime.

* handle signing with coordinator

* dispute coordinator chain ops

* remove dead file

* remove keystore from dispute participation

* adjust ApprovedAncestor to return the necssary data

* discuss how approved ancestor and determine undisputed chain are used together

* add TODO

* initiate disputes from approval voting

* route statements from candidate backing and approval voting

* fix guide build
2021-04-01 03:00:46 +02:00
Andronik Ordian caebf642dd statement-distribution: do not use OurViewChange (#2790)
* quickfix for statement-distribution

* some logs
2021-03-31 23:28:17 +02:00
Robert Klotzner eb6786ad05 Better timeout values now that we are going to be connected to all nodes. (#2778)
* Better timeout values.

* Fix typo.

* Fix validator bandwidth.

* Fix compilation.
2021-03-31 22:34:12 +02:00
Kian Paimani c56f9e0514 proper executor/block type for benchmarks and try-runtime (#2771)
* proper executor/block type for benchmarks and try-runtime

* Fix

* update benches as well to latest work

* Remvoe dbg
2021-03-31 20:40:15 +02:00
Robert Habermeier e65cad69ec Fix future-polling loop in availability and add a better early-exit (#2779)
* onto the front

* fix early exit for waiting for requests

* add logging back
2021-03-31 17:35:17 +02:00
Gavin Wood bd9bdf5f5f Add proxy and utility pallets. (#2774)
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-03-31 12:32:32 +00:00
Kian Paimani da1f54a19d Add some tests to demonstrate an estimate of the nominator limit (#2724)
* Add some tests to demonstrate an estimate of the nominator limit

* Update runtime/kusama/src/lib.rs

* Update runtime/polkadot/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Fix build

* remove max

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-03-31 09:13:46 +00:00
Shawn Tabrizi 02166f4e0c Add Root Functions to Parachains System (#2777)
* Add Root functions to Paras

* note past code

* fix build

* fix test build

* compile fix and force_queue_action
2021-03-30 21:54:21 +02:00
Andronik Ordian 9ac35d9f2b gossip: choose a random subset on send instead of limiting connections (#2776)
* gossip: choose random subset on send

* naming bikeshed
2021-03-30 20:59:53 +02:00
Andronik Ordian a3115401c3 network-bridge: elevate log level for connections (#2772) 2021-03-30 20:01:57 +02:00
Guillaume Thiolliere 445f626c34 Fix usage (#2775) 2021-03-30 18:37:24 +02:00
Gavin Wood 0395b5dd16 Fungibles adapter and MultiLocation::X8 (#2662)
* Fungibles adapter and MultiLocation::X8

* Comments

* Comment & whitespace

* Fix up CI/CD for the new labels.

* Fix labels

* Whitespace

* Fixes

* Update Cargo.lock

* Update xcm/xcm-builder/src/fungibles_adapter.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-03-30 17:52:50 +02:00
Robert Habermeier 08d5b268a0 Retry availability until the receiver of the request is dropped (#2763)
* guide updates

* keep interactions alive until receivers drop

* retry indefinitely

* cancel approval tasks on finality

* use swap_remove instead of remove
2021-03-30 17:33:38 +02:00
Robert Klotzner 6514e00144 Add tags to pov-fetcher. (#2768)
* Add tags to pov-fetcher.

* Add stage as well.

* Get rid of redundant tags.
2021-03-30 15:07:07 +02:00
Andronik Ordian bdee5a3923 approval-distribution: add an assertion (#2761) 2021-03-30 14:18:34 +02:00
Bastian Köcher c047a2bde2 Companion for Substrate#8472 (#2759)
* Companion for Substrate#8472

https://github.com/paritytech/substrate/pull/8472

* update Substrate

Co-authored-by: parity-processbot <>
2021-03-30 12:15:50 +02:00
Robert Klotzner 0bc42785b4 availability-distribution: Retry failed fetches on next block. (#2762)
* availability-distribution: Retry on fail on next block.

Retry failed fetches on next block when still pending availability.

* Update node/network/availability-distribution/src/requester/fetch_task/mod.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Fix existing tests.

* Add test for trying all validators.

* Add test for testing retries.

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-03-30 00:28:43 +02:00
Robert Habermeier e906598e94 tracing for pending_known map (#2755)
* tracing for pending_known map

* fix condition in retain

* add block hash to pending tracing
2021-03-29 21:38:03 +02:00
Robert Habermeier 235c2e26b1 more logs in issue-approval (#2758) 2021-03-29 21:06:44 +02:00
Robert Habermeier 54074d2d76 send assignments even when we have an approval (#2757) 2021-03-29 16:34:14 +00:00
Shawn Tabrizi 76db00f398 Add Many Sanity Checks to Crowdloans (#2745)
* Add many sanity checks to Crowdloans

* fix tests

* test contribution block in integration test

* fix rococo build

* remove leaser from crowdloans

* fix docs and terms

* fix compile
2021-03-29 16:52:55 +02:00
Robert Habermeier 7c21dbbdf4 use unbounded when notifying distribution of new blocks (#2752) 2021-03-29 14:30:20 +00:00
legacycode f2f0951b58 Updated docker documentation. (#2313) 2021-03-29 16:03:36 +02:00
Robert Habermeier fc154d2ada Reduce signal channel sizes and more logging on approval-voting (#2751)
* reduce signal channel capacity

* more tracing for approval-voting
2021-03-29 15:46:12 +02:00
Shawn Tabrizi afc9bf8f25 Allow Non Intersecting Bids in Para Auction (#2741)
* Update auctions.rs

* Add integration test

* check that bid to another para requires new funds
2021-03-29 14:41:43 +02:00
Robert Klotzner 0a9fe852df Move non runtime related stuff into node/primitives (#2743)
* Remove stuff out of the runtime that does not belong there.

There might be more, but it is a start.

* White space fixes.

* Fix tests.

* Leave whitespace in ui tests alone.

* Add back zstd for no reason.

* Fix browser wasm (hopefully)
2021-03-29 02:15:44 +02:00
Shawn Tabrizi a6fd8f78e6 Emit an Event for Random Block Selected in Auction (#2740)
* Update auctions.rs

* Update auctions.rs
2021-03-29 01:38:16 +02:00
Robert Habermeier 8ebbe19d10 Split NetworkBridge and break cycles with Unbounded (#2736)
* overseer: pass messages directly between subsystems

* test that message is held on to

* Update node/overseer/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* give every subsystem an unbounded sender too

* remove metered_channel::name

1. we don't provide good names
2. these names are never used anywhere

* unused mut

* remove unnecessary &mut

* subsystem unbounded_send

* remove unused MaybeTimer

We have channel size metrics that serve the same purpose better now and the implementation of message timing was pretty ugly.

* remove comment

* split up senders and receivers

* update metrics

* fix tests

* fix test subsystem context

* use SubsystemSender in jobs system now

* refactor of awful jobs code

* expose public `run` on JobSubsystem

* update candidate backing to new jobs & use unbounded

* bitfield signing

* candidate-selection

* provisioner

* approval voting: send unbounded for assignment/approvals

* async not needed

* begin bridge split

* split up network tasks into background worker

* port over network bridge

* Update node/network/bridge/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* rename ValidationWorkerNotifications

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
2021-03-29 01:18:53 +02:00
Andronik Ordian 6f464a360f approval-distribution: limit the amount of assignments on unify (#2737)
* approval-distribution: limit the amount of packets on unify

* guide: fix a typo

* compilation fix

* grammar

* Update roadmap/implementers-guide/src/node/approval/approval-distribution.md

Co-authored-by: David <dvdplm@gmail.com>

* more grammar

* propagate only local assignments/approvals after a certain depth

* increase the threshold

* guides update

Co-authored-by: David <dvdplm@gmail.com>
2021-03-28 23:30:06 +02:00
Andronik Ordian 171fc69961 approval-voting: more spans and metrics (#2742)
* approval-voting: more spans and metrics

* s/db/approval db
2021-03-28 23:28:49 +02:00
Pierre Krieger e3dc9024ce Call NetworkService::add_known_address before sending a request (#2726)
* Call NetworkService::add_known_address before sending a request

* Better doc

* Update Substrate

* Update Substrate

* Restore the import 🤷‍♀️ I don't know why it compiles locally

* imports correctly

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2021-03-28 16:01:49 +00:00
Robert Habermeier 5952e790fa Overseer: subsystems communicate directly (#2227)
* overseer: pass messages directly between subsystems

* test that message is held on to

* Update node/overseer/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* give every subsystem an unbounded sender too

* remove metered_channel::name

1. we don't provide good names
2. these names are never used anywhere

* unused mut

* remove unnecessary &mut

* subsystem unbounded_send

* remove unused MaybeTimer

We have channel size metrics that serve the same purpose better now and the implementation of message timing was pretty ugly.

* remove comment

* split up senders and receivers

* update metrics

* fix tests

* fix test subsystem context

* fix flaky test

* fix docs

* doc

* use select_biased to favor signals

* Update node/subsystem/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
2021-03-28 15:55:10 +00:00
Robert Klotzner c6f07d8f31 Request based PoV distribution (#2640)
* Indentation fix.

* Prepare request-response for PoV fetching.

* Drop old PoV distribution.

* WIP: Fetch PoV directly from backing.

* Backing compiles.

* Runtime access and connection management for PoV distribution.

* Get rid of seemingly dead code.

* Implement PoV fetching.

Backing does not yet use it.

* Don't send `ConnectToValidators` for empty list.

* Even better - no need to check over and over again.

* PoV fetching implemented.

+ Typechecks
+ Should work

Missing:

- Guide
- Tests
- Do fallback fetching in case fetching from seconding validator fails.

* Check PoV hash upon reception.

* Implement retry of PoV fetching in backing.

* Avoid pointless validation spawning.

* Add jaeger span to pov requesting.

* Add back tracing.

* Review remarks.

* Whitespace.

* Whitespace again.

* Cleanup + fix tests.

* Log to log target in overseer.

* Fix more tests.

* Don't fail if group cannot be found.

* Simple test for PoV fetcher.

* Handle missing group membership better.

* Add test for retry functionality.

* Fix flaky test.

* Spaces again.

* Guide updates.

* Spaces.
2021-03-28 17:11:38 +02:00
Peter Goodspeed-Niklaus 27b6d83974 use named generic types in generate_solution_type (#2707)
* use named generic types in generate_solution_type

* "Update Substrate"

Co-authored-by: parity-processbot <>
2021-03-28 08:47:52 +00:00
Robert Habermeier ef816b089d Approval voting failsafe (#2675)
* add consensus log type

* origin and issue force_approve

* add origin in runtimes

* ref API

* scrape force_approve digest from header

* add parent_hash to BlockEntry

* add block_number to block entry and force_approve skeleton

* implement and plug in force-approve

* test force_approve

* test force_approve extraction

* westend runtime

* Update node/core/approval-voting/src/approval_db/v1/mod.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* rename

* Update runtime/parachains/src/initializer.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-03-28 00:57:04 +00:00
Andronik Ordian dce20644c8 approval-distribution: moar metrics (#2734) 2021-03-28 00:23:32 +01:00
Andronik Ordian 71f1985172 approval-distribution: moar logs (#2732) 2021-03-27 23:21:25 +01:00
Shawn Tabrizi 69c7c52de6 Handle Onboarding in Current Lease Period (#2731)
* Lease out current period and trigger onboard

* Add test for trigger_onboard

* patch and add benchmark

* finish benchmarks

* Use result instead of panic for test_registrar

* nits
2021-03-27 20:16:10 +00:00
RRTTI 66c4936e21 Increase rotation period (#2688)
* Cancel treasury burn and increase rotation period 

What this PR changes:

- Kusama treasury burn to be transferred to the Saociety pot: cancelled temporarily
- Increase the rotation period for candidates to show PoI FROM 3,3 days to 7 days.

* Update runtime/kusama/src/lib.rs

* Update lib.rs

Co-authored-by: Gavin Wood <gavin@parity.io>
2021-03-27 17:37:30 +01:00
Shawn Tabrizi 7f6ef6fb7b Introduce add_memo for Crowdloans (#2728)
* Add memo, but don't use it yet

* add_memo

* add weights

* Update lib.rs

* Update crowdloan.rs

* add event
2021-03-27 16:36:48 +00:00
Robert Habermeier c503fbc2a0 duplicate logging fix (#2729)
* duplicate logging fix

* remove duplicate peer IDs
2021-03-27 16:17:35 +01:00
Robert Habermeier 15a956321a use a gauge for approval lag (#2725) 2021-03-27 16:13:34 +01:00
Shawn Tabrizi b1e81a7fb0 Fix Crowdloan Withdraw Requirements (#2723)
* prevent crowdloan withdraw from being griefed

* Update crowdloan.rs

* Update runtime/common/src/crowdloan.rs

* Update runtime/common/src/crowdloan.rs
2021-03-27 12:06:29 +01:00
Bernhard Schuster 3f50465d49 use novelpoly 1.0.0 (#2727) 2021-03-27 09:07:28 +01:00
Robert Klotzner 6ea6299bca Reduce network bridge logging verbosity (#2717)
* Those should really be trace.

- Very spammy
- And they in fact trace the execution
- Should not be enabled lightly - will slow network bridge down.

* Make report peers debug again.
2021-03-27 00:19:43 +01:00
Shawn Tabrizi 67e95d1b8e Parachain Onboarding Extras (#2713)
* clear_lease

* schedule upgrade and downgrade

* fix compile

* comments
2021-03-26 22:41:07 +01:00