Commit Graph

120 Commits

Author SHA1 Message Date
Bastian Köcher 2e9b2be8fa Update parity-codec/-derive to 3.1 (#1900)
* Update parity-codec/-derive to 3.1

* Update wasm stuff
2019-03-01 09:04:53 +01:00
Robert Habermeier 4c8fec17fc Some gossip improvements (#1892)
* queue messages in future

* use new gossip API in GRANDPA

* implement message_expired for grandpa

* fix indent
2019-02-28 20:19:42 +01:00
Pierre Krieger c21d7436cc Add an RPC request for the state of the network (#1884)
* Add an RPC request for the state of the network

* Fix concerns

* Fix tests

* Replace comment with TODO

* Rename the RPC
2019-02-28 19:28:38 +01:00
Gregory Terzian 8a72abffdd Remove blocking sync -> import-queue operations (#1818)
* remove blocking sync -> import-queue operations

add specialization to testnet

remove add peer default impl on TestNetFactory

* remove empty brackets from dummy specialization

* nits

* make mut_peers take an fn once

* add SpecializationFactory trait in test

* remove add_peer imple in grandpa, fix typo

* use cmp::max for best importing number comparison

* remove import of non-existent create_peer

* add sender to start message
2019-02-28 12:11:18 +01:00
Gregory Terzian bc15fa31ff Add connected peers to protocol, use in sync provider (#1857)
* add connected peers to protocol, use in sync provider

* use PeerId::random

* address comments`

* docs

* fix import of PeerId

* rewrite rpc tests using PeerId::random

* whitespace

* nits

* remove option around peer id and remove field

* further removal of the option around peer id

* fix rpc tests
2019-02-27 18:50:47 +01:00
Gregory Terzian 5282615416 Use bounded channels for network -> sync -> import (#1874)
* use bounded channels for network -> sync -> import

* bound at 4

* indent

* use return value of handle_network_msg
2019-02-27 09:19:57 +01:00
Pierre Krieger 34fccecc02 Add names to tokio threads (#1866) 2019-02-25 15:40:51 -05:00
Gregory Terzian 077ed00951 Remove blocking operations in SyncOracle implementation (#1852)
* remove blocking operations in SyncOracle implementation

* docs

* docs
2019-02-25 14:00:13 +01:00
Pierre Krieger 733ce7d616 Split Substrate messages into multiple substreams (#1796)
* Split Substrate messages into multiple substreams

* Add back Clogged event
2019-02-25 11:55:45 +01:00
Pierre Krieger eaa8f0c97f Add back logging when we ban a peer (#1843)
* Add back logging when we ban a peer

* Make it less verbose
2019-02-22 13:48:59 +01:00
Toralf Wittner 4bf4b37185 Limit the set of known extrinsics and blocks. (#1842)
* Limit the set of known extrinsics and blocks.

* Update Cargo.lock

* Always insert and pop oldest entry if needed.

Always inserting first ensures that the element's LRU position is
updated.
2019-02-21 19:24:23 +01:00
Pierre Krieger 2b71b3e84b Update to libp2p v0.4 (#1836) 2019-02-21 10:04:53 +01:00
Pierre Krieger f5ba8a2ad9 Report the network PeerId to the telemetry (#1835)
* Report the network PeerId to the telemetry

* Fix accidental paste
2019-02-21 10:03:47 +01:00
Arkadiy Paronyan b3eae17f65 Gossip refactoring (#1811)
* First part of gossip protocol refactoring

* Message validation in GRANDPA

* Reverted to time-based expiration for future round messages

* Removed collect_garbage_for_topic

* Use consensus engine id instead of kind

* Use try_init

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Comment

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Added expiration check on broadcast

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Apply suggestions from code review

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Style

* Style
2019-02-20 12:40:03 +01:00
André Silva 21779b8cf2 Import multiple authority set change blocks (#1808)
* core: implement logic for tracking dag of possible pending changes

* core: move pending justifications dag to its own crate

* core: remove unnecessary clone bounds on dag

* core: request justifications in-order from the dag

* core: dag: rename changes variables to node

* core: dag: allow finalizing blocks not part of dag

* core: dag: track best finalized number

* core: dag: add more tests

* core: sync: clean up pending justifications dag

* core: dag: derive codec decode encode

* core: dag: better error support

* core: dag: add finalization guarded by predicate

* core: grandpa: track multiple authority set changes in dag

* core: dag: add pre-order iterator

* core: grandpa: request justifications on startup

* core: dag: rearrange order of definitions

* core: rename util/dag to util/fork_tree

* core: fork_tree: add docs

* core: fork_tree: add more tests

* core: fork_tree: fix issues found in tests

* core: grandpa: fix authorities tests

* core: grandpa: add docs for is_descendent_of

* core: sync: add docs for PendingJustifications

* core: sync: add test for justification requests across forks

* core: sync: don't resend import or finality notifications in tests

* core: grandpa: add test for importing multiple change blocks

* core: grandpa: fix logic for checking if a block enacts a change

* core: grandpa: fix authorities tests
2019-02-20 00:08:43 +01:00
Toralf Wittner c5d3da32f2 Small refactoring. (#1826) 2019-02-19 17:29:13 -03:00
Gregory Terzian 72bb8ef4c5 Rewrite the BasiQueue using channels (#1327)
* use channels to implement basic import queue

* async justification import

* better conditional for is_done in tests

* reword the test for presence of link

* fix conditional

* trace instead of panic when no link present

* reword expectations when sending to importers

* fix

* debug justification import error

* update expectations

* use NumberFor

* nits

* add general description

* move error handling into closure
2019-02-17 10:13:14 +01:00
Svyatoslav Nikolsky 6a6c3155a6 Use DB backend in test client (#1779)
* use DB backend in test client

* Update core/client/db/src/lib.rs

Co-Authored-By: svyatonik <svyatonik@gmail.com>
2019-02-15 11:03:22 +01:00
André Silva f8246ebcdf core: grandpa: collect garbage for topic (#1780)
* core: grandpa: collect garbage for topic

* core: grandpa: use collect_garbage_for_topic in tests
2019-02-13 15:37:44 +01:00
Pierre Krieger 9e999cdd81 Make network-libp2p's Service generic over the message (#1708)
* Make network-libp2p's Service generic over the message

* Apply suggestions from code review

Co-Authored-By: tomaka <pierre.krieger1708@gmail.com>

* Fix warning
2019-02-12 15:36:15 +01:00
André Silva 78a60194eb Fix block justification response handling (#1729)
* core: sync: dont report peers on empty justification response

* core: sync: log dispatched block justification requests
2019-02-08 10:34:50 +01:00
Arkadiy Paronyan 1dc15be5bd Check for block parent before verification. (#1714)
* Treat verification errors more seriously

* Track obsolete requests

* Check block parent before verification

* Style
2019-02-07 11:58:27 +01:00
André Silva eb6dc0394e core: network: fix sync on testing network (#1713) 2019-02-06 19:33:08 +01:00
Stanislav Tkach e60be1ad12 Migrate network, primitives and rpc to the 2018 edition (#1710) 2019-02-06 19:07:48 +01:00
Gregory Terzian a2d2ed69ab Rewrite network protocol/service to use channels (#1340)
* rewrite network protocol/service to use channels

* remove use of unwrap

* re-introduce with_spec

* remove unnecessary mut

* remove unused param

* improve with_spec, add with_gossip

* rename job to task

* style: re-add comma

* remove extra string allocs

* rename use of channel

* turn TODO into FIXME

* remove mut in match

* remove Self in new

* pass headers by value to network service

* remove network sender from service

* remove TODO

* better expect

* rationalize use of network sender in ondemand
2019-02-06 12:54:02 +01:00
André Silva 7c33728510 Lower log level of grandpa until_imported message (#1691)
* core: gossip: add missing docs

* core: grandpa: lower log level on until_imported
2019-02-05 15:37:04 -03:00
André Silva 4983f113e6 Reduce consensus spam (#1658)
* core: fix predicate for dropping grandpa round messages

* core: grandpa: drop commits topic on authority set change

* core: gossip: only drop known messages based on expiration time

* core: grandpa: don't broadcast commit messages

* core: gossip: don't assume topics are header hashes

* core: gossip: expire messages more agressively

* core: grandpa: fix test environment

* core: gossip: fix tests

* core: gossip: track dead topics (and ignore messages)

* core: gossip: test dead topic pruning
2019-02-01 14:25:07 -03:00
Arkadiy Paronyan 641bb7cb46 Stop authoring blocks when offline (#1655)
* Don't author blocks when offline

* Increased canonicalization delay

* Fixed test
2019-02-01 17:17:53 +01:00
Benjamin Kampmann 15ae7cfef6 Document more TODOs as tickets (#1418)
Went through the TODOs, removed a bunch, which are outdated or nothing more than a regular comment, documented a bunch more as actual tickets and made them FIXMEs and unified their structure (`FIXME #TICKETNO DESC` for local tickets, `FIXME: DESC LINK` for external tickets) for easier in-editor support. Further more remove unnecessary remarks and related old code that I noticed in that instance.
2019-01-30 10:29:48 +01:00
Robert Habermeier ced9e72824 Download unknown but announced forks (#1604)
* test reproducing fork sync issues

* update to new announce_block API

* Download unknown forks

* Reverted download_stale

* Avoid cloning the hash

* Typo
2019-01-28 16:41:33 -03:00
Robert Habermeier d027123059 GRANDPA: announce blocks we vote on to peers (#1593)
* announce blocks we vote on to peers (missing network impl)

* Implemented 'announce'

* Fixed test

* improve announce docs and add logging

* Track recently announced blocks
2019-01-28 16:41:36 +00:00
thiolliere 0078927ac5 update codec and grandpa (#1582)
* update codec

* use new codec

* update grandpa code

* build .lock and binaries
2019-01-28 14:16:17 +01:00
Arkadiy Paronyan f16ddcb622 Only request justifications from available peers (#1586) 2019-01-25 20:09:56 -03:00
André Silva 5ddcbe0ca6 core: send justification requests in parallel to block requests (#1563)
* core: send justification requests in parallel to block requests

* core: fix grandpa test
2019-01-25 11:07:26 -03:00
Benjamin Kampmann ab4c70b379 Update dependencies (#1527)
* update to latest env_logger

* roundabout dependency update

* Update to latest vengen

* Update rand to 0.6

* Update to latest rng shuffle API

* Roundabout update of dependencies

* Update rustc_hex

* Update wasm

* Fix logging setup in tests

* revert to fix test
2019-01-25 12:44:41 +01:00
Bastian Köcher 27a882bfac Refactor CLI handling (#1368)
* Rework cli handling

* Update readme

* Adds support for custom subcommands and extra run parameters

* Update readme

* Fixes compilation after master merge

* Make "Run" the default subcommand

Actually its hidden to the outside that is an subcommand.

* Rewrite CLI to work without breaking old CLI behavior

* Some cleanup

* Fix incorrect config setup

* Update README

* Fixes after merge

* Fixes incorrect README
2019-01-25 10:48:46 +00:00
Pierre Krieger a1586df41b Report the average network bandwidth (#1551)
* Report the average network bandwidth

* Fix concerns
2019-01-25 10:34:34 +00:00
Arkadiy Paronyan f38aa0d2de Test syncing to all forks (#1544) 2019-01-24 15:52:48 +00:00
Bastian Köcher 5be237030d Fix warnings in substrate (#1523)
* Fix warnings in substrate

* More warnings removed and wasm updated

* Fixes error on stable and grumbles

* Update wasm files

* Add links to the github issue for replacing `error-chain`
2019-01-24 16:10:07 +01:00
André Silva cff0387af0 Reduce consensus gossip spam (#1538)
* core: keep known gossip messages for twice their expiration

* core: test expiration of known gossip messages

* core: only broadcast grandpa votes if authority in current set

* core: only broadcast grandpa commits if authority in current set
2019-01-23 22:12:51 +00:00
Wei Tang dd88dc6cd6 Split BlockImport and JustificationImport (#1521)
* Split BlockImport and JustificationImport

* Remove unused trait impl

* Fix compile

* Fix grandpa tests

* Fix network tests
2019-01-23 12:22:08 -03:00
Pierre Krieger d30a8e0bc3 Log the clogging networking messages (#1532) 2019-01-23 15:24:25 +01:00
Pierre Krieger cd86643f33 Report when peer is clogged (#1528) 2019-01-23 14:30:20 +01:00
Bastian Köcher 70b1af7b1e Rewrite Inherent data (#1488)
* Implement new inherent data

* Fixes compilation on wasm

* Fixes after rebase

* Switch back to generate inherent stuff by macro

* Update after rebase

* Apply suggestions from code review

Co-Authored-By: bkchr <bkchr@users.noreply.github.com>

* Fix compilation after rebase

* Address grumbles

* Remove `InherentDataProviders` from `Client`

* Update wasm files after rebase

* Address grumbles

* Fixes compilation after latest merge

* Last fix
2019-01-22 17:52:08 +01:00
André Silva 399cee310a Sync block justifications (#1410)
* core: sync protocol for justifications

* core: basic test for justification sync

* core: pass block number with justification

* grandpa: request justifications when importing change blocks

* core: pass finality notifications to chain sync

* core: require justifications for pending change blocks on start

* core: avoid requesting justifications from previous failed peers

* core: timeout block justification requests

* core: add some docs

* core: fix unused variables warning

* core: tick pending justifications fetch periodically

* grandpa: add test for syncing justifications

* core: early exit dispatch of pending justifications

* core: style fix

* core: grandpa: change logging level

* core: sync: add missing docs

* core: network: report peer on bad justification

* core: replace mem::replace with Option::take

* core: revert authority set changes on failed block finalization

* core: grandpa: add docs to import_justification

* core: warn on re-finalization of last finalized block

* core: only notify sync with last finality notification

* core: style fix

* core: add docs for PendingJustifications

* core: network: use BlockRequest messages for justification requests

* core: reference issues in todo comments

* core: grandpa: revert authority set changes on db

* core: grandpa: remove inconsistent state warning
2019-01-21 07:04:01 +01:00
thiolliere beeacf9cfa impl HasCompact for Perbill and Permill (#1411)
* impl Compact<> and HasCompact for Permill Perbill

* update parity-codec to 2.2

* add Cargo.lock

* add lock and build for runtime

* rebuild Cargo.lock after rebase
2019-01-17 11:25:48 +01:00
Svyatoslav Nikolsky da1fb3f273 Use changes tries in query_storage RPC (#1082)
* use changes tries in query_storage RPC

* let + match + return + call -> match
2019-01-17 10:08:50 +01:00
Pierre Krieger a0d458aa06 Update to latest libp2p (#1386)
* Update to latest libp2p

* Fix indentations

* Add basic test

* Apply suggestions from code review

Co-Authored-By: tomaka <pierre.krieger1708@gmail.com>

* Remove Mutex from topology

* Remove unused method

* Fix concerns
2019-01-14 13:38:36 +01:00
Svyatoslav Nikolsky 616716cb4b GRANDPA finality proof draft (#1268)
* grandpa finality proof

* prove GrandpaApi::grandpa_authorities using parent block + some docs

* create justification when consensus data is changed

* generate justifications periodically

* test for ConsensusChanges
2019-01-11 19:25:03 +01:00
Gav Wood fc5fdab5f2 Cherry updates (#1373)
- Introduce compact extrinsics into Cherry
- Remove UpgradeKey from Cherry (Sudo is enough)
- Remove networkId from Cherry properties
2019-01-09 12:52:16 +01:00