Commit Graph

37 Commits

Author SHA1 Message Date
Pierre Krieger 89b312fe9c Revert #2745 (#2791)
* Revert #2745

* Fix error in merge resolve

* Fix compilation
2019-06-05 11:56:36 +02:00
Pierre Krieger 4213fe15cf Split specialization context from network context (#2745)
* Split specialization context from network context

* Line width
2019-06-03 20:26:17 +02:00
Pierre Krieger e9a4c80c40 Rewrite sync and network layers to use futures (#2683)
* Remove background thread from network

* Primarily use tokio-timer instead of tokio

* Minor style fix

* Reintroduce tokio as an optional dependency

* Renames

* Line width

* Ctxt -> Context

* Merge NetworkWorker and NetworkService
2019-05-29 11:58:02 +02:00
Pierre Krieger ff479c4e23 Rework the OnDemand service (#2670)
* Rework the OnDemand service

* Try fix line widths
2019-05-23 20:13:23 +01:00
Pierre Krieger 6ccdbea01c Move extra_requests.rs as a submodule of sync (#2573) 2019-05-14 12:08:01 +02:00
Svyatoslav Nikolsky 22586113ea Light GRANDPA import handler (#1669)
* GrandpaLightBlockImport

* extract authorities in AuraVerifier

* post-merge fix

* restore authorities cache

* license

* new finality proof draft

* generalized PendingJustifications

* finality proof messages

* fixed compilation

* pass verifier to import_finality_proof

* do not fetch remote proof from light import directly

* FinalityProofProvider

* fixed authorities cache test

* restored finality proof tests

* finality_proof docs

* use DB backend in test client

* justification_is_fetched_by_light_client_when_consensus_data_changes

* restore justification_is_fetched_by_light_client_when_consensus_data_changes

* some more tests

* added authorities-related TODO

* removed unneeded clear_finality_proof_requests field

* truncated some long lines

* more granular light import tests

* only provide finality proof if it is generated by the requested set

* post-merge fix

* finality_proof_is_none_if_first_justification_is_generated_by_unknown_set

* make light+grandpa test rely on finality proofs (instead of simple justifications)

* empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* missing trait method impl

* fixed proof-of-finality docs

* one more doc fix

* fix docs

* initialize authorities cache (post-merge fix)

* fixed cache initialization (post-merge fix)

* post-fix merge: fix light + GRANDPA tests (bad way)

* proper fix of empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* fixed easy grumbles

* import finality proofs in BlockImportWorker thread

* allow import of finality proofs for non-requested blocks

* limit number of fragments in finality proof

* GRANDPA post-merge fix

* BABE: pos-merge fix
2019-05-13 11:36:52 +02:00
Pierre Krieger 8ca343ca8c Add reputation system to network crate (#2454) 2019-05-04 13:51:06 +02:00
Pierre Krieger 648dcc2728 Add warnings about breaking changes in network (#2348) 2019-04-23 16:34:47 +02:00
Robert Habermeier 1aa6eb9fa8 Polite-grandpa improvements (#2229)
* send neighbor packets in more generic way

* integrate periodic neighbor-packet rebroadcaster

* integrate reporting

* attach callbacks to commit messages for rebroadcasting and reporting

* Tests for commit relay

* crunch up some nice warnings

* exit-scope sub-futures of grandpa

* address small grumbles

* some changes to commit handling
2019-04-16 15:25:46 +08:00
DemiMarie-parity a10e86ba5a Refactor DigestItem (#2108)
* Add `start_aura2`.

* .gitignore patch conflict files

and remove one that accidentally got committed

* Fix build

The tests still don’t work.

* Fix compilation errors

* Fix compile errors (again)

* Try (and fail) to fix tests

* Properly deserialize data

Previously, `DigestItem::Consensus` had no separate `DigestItemType`,
so it did not get properly serialized and deserialized.

* Add extra debug logging.  Always allow old seals.

A `RUST_LOG=substrate_aura_consensus cargo test --all -- --nocapture \
tests::authoring_blocks` revealed that old seals were being and
rejected, causing the test to hang.  As a temporary debug measure, allow
old seals unconditionally, so that CI can test if this fixes the
problem.

* Forcibly disable rejection of old seals

* Use old trait, but newer serialization

The old trait for `CompatibleDigestItem` actually worked.  By changing
its implementation, one can ensure that all *new* seals have the modern
form, but *legacy* seals are still decoded correctly.

* Bump impl version

* Squash spurious deprecation warning

`rustc` should not be emitting a deprecation warning in deprecated
code, but it does, so silence it.

* Rip out unused Cargo feature

* Move AURA to aura_primitives

* Respond to code review

* Wrap overly-long line

* Reduce logging verbosity and add target

* Add dependency on `sr-primitives` to `aura_primitives`

* Fix build

It failed with a message about Cargo.lock being out of date.

* core: aura: rename aura engine id const

* core: aura: remove superfluous logging

* core: primitives: add removed semicolons

* core: aura: remove unused import

* core: network: style fix

* runtime: update wasm blobs

* runtime: bump impl_version

* core: primitives: tag all DigestItemType variants explicitly
2019-03-29 12:15:23 +00:00
Pierre Krieger 6e394464b8 Replace NodeIndex with PeerId everywhere (#2077)
* Replace NodeIndex with PeerId

* Fix tests

* More test fixing

* Whitespace
2019-03-23 10:34:28 +01:00
Roman Borschel 57387ef585 libp2p-0.5.0 (#1971)
* Update libp2p. Add support for ed25519 node (network) keys.

  * Update networking to the changes from https://github.com/libp2p/rust-libp2p/pull/972.
  * Add support for using ed25519 keys for libp2p networking.
  * Add support for reading libp2p secret keys from (external) files.

* Adapt to changes from https://github.com/libp2p/rust-libp2p/pull/992

* More tests.

* Cosmetics

* Deduplicate tests.

* Remove quickcheck from tests that don't use extra random inputs.

* Remove quickcheck.

* Swap new/default impls for NetworkConfiguration.

* Use libp2p-0.5.0 from crates.io.

* Post-rebase update.

* Remove unnecessary wildcard pattern.

* Combine two overlapping tests.
2019-03-19 11:16:25 +01:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Pierre Krieger c21d7436cc Add an RPC request for the state of the network (#1884)
* Add an RPC request for the state of the network

* Fix concerns

* Fix tests

* Replace comment with TODO

* Rename the RPC
2019-02-28 19:28:38 +01:00
Toralf Wittner 4bf4b37185 Limit the set of known extrinsics and blocks. (#1842)
* Limit the set of known extrinsics and blocks.

* Update Cargo.lock

* Always insert and pop oldest entry if needed.

Always inserting first ensures that the element's LRU position is
updated.
2019-02-21 19:24:23 +01:00
Pierre Krieger 2b71b3e84b Update to libp2p v0.4 (#1836) 2019-02-21 10:04:53 +01:00
Arkadiy Paronyan b3eae17f65 Gossip refactoring (#1811)
* First part of gossip protocol refactoring

* Message validation in GRANDPA

* Reverted to time-based expiration for future round messages

* Removed collect_garbage_for_topic

* Use consensus engine id instead of kind

* Use try_init

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Comment

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Added expiration check on broadcast

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Apply suggestions from code review

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Style

* Style
2019-02-20 12:40:03 +01:00
Stanislav Tkach e60be1ad12 Migrate network, primitives and rpc to the 2018 edition (#1710) 2019-02-06 19:07:48 +01:00
Gregory Terzian a2d2ed69ab Rewrite network protocol/service to use channels (#1340)
* rewrite network protocol/service to use channels

* remove use of unwrap

* re-introduce with_spec

* remove unnecessary mut

* remove unused param

* improve with_spec, add with_gossip

* rename job to task

* style: re-add comma

* remove extra string allocs

* rename use of channel

* turn TODO into FIXME

* remove mut in match

* remove Self in new

* pass headers by value to network service

* remove network sender from service

* remove TODO

* better expect

* rationalize use of network sender in ondemand
2019-02-06 12:54:02 +01:00
André Silva 4983f113e6 Reduce consensus spam (#1658)
* core: fix predicate for dropping grandpa round messages

* core: grandpa: drop commits topic on authority set change

* core: gossip: only drop known messages based on expiration time

* core: grandpa: don't broadcast commit messages

* core: gossip: don't assume topics are header hashes

* core: gossip: expire messages more agressively

* core: grandpa: fix test environment

* core: gossip: fix tests

* core: gossip: track dead topics (and ignore messages)

* core: gossip: test dead topic pruning
2019-02-01 14:25:07 -03:00
Xiliang Chen eb3503b0c7 expose peer information via rpc (#1362)
* expose peer information via rpc

* fixes tests

* cleanup

Co-Authored-By: xlc <xlchen1291@gmail.com>

* Update docs

Co-Authored-By: xlc <xlchen1291@gmail.com>

* Add missing docs

* keep original type for PeerInfo best_hash/best_number

* cleanup

* Update mod.rs
2019-01-09 12:18:24 +01:00
Gregory Terzian 3add75910a move import queue to consensus-common (#1282) 2018-12-31 12:33:21 +01:00
Andrew Jones c36998aeaf Default boot_nodes to local node in build-spec (#1181)
* Default to local bootnode with build-spec

* Rexport libp2p stuff via network crate

* Remove unused imports and fix where formatting

* Remove spurious blank lines

* Remove unnecessary else clause
2018-12-01 16:47:40 +08:00
Robert Habermeier 11fe84a742 Minimal switch of substrate-node to GRANDPA /Aura (#1128)
* add beginnings of SRML grandpa library

* get srml-grandpa compiling

* tests for srml-grandpa

* add optional session integration to grandpa SRML

* start integration into node runtime

* Allow extracting pending change from header digest

* Make it compile on wasm

* make tests compile again

* Move Authority Key fetching into service, simplify service factory construction

* Generalize Authority Consensus Setup system

* Add Authority Setup Docs

* Allow CLI params to be extensible

 - move params to structopts
 - split parsing and default command execution
 - add custom config to node
 - extended parsing of custom config
 - extending params via structop's flatten

* Minor fixes on cli extension params:
 - added docs
 - re-add actual app name, rather than node-name
 - make strategy and subcommand optional

* better cli params

* synchronize GRANDPA and normal node authorities

* Implement grandpa::network for gossip consensus

* run_grandpa in Node

* Fix missed merge error

* Integrate grandpa import queue

* more specific type def

* link up linkhalf and import block

* make grandpa future send

* get compiling

* Fix new params convention and license header

* get it running

* rebuild node runtime WASM

* change logging level

* Update node/cli/src/params.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/cli/src/params.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/cli/src/lib.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/runtime/src/lib.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/cli/src/lib.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Clean up and Fixme for mutable config

* Move GrandpaService Integration into grandpa, feature gated but on per default

* Fixing grandpa runtime module test

* Update wasm runtime hashes for tests

* GRANDPA: use post-header hash when logging scheduled changes

* add an extra bit of logging to authorities

* fixing missing constrain

* remove old code

* move `NewAuthorities` to an event in srml-grandpa

* fix node-executor tests to use grandpa log

* Remove GossipConsensus from tests, use newly provided sync-feature, fixes tests

* Update to latest wasm runtimes

* address grumbles

* address grumbles

* only derive deserialize when using std

* Clean up use of Deserialize
2018-11-21 18:42:50 +01:00
Pierre Krieger 74e907f3cb Some tweaks in the network crate (#1108)
* Move Roles to network::config

* Make network::config public

* Move NetworkConfig and NonReservedMode to config

* Move Params to config

* Move node_id() to NetworkManager and fix tests

* Rename Specialization to NetworkSpecialization
2018-11-14 11:25:16 +01:00
Luke Schoen 037f9dde10 docs: Extends #999. Remove disused package readmes. Rename to sr-metadata. Fix Readme Rustdoc names (#1016)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme

* docs: Remove link to old docs

* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names

* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory

* Change example to use package name that opens all packages in RustDocs

* Fix list of packages names that are available to open in RustDocs

* fix typo

* fix ordering of CLI options and add missing backslash
2018-11-12 21:18:06 +01:00
Benjamin Kampmann 50adea6220 Consensus Engines Implementation: Aura (#911)
* Generalize BlockImport

 - move ImportBlock, BlockOrigin, ImportResult into shared sr-primitives
 - let Consensus provide  and  traits again
 - update consensus traits to latest development
 - implement traits on client::Client, test_client::TestClient
 - update RHD to use the new import_block API

* Move ImportBlock into consensus-common
* Send import notification in aura tests
* Integrating aura into service
* Make Signatures more generic
* Aura Block Production with the given key
* run aura on the thread pool
* start at exact step start in aura
* Add needed wasm blob, in leiu of better solutions.
* Make API ids consistent with traits and bring upstream for sharing.
* Add decrease_free_balance to Balances module
* Encode `Metadata` once instead of two times
* Bitops include xor
* Upgrade key module.
* Default pages to somewhat bigger.
* Introduce upgrade key into node
* Add `Created` event
2018-10-27 15:59:18 +02:00
Benjamin Kampmann ac4bcf879f Generalize the Consensus Infrastructure (#883)
* Split out Consensus
* Supply ImportQueue through network-service
  - simplify ImportQueue.import_blocks
  - remove Deadlock on import_block
  - Adding Verifier-Trait
  - Implement import_queue provisioning in service; allow cli to import
* Allow to actually customize import queue
* Consensus Gossip: Cache Message hash per Topic
2018-10-16 13:40:33 +02:00
Bastian Köcher db427cb45c Adds construct_simple_protocol macro for simplifying the creation of protocols (#897)
* Make `on_message` take the message as `&mut Option<_>`

* Make `ConsensusGossip` implement `Specialization`

* Move `new_session` into `ConsensusGossip`

* Adds `construct_simple_protocol` macro for simplifying the creation of protocols
2018-10-12 13:10:36 +02:00
Bastian Köcher 12f12bff28 Make all copyrights use the correct year (#894)
* Adds a script for automatically update all copyrights

* Ran the `update-copyright` script
2018-10-10 10:28:41 +02:00
Pierre Krieger 0666759b16 Clean up the networking layer (#881) 2018-10-08 18:20:38 +02:00
Robert Habermeier 68b4be14ff Initial GRANDPA integration (#866)
* implement grandpa client

* consensus gossip with arbitrary topics

* defer GRANDPA messages until referenced blocks imported

* set up communication for voter in a transparent way

* instantiate GRANDPA voter

* keep last round state on disk

* switch back to crates.io finality-grandpa

* update cargo.lock

* use new `collect_garbage` API

* update sync test framework and make public

* test that observers can observe

* fix warning

* use more idiomatic predicate for collecting garbage in gossip

* kill spaces

* fix date
2018-10-05 16:30:39 +02:00
Arkadiy Paronyan e1e6c19f64 Limit gossip for non-authorities (#838)
* Limit gossip for non-authorities

* Random shuffle
2018-09-29 13:08:22 +01:00
Arkadiy Paronyan 9a660f82ed Integration tests (#805)
* Started substrate tests

* Sync test

* Test updates

* Improved tests

* Use on-chain block delay

* Parallel test execution

* Otimized tests

* Logging

* Fixed racing test

* Fixed compilation

* Fixed timestamp test

* Removed rlp dependency

* Minor fixes

* Fixed tests

* Removed best_block_id and resolved fdlimit issue

* Whitespace

* Use keyring

* Style

* Added API execution setting

* Removed stale import
2018-09-28 10:37:55 +01:00
Pierre Krieger 52dbf0cace Rewrite the libp2p networking (#742)
* Rewrite the libp2p networking

* Fix erroneous replacement during rebase

* Update libp2p

* Update libp2p

* Remove the logic error about useless substreams

* Use the new NodeHandler system of libp2p

* Commit the Cargo.lock

* Upgrade yamux

* Forward port latest changes

* Fix compilation issues

* Implement the external URL

* Update to latest networking fixes

* Forward port rest of v0.2

* Fix reserved peers being dropped when we're full
2018-09-26 11:56:32 +02:00
Arkadiy Paronyan 0ed48c89ab Remove dependency on the parity repo (#734) 2018-09-13 14:25:38 +02:00
Gav Wood 1e01162505 Phase 1 of repo reorg (#719)
* Remove unneeded script

* Rename Substrate Demo -> Substrate

* Rename demo -> node

* Build wasm from last rename.

* Merge ed25519 into substrate-primitives

* Minor tweak

* Rename substrate -> core

* Move substrate-runtime-support to core/runtime/support

* Rename/move substrate-runtime-version

* Move codec up a level

* Rename substrate-codec -> parity-codec

* Move environmental up a level

* Move pwasm-* up to top, ready for removal

* Remove requirement of s-r-support from s-r-primitives

* Move core/runtime/primitives into core/runtime-primitives

* Remove s-r-support dep from s-r-version

* Remove dep of s-r-support from bft

* Remove dep of s-r-support from node/consensus

* Sever all other core deps from s-r-support

* Forgot the no_std directive

* Rename non-SRML modules to sr-* to avoid match clashes

* Move runtime/* to srml/*

* Rename substrate-runtime-* -> srml-*

* Move srml to top-level
2018-09-12 11:13:31 +02:00