Commit Graph

108 Commits

Author SHA1 Message Date
Pierre Krieger eabdcbdd4f Remove support for secp256k1 for network keys (#3897) 2019-10-23 22:05:05 +02:00
Bastian Köcher d82216673c Throw an error if a bootnode is registered with two different peer ids (#3891)
* Throw an error if a bootnode is registered with two different peer ids

* Rename error

* Fix compilation :(

* Review feedback
2019-10-23 17:17:12 +02:00
Svyatoslav Nikolsky 387c31598d Prepare for asynchronous transaction validation in tx pool (#3650)
* async txpool API

* Update core/rpc/src/author/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/transaction-pool/graph/src/pool.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Pool -> Pool + ValidatedPool

* removed lost block_on when importing xt from network

* fix grumbles

* alias for future::Executor in rpc

* removed executor from Author RPCs

* Pool + SharedValidatedPool -> Pool

* fix compilation after merge

* another fix

* another fix
2019-10-01 12:14:25 +03:00
Arkadiy Paronyan 3242d7f2b6 Explicit sync API for downloading important, possibly orphaned, forks (#3633)
* Explicit sync API

* Keep sync requests

* Don't request the finalized block we already have

* Dropping requests & docs

* Renamed a function
2019-09-28 14:02:36 +02:00
Toralf Wittner af0d71d389 Add block announce validator. (#3346)
* Add `BlockAnnounceValidator` trait.

* Add associated data to block announcement.

* Make tests compile.

* Move validator into `sync.rs`.

* Smaller changes.

* Update core/network/src/protocol.rs

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update core/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/network/src/test/sync.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Formatting.

* Remove assoc. data from `BlockImportNotification`.

* Use `Option<Vec<u8>>` for associated data.

* Update core/network/src/protocol/sync.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix type error.
2019-09-24 10:48:21 +02:00
Arkadiy Paronyan 84d0c790f3 Send block status with announcement (#3607)
* Send block status with announcement

* Fixed tests

* Whitespace

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Additional comment

* Update comment

Co-Authored-By: André Silva <andre.beat@gmail.com>
2019-09-17 11:19:46 +02:00
Max Inden 027d88796b core/authority-discovery: Enable authorities to discover each other (#3452)
With the *authority-discovery* module an authoritative node makes itself
discoverable and is able to discover other authorities. Once discovered, a node
can directly connect to other authorities instead of multi-hop gossiping
information.

1. **Making itself discoverable**

    1. Retrieve its external addresses

    2. Adds its network peer id to the addresses

    3. Sign the above

    4. Put the signature and the addresses on the libp2p Kademlia DHT

2. **Discovering other authorities**

    1. Retrieve the current set of authorities

    2. Start DHT queries for the ids of the authorities

    3. Validate the signatures of the retrieved key value pairs

    4. Add the retrieved external addresses as ~reserved~ priority nodes to the
       peerset


* node/runtime: Add authority-discovery as session handler

The srml/authority-discovery module implements the OneSessionHandler in
order to keep its authority set in sync. This commit adds the module to
the set of session handlers.

* core/network: Make network worker return Dht events on poll

Instead of network worker implement the Future trait, have it implement
the Stream interface returning Dht events.

For now these events are ignored in build_network_future but will be
used by the core/authority-discovery module in subsequent commits.

* *: Add scaffolding and integration for core/authority-discovery module

* core/authority-discovery: Implement module logic itself
2019-09-06 17:43:03 +02:00
Pierre Krieger be732a2d68 Update to libp2p 0.12 (#3408) 2019-08-15 16:41:39 +02:00
Max Inden 83052fe573 core/network: Surface peerset's set_priority_group in NetworkService (#3376)
`PeerSetHandle.set_priority_group` allows modifying a priority group by
group identifier. With this commit the function can be accessed through
`NetworkService`.

This is need in order for a validator to connect to as many other
validators as configured without reserving a specific connection slot
for them.
2019-08-12 19:23:59 +02:00
Arkadiy Paronyan c0fd256c01 Fixed test block conditions (#3332) 2019-08-08 01:01:16 +02:00
Pierre Krieger b5b1c2a4d8 Rename on_demand to light_dispatch and various minor changes (#3315)
* Rename on_demand to light_server

* Small docs improvement

* Rename on_block_announce to update_best_number

* More minor documentation

* Light server -> light dispatch

* is_light_rq_response -> is_light_response
2019-08-07 12:40:49 +02:00
Kian Paimani 79feb23a22 Refactor sr_primitives. (#3214)
* refactor sr_primitives.

* Fix try build error.

* Line-width

* Ui test.

* Final fixes.

* Fix build again.

* bring back ui test.

* Fix unsigned import.

* Another ui fix.

* Also refactor substrate-primitives

* Fix benchmarks.

* Fix doc test.

* fix doc tests
2019-07-29 14:43:53 +02:00
Pierre Krieger 9370a4a6b6 Clean up the basic-authorship crate (#3206)
* Switch consensus-common to new futures

* Fix tests

* More tests fixing

* Make Proposer, OnSlot and SyncOracle mut

* Make the Environment mut as well

* Fix test

* Fix Babe tests

* Babe fixes

* Fix CLI service tests

* Fix Babe tests

* Remove unnecessary trait bound

* Inline the code of BlockBuilder and AuthoringApi

* Remove warning lint

* Bounds simplification

* Imports simplification

* Don't panic on bad generated block

* Code style

* Add doc example

* Remove dependency on aura

* Order dependencies alphabetically

* Minor style
2019-07-27 20:24:51 -04:00
Pierre Krieger b31dcdf342 Switch consensus crates to new futures (#3146)
* Switch consensus-common to new futures

* Fix tests

* More tests fixing

* Fix Babe tests

* Fix Babe tests
2019-07-25 10:55:50 +08:00
Roman Borschel 343f4a2a50 libp2p-next (#3076)
* Changes for the next libp2p release:

  * Updates to the Kademlia APIs.
  * Updated imports due to the extracted libp2p-swarm crate.
  * ...

Still pending at least the following:

  * rust-libp2p/#1189
  * rust-libp2p/#1191
  * rust-libp2p/#1194

* Use Quorum::One.

The previous choice was apparently arbitrary.

* Use libp2p-0.11 from crates.io. Address feedback.

* Correct imports after merge.
2019-07-24 17:32:25 +02:00
Michael Müller c70b81444a Introduce srml/im-online (#3079)
* Fix grammar and typo

* Extend network service

* Extend offchain API

* Support creating unsigned UncheckedExtrinsic

* Introduce srml/im-online

* Bump impl and spec version

* Fix web-wasm test

* Apply suggestions from code review

Remove parity-multiaddr dependency

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Replace transmute with from_raw_parts

* Replace PeerId.to_string() with .to_base58()

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update Cargo.lock

* Bump impl and spec version (again)

It was updated in master in the meantime.

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Address comments

* Add public function is_online_in_current_session()

* Bump spec_version

* Fix doc tests

* Improve comments

* Remove superfluous line

* Name parameters consistently

* Implement comments

* Switch From to TryFrom

* Use Vec instead of HashSet

* Fix tests

* Revert me: local testing

* Fix check if already sent during session

We gossip each session, hence we need to check
if already sent in this session (not era).

* Fix typos

* Consistent terminology

* Revert "Revert me: local testing"

This reverts commit 73fbc29ff3e5ed71d99436318260b4f007e837f4.

* Introduce IsMember trait

* Implement misc comments

* Remove unused function

* Fix test

* Fix external_addresses being written

* Fix test

* Add necessary trait bound

* Do not increment version

* Update lib.rs
2019-07-20 09:19:44 +08:00
Max Inden 515a56af4a core/network: Remove self mutability on put and get value (#3118)
Make `NetworkService.{get,put}_value` only take a self reference instead
of a mutable self reference. When retrieving `NetworkService` via
`NetworkWorker`, one only gets an immutable reference.
2019-07-16 08:49:27 +09:00
Pierre Krieger 5bd806bd9b Improvements to the import queue (#3101)
* Remove block_imported

* Move blocks results processing to sync

* Remove methods from Link

* Better errors

* Allow cancelling the import queue

* Restore the import trace

* Fix network tests

* Line widths

* Use has_error instead

* Minor style
2019-07-14 13:22:32 +01:00
Pierre Krieger efed2e3098 Remove set_finality_proof_request_builder (#3087)
* Remove set_finality_proof_request_builder

* Fix Babe

* Fix Grandpa

* Fix service doctests
2019-07-11 09:44:32 +02:00
Pierre Krieger d7b6720663 Change the import queue traits to take &mut self instead of &self (#3058)
* SharedFinalityProofRequestBuilder -> BoxFinalityProofRequestBuilder

* SharedThings -> BoxThings

* Fix tests

* build_request_data now takes &mut self

* The other traits now also take &mut self

* More or less fix tests

* Fix tests

* Fix more tests

* Moar tests

* Don't call make_block_import multiple time

* Fix doctest
2019-07-09 17:11:25 +02:00
Pierre Krieger 1e126eab2f Network crate cleanups (#3049)
* Remove useless internal messages

* Remove NetworkService::disconnect_peer

* Remove NetworkMsg altogether

* Rename ProtocolMsg ServerToWorkerMsg

* Remove useless code

* Add example for parse_str_addr

* Move parse_str_addr and ProtocolId to config

* Don't reexport the content of config

* Rework the imports

* More reexports rework

* Add documentation

* Move finalization report to network future

* Move on_block_imported to worker

* get_value/put_value no longer locking

* local_peer_id() no longer locks

* Remove FetchFuture

* Service imports cleanup

* Produce the network state in the network task

* Merge network task and RPC network task

* Move network methods to NetworkWorker

* Remove Arc peers system from network

* add_reserved_peer now goes through the channel

* Remove Mutex around network swarm

* Remove the FnOnce alias traits

* Replace is_offline with num_connected

* Improve style of poll()

* Fix network tests

* Some doc in service module

* Remove macro export

* Minor doc changes

* Remove the synchronized() method of the import queue

* Line width

* Line widths

* Fix import queue tests

* Fix CLI tests
2019-07-08 15:33:29 +02:00
Pierre Krieger 489a5ce9d7 Merge Protocol and ProtocolBehaviour (#2962)
* Pass the peerset config to ProtocolBehaviour

* Don't pass the protocol versions

* Move fields from protocol_behaviour.rs to protocol.rs

* Remove LocalNetworkOut

* Move CustomProtos from protocol_behaviour.rs to protocol.rs

* Remove ProtocolBehaviour

* Inline poll()

* Force Behaviour to use Protocol

* Don't even attempt to have working tests

* Remove NetworkOut trait

* Line widths
2019-07-05 16:06:07 +02:00
Pierre Krieger 22ec13cf65 Rewrite the network TestNet (#3016)
* Add a memory-only option for the network

* Tests cleanup

* Make grandpa/aura/babe compile

* Aura and Babe tests now passing

* More work on tests rewrite

* Attempt to fix grandpa

* Make more grandpa tests pass

* More grandpa tests work

* Work on sync tests

* More work

* light_peer_imports_header_from_announce passes

* can_sync_small_non_best_forks passes

* syncing_node_not_major_syncing_when_disconnected passes

* blocks_are_not_announced_by_light_nodes passing

* All sync tests passing 🎉

* Some TestNet cleanup

* Work on grandpa tests

* More grandpa work

* GrandPa work

* Add check about block_import

* Remove the temporarily added Sync

* Fix network tests warnings

* voter_persists_its_votes passes

* Fix imports in network tests

* Fix service tests

* Call on_block_imported 🤷

* Add shortcut

* Finish using shortcut
2019-07-05 11:19:03 +01:00
Fedor Sakharov e735853ca3 Add DHT key-value behaviour (#2937)
* Add DHT key-value behaviour

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Return which key failed to be inserted
2019-06-26 13:30:39 +02:00
Pierre Krieger e919d03331 Add ProtocolBehaviour (#2922)
* Add ProtocolBehaviour

* Fix tests

* Line widths

* Address concerns

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove TODO
2019-06-24 13:22:03 +02:00
Pierre Krieger 437a6bc6b1 Move the network status reporting to the service (#2916)
* Move the network status reporting to the service

* Fix tests

* Fix build
2019-06-21 17:13:11 -04:00
Pierre Krieger 7efc504d59 Import queue API revamp (#2856)
* ImportQueue methods are now mut

* Link methods are now mut

* Remove Arc from BasicSyncQueue

* Fix tests

* Remove BasicSyncQueue

* Change the import queue API

* Add buffered_link

* Remove obsolete tests

* Comments and style improvement pass

* Fix grandpa and comment cleanup

* Update core/consensus/common/src/import_queue.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>
2019-06-18 21:50:48 -04:00
Pierre Krieger 4a44a07fd0 Move libp2p tests to custom_proto (#2884) 2019-06-18 10:36:42 +02:00
Pierre Krieger f7bd56d2a8 Handle RPC requests in the substrate-service (#2866)
* Rework RPC queries

* Remove SyncProvider trait

* Fix RPC tests
2019-06-15 15:44:04 +02:00
Pierre Krieger 12bbc2ffd9 Merge network-libp2p into network (#2843)
* Move network-libp2p into network

* Merge libp2p_service into service

* Don't expose RegisteredProtocol in the API

* Extract DiscoveryBehaviour from Behaviour

* Restore libp2p tests

* Add a test for discovery

* Line width

* Remove bandwidth check

* Fix gitlab
2019-06-13 18:21:31 +02:00
Pierre Krieger 4d3396d095 Move modules around in network (#2822)
* Rename modules in protocol

* Line widths
2019-06-07 15:30:45 +02:00
Bastian Köcher 6142f95611 Fix tons of warnings in newest nightly (#2784)
* Fix tons of warnings in newest nightly

* Fix sr-api-macro doc tests
2019-06-04 20:09:49 +02:00
Pierre Krieger e9a4c80c40 Rewrite sync and network layers to use futures (#2683)
* Remove background thread from network

* Primarily use tokio-timer instead of tokio

* Minor style fix

* Reintroduce tokio as an optional dependency

* Renames

* Line width

* Ctxt -> Context

* Merge NetworkWorker and NetworkService
2019-05-29 11:58:02 +02:00
Pierre Krieger f8303a6b57 The network configuration now makes more sense (#2706)
* The network configuration now makes more sense

* Reexport ProtocolConfig
2019-05-28 16:50:36 +02:00
Svyatoslav Nikolsky 48b2ba041f Synchronous import queue + fix async inport queue shutdown (#2701)
* sync implementation of ImportQueue

* fix import queue shutdown

* never clone import queue
2019-05-28 15:07:35 +02:00
Pierre Krieger ff479c4e23 Rework the OnDemand service (#2670)
* Rework the OnDemand service

* Try fix line widths
2019-05-23 20:13:23 +01:00
Pierre Krieger c357854015 More extractions from the Protocol struct (#2641)
* Pass the TransactionPool explicitly

* Extract finality_proof_provider

* Remove Protocol::connected_peers

* Add note and rename function

* Fix tests

* More test fixing

* Revert the WASM locks, I guess

* Add space

* Remove space
2019-05-23 12:07:51 +02:00
Pierre Krieger 062b734571 Remove the NetworkChan (#2577)
* Remove the NetworkChan from the API

* Remove the NetworkChan altogether

* Address review

* Fix line widths

* More line width fixes

* Remove pub visibility from entire world

* Fix tests
2019-05-21 14:07:01 +02:00
Pierre Krieger 18f7acce98 Move ProtocolMsg to service.rs (#2560)
* Move ProtocolMsg to service.rs

* Fix line width

* Fix line width again

* Correct whitespace

* Another whitespace correction
2019-05-14 14:26:52 +01:00
Pierre Krieger 2724cdac33 Remove the NetworkChan from the OnDemandService (#2561) 2019-05-13 20:55:27 +02:00
Svyatoslav Nikolsky 22586113ea Light GRANDPA import handler (#1669)
* GrandpaLightBlockImport

* extract authorities in AuraVerifier

* post-merge fix

* restore authorities cache

* license

* new finality proof draft

* generalized PendingJustifications

* finality proof messages

* fixed compilation

* pass verifier to import_finality_proof

* do not fetch remote proof from light import directly

* FinalityProofProvider

* fixed authorities cache test

* restored finality proof tests

* finality_proof docs

* use DB backend in test client

* justification_is_fetched_by_light_client_when_consensus_data_changes

* restore justification_is_fetched_by_light_client_when_consensus_data_changes

* some more tests

* added authorities-related TODO

* removed unneeded clear_finality_proof_requests field

* truncated some long lines

* more granular light import tests

* only provide finality proof if it is generated by the requested set

* post-merge fix

* finality_proof_is_none_if_first_justification_is_generated_by_unknown_set

* make light+grandpa test rely on finality proofs (instead of simple justifications)

* empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* missing trait method impl

* fixed proof-of-finality docs

* one more doc fix

* fix docs

* initialize authorities cache (post-merge fix)

* fixed cache initialization (post-merge fix)

* post-fix merge: fix light + GRANDPA tests (bad way)

* proper fix of empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* fixed easy grumbles

* import finality proofs in BlockImportWorker thread

* allow import of finality proofs for non-requested blocks

* limit number of fragments in finality proof

* GRANDPA post-merge fix

* BABE: pos-merge fix
2019-05-13 11:36:52 +02:00
Pierre Krieger 6c41d0b3ec Isolate the code of ChainSync and turn it into a state machine (#2497)
* Move the is_offline and is_major_syncing logic to service.rs

* Move the ImportQueue to service.rs

* Remove stop() and abort()

* Add some more documentation to sync.rs
2019-05-12 19:23:45 +02:00
Pierre Krieger c3f20bf226 Merge the protocol thread with the network thread (#2475)
* Remove the Incoming enum

* Use tokio in the protocol background thread

* Some internal protocol logic simplifications

* Merge the protocol thread with the network thread

* More the status_sinks logic to Service

* Remove FromNetworkMsg
2019-05-08 13:49:29 +02:00
Pierre Krieger 35b84ae8f6 Remove NetworkService::report_handle (#2462) 2019-05-04 13:58:52 +02:00
Pierre Krieger 8ca343ca8c Add reputation system to network crate (#2454) 2019-05-04 13:51:06 +02:00
DemiMarie-parity fb19684358 BABE signing and verification (#2270)
* Add basic BABE consensus type

* Update core/consensus/babe/slots/Cargo.toml

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Fix parameterization and run `rustfmt`

* Respond to review comments

* Update various Cargo.lock files

* Revert "Update various Cargo.lock files"

This reverts commit af53d7624752a744320e9cbb25749fdd8e6f46d2.

* `BabeSealSignature` → `BabeSeal`

* Move slot code to its own crate

This was highly non-trivial, due to cyclic dependencies.

* Remove redundancy between AuRa and BABE

Some of the code duplication was removed using a macro.

* Fix build error

* Avoid non-`#[doc(hidden)]` re-exports

Also, bump some library versions in `Cargo.toml`.

* Remove dead code in AuRa

* Remove impl_slot macro

It was more trouble than it was worth.

Also, delete useless dependencies on Serde.

* AuRa and BABE need different DB keys

* Bring back `aura::Network`, but deprecate it.

* Improve docs and add `slot_duration` inherent method

* Add docs to `substrate_consensus_aura::SlotDuration`

* Add missing documentation and #![forbid(missing_docs, unsafe_code)]

* Add a #![forbid(missing_docs)]

* Remove dependency of `test-runtime` on `slots`

* Update core/consensus/babe/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Remove wrongly added file

* Fix copyright notice

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Bump `impl_version` and `spec_version`

* Add more code to BABE

Most of it is copied from AuRa code, but at least the initial core is
there.

* Stuck on horrible compiler error message

* add missing files

* Spaces → tabs

* Simplify code

* Fix compilation

This involved fixing dependencies and adding a `Mutex`.

* More work on BABE

* Fix deprecation version

* Fix deprecation version; remove spurious carets

* Fix Cargo.toml

* Implement VRF signing logic

* The import queue code compiles, though it probably doesn’t work.

* Add VRF verification

* Update Cargo.lock

* Update dependencies

* Move test network to sr25519 authority keys

* Fix accidental build bustage

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Working testsuite at last!

The problem was with serialization and deserialization.  Normally,
those functions are generated automatically, but those for `BabeSeal`
had to be written manually.  The hand-written versions were not
correct, however, as shown by the decoder not being able to decode the
output of the encoder.

* Enable BabeSeal::Encode asserts in --release tests

* Bump runtime and dependency versions

* Fix wasm compilation

The wasm build was broken because of a typo in
`core/test-runtime/src/lib.rs`, and missing gates on the `std` feature
in `core/consensus/{aura,babe}/primitives/Cargo.toml`.  Additionally,
improve the quotation in the build scripts.

* Merge Cargo.lock

* Change expected JSON string

The test was also broken on `master`, so I suspect that the test was
incorrect.

* Responded to review

* Remove hard-coded threshold from production code

A hard-coded threshold is now only used in tests.

* Fix swapped doc comments

* Fix unused import warnings

* fix ci error

* fix typo

* Fix spacing in docs

* Minor changes suggested by @joepetrowski

on https://github.com/paritytech/substrate/pull/2372

* Remove unnecessary getters

* fix compile error

* Fix silly unused-variable error

* Improve documentation formatting

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Add issue links

* Revert excess verbosity and #![forbid(warnings)]

* Apply suggestions from code review

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Reformat some comments

* Threshold should depend on number of validators

Also, respond to code review

* Fix silly compilation errors

* Reduce logging verbosity

* Fix missing import
2019-05-02 11:16:54 -04:00
Pierre Krieger 8b7258f850 Simplify the network background thread future (#2419) 2019-04-29 18:57:12 +02:00
Pierre Krieger 3f06fe32f3 Update to libp2p v0.7.0 (#2343)
* Update to libp2p master

* Fix tests

* More tests fixing
2019-04-23 18:46:30 +01:00
Robert Habermeier 1aa6eb9fa8 Polite-grandpa improvements (#2229)
* send neighbor packets in more generic way

* integrate periodic neighbor-packet rebroadcaster

* integrate reporting

* attach callbacks to commit messages for rebroadcasting and reporting

* Tests for commit relay

* crunch up some nice warnings

* exit-scope sub-futures of grandpa

* address small grumbles

* some changes to commit handling
2019-04-16 15:25:46 +08:00
Arkadiy Paronyan e30b362193 Disable gossip for peers running old version (#2245) 2019-04-15 12:18:08 +02:00