Commit Graph

182 Commits

Author SHA1 Message Date
Niklas Adolfsson e330b69cb9 fix(redundant import) (#2582)
Fixes warning: the item `HashSet` is imported redundantly
2019-05-14 18:07:13 +03:00
Pierre Krieger 18f7acce98 Move ProtocolMsg to service.rs (#2560)
* Move ProtocolMsg to service.rs

* Fix line width

* Fix line width again

* Correct whitespace

* Another whitespace correction
2019-05-14 14:26:52 +01:00
Svyatoslav Nikolsky a29fd10859 Do not try to synchronize from light client (#2039)
* do not try to sync from light node

* use Roles' utility methods
2019-05-14 14:41:10 +02:00
Pierre Krieger 6ccdbea01c Move extra_requests.rs as a submodule of sync (#2573) 2019-05-14 12:08:01 +02:00
Pierre Krieger e5b0a98f1a Split SyncContext from protocol Context (#2550) 2019-05-13 21:16:52 +02:00
Pierre Krieger 2724cdac33 Remove the NetworkChan from the OnDemandService (#2561) 2019-05-13 20:55:27 +02:00
Svyatoslav Nikolsky 22586113ea Light GRANDPA import handler (#1669)
* GrandpaLightBlockImport

* extract authorities in AuraVerifier

* post-merge fix

* restore authorities cache

* license

* new finality proof draft

* generalized PendingJustifications

* finality proof messages

* fixed compilation

* pass verifier to import_finality_proof

* do not fetch remote proof from light import directly

* FinalityProofProvider

* fixed authorities cache test

* restored finality proof tests

* finality_proof docs

* use DB backend in test client

* justification_is_fetched_by_light_client_when_consensus_data_changes

* restore justification_is_fetched_by_light_client_when_consensus_data_changes

* some more tests

* added authorities-related TODO

* removed unneeded clear_finality_proof_requests field

* truncated some long lines

* more granular light import tests

* only provide finality proof if it is generated by the requested set

* post-merge fix

* finality_proof_is_none_if_first_justification_is_generated_by_unknown_set

* make light+grandpa test rely on finality proofs (instead of simple justifications)

* empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* missing trait method impl

* fixed proof-of-finality docs

* one more doc fix

* fix docs

* initialize authorities cache (post-merge fix)

* fixed cache initialization (post-merge fix)

* post-fix merge: fix light + GRANDPA tests (bad way)

* proper fix of empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* fixed easy grumbles

* import finality proofs in BlockImportWorker thread

* allow import of finality proofs for non-requested blocks

* limit number of fragments in finality proof

* GRANDPA post-merge fix

* BABE: pos-merge fix
2019-05-13 11:36:52 +02:00
Pierre Krieger 6c41d0b3ec Isolate the code of ChainSync and turn it into a state machine (#2497)
* Move the is_offline and is_major_syncing logic to service.rs

* Move the ImportQueue to service.rs

* Remove stop() and abort()

* Add some more documentation to sync.rs
2019-05-12 19:23:45 +02:00
André Silva 9d23d2a25a Bump master to v2.0.0 (#2514)
* bump crates to v2.0.0

* fix crate authors

* bump node runtime spec_version

* remove unused import for no-std
2019-05-10 16:55:15 +02:00
cheme 0d8379d5d2 Child trie storage proof (#2433)
* proof on child trie

* higher level api for child storage proof

* boilerplate for proof from light fetch

* actually check proof on light fetch

* Do not break former encoding

* tabify

* tabify2

* Add child trie root tx to full_storage_root transaction.

* Shorten long lines.

* Temp rename for audit

* Make full_storage a trait method

* Name back and replace some code with full_storage where it looks fine.

* fix indentations, remove unused import

* flush child root to top when calculated

* impl +1
2019-05-10 14:31:41 +02:00
Svyatoslav Nikolsky 42fa5f6209 Fix deadlock in tests that are using with_gossip (#2533)
* fixed gossip related tests deadlock

* cleaning sync tests log

* fixed typo in logs

* send GRANDPA messages without holding validator lock

* Revert "fixed gossip related tests deadlock"

This reverts commit d3fe55e45e32b7ea2e9f05f1c511972c708209ad.
2019-05-10 14:14:42 +02:00
André Silva 32fdeed21c core: use current_thread tokio runtime to poll network to proto channel (#2531) 2019-05-10 10:26:58 +01:00
Pierre Krieger c3f20bf226 Merge the protocol thread with the network thread (#2475)
* Remove the Incoming enum

* Use tokio in the protocol background thread

* Some internal protocol logic simplifications

* Merge the protocol thread with the network thread

* More the status_sinks logic to Service

* Remove FromNetworkMsg
2019-05-08 13:49:29 +02:00
Svyatoslav Nikolsky a63635b952 Fix light connection issues (#2499)
* ignore light nodes in ConsensusGossip

* fixed method name

* temporary disabled penalty when block is announced

* remove traces of BLOCK_ANNOUNCE_REPUTATION_CHANGE
2019-05-08 12:24:20 +02:00
Pierre Krieger 3935b247dd Move construct_simple_protocol! to appropriate module (#2476) 2019-05-06 09:37:25 +02:00
Pierre Krieger 35b84ae8f6 Remove NetworkService::report_handle (#2462) 2019-05-04 13:58:52 +02:00
Pierre Krieger 8ca343ca8c Add reputation system to network crate (#2454) 2019-05-04 13:51:06 +02:00
Marcio Diaz 18ec2d14d0 Fix grandpa issue for InvalidViewChanges (#2468)
* fix: add message_allowed to send_topic to avoid InvalidViewChanges

* fix: this should be strict

* fix typo

* fix: simplify
2019-05-04 01:16:14 +02:00
DemiMarie-parity fb19684358 BABE signing and verification (#2270)
* Add basic BABE consensus type

* Update core/consensus/babe/slots/Cargo.toml

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Fix parameterization and run `rustfmt`

* Respond to review comments

* Update various Cargo.lock files

* Revert "Update various Cargo.lock files"

This reverts commit af53d7624752a744320e9cbb25749fdd8e6f46d2.

* `BabeSealSignature` → `BabeSeal`

* Move slot code to its own crate

This was highly non-trivial, due to cyclic dependencies.

* Remove redundancy between AuRa and BABE

Some of the code duplication was removed using a macro.

* Fix build error

* Avoid non-`#[doc(hidden)]` re-exports

Also, bump some library versions in `Cargo.toml`.

* Remove dead code in AuRa

* Remove impl_slot macro

It was more trouble than it was worth.

Also, delete useless dependencies on Serde.

* AuRa and BABE need different DB keys

* Bring back `aura::Network`, but deprecate it.

* Improve docs and add `slot_duration` inherent method

* Add docs to `substrate_consensus_aura::SlotDuration`

* Add missing documentation and #![forbid(missing_docs, unsafe_code)]

* Add a #![forbid(missing_docs)]

* Remove dependency of `test-runtime` on `slots`

* Update core/consensus/babe/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Remove wrongly added file

* Fix copyright notice

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Bump `impl_version` and `spec_version`

* Add more code to BABE

Most of it is copied from AuRa code, but at least the initial core is
there.

* Stuck on horrible compiler error message

* add missing files

* Spaces → tabs

* Simplify code

* Fix compilation

This involved fixing dependencies and adding a `Mutex`.

* More work on BABE

* Fix deprecation version

* Fix deprecation version; remove spurious carets

* Fix Cargo.toml

* Implement VRF signing logic

* The import queue code compiles, though it probably doesn’t work.

* Add VRF verification

* Update Cargo.lock

* Update dependencies

* Move test network to sr25519 authority keys

* Fix accidental build bustage

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Working testsuite at last!

The problem was with serialization and deserialization.  Normally,
those functions are generated automatically, but those for `BabeSeal`
had to be written manually.  The hand-written versions were not
correct, however, as shown by the decoder not being able to decode the
output of the encoder.

* Enable BabeSeal::Encode asserts in --release tests

* Bump runtime and dependency versions

* Fix wasm compilation

The wasm build was broken because of a typo in
`core/test-runtime/src/lib.rs`, and missing gates on the `std` feature
in `core/consensus/{aura,babe}/primitives/Cargo.toml`.  Additionally,
improve the quotation in the build scripts.

* Merge Cargo.lock

* Change expected JSON string

The test was also broken on `master`, so I suspect that the test was
incorrect.

* Responded to review

* Remove hard-coded threshold from production code

A hard-coded threshold is now only used in tests.

* Fix swapped doc comments

* Fix unused import warnings

* fix ci error

* fix typo

* Fix spacing in docs

* Minor changes suggested by @joepetrowski

on https://github.com/paritytech/substrate/pull/2372

* Remove unnecessary getters

* fix compile error

* Fix silly unused-variable error

* Improve documentation formatting

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Add issue links

* Revert excess verbosity and #![forbid(warnings)]

* Apply suggestions from code review

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Reformat some comments

* Threshold should depend on number of validators

Also, respond to code review

* Fix silly compilation errors

* Reduce logging verbosity

* Fix missing import
2019-05-02 11:16:54 -04:00
Pierre Krieger 8b7258f850 Simplify the network background thread future (#2419) 2019-04-29 18:57:12 +02:00
Pierre Krieger ebbd6c8ec4 Split Context::send_message into multiple methods (#2409)
* Split Context::send_message into multiple methods

* Apply suggestions from code review

Co-Authored-By: tomaka <pierre.krieger1708@gmail.com>
2019-04-29 13:54:19 +02:00
Michael Müller a0e0d9b03d Fix typos and markdown (#2388)
* Fix typos

* Align properly

* Update core/consensus/slots/src/lib.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Update core/network/src/test/mod.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Update core/finality-grandpa/src/communication/mod.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Update core/consensus/common/src/import_queue.rs

Co-Authored-By: cmichi <mich@elmueller.net>
2019-04-26 14:09:13 +02:00
Pierre Krieger f3df7250e9 Remove the multiplexed networking system (#2373)
* Remove the multiplexed networking system

* Rename BackCompat to Normal

* Remove CustomMessageId

* Fix tests
2019-04-25 11:39:57 +02:00
Pierre Krieger 3f06fe32f3 Update to libp2p v0.7.0 (#2343)
* Update to libp2p master

* Fix tests

* More tests fixing
2019-04-23 18:46:30 +01:00
Tomasz Drwięga 08fda211d8 Fix MultiSigner, simplify tests (#2033)
* Fix MultiSigner, use `into_signed_tx`

* Rebuild.
2019-04-23 17:18:32 +01:00
Pierre Krieger 648dcc2728 Add warnings about breaking changes in network (#2348) 2019-04-23 16:34:47 +02:00
Robert Habermeier 1aa6eb9fa8 Polite-grandpa improvements (#2229)
* send neighbor packets in more generic way

* integrate periodic neighbor-packet rebroadcaster

* integrate reporting

* attach callbacks to commit messages for rebroadcasting and reporting

* Tests for commit relay

* crunch up some nice warnings

* exit-scope sub-futures of grandpa

* address small grumbles

* some changes to commit handling
2019-04-16 15:25:46 +08:00
Arkadiy Paronyan e30b362193 Disable gossip for peers running old version (#2245) 2019-04-15 12:18:08 +02:00
Kian Peymani dcb1cafe2a Remove unused imports/ doc comments. (#2276)
* Remove unused imports/ doc comments

* Better doc for node template.
2019-04-14 12:42:45 +02:00
Svyatoslav Nikolsky 5725e25448 fix race in sync tests (#2259) 2019-04-12 09:32:00 +02:00
Andrew Jones 7f59cdb900 Replace error-chain for client error (#2231)
* WIP: convert client error

* Remove error_chain for client error

* Ignore tx-pool error deprecation warning

* Update Cargo.lock files

* Fix tests

* Increment impl_version

* Derive From impls, remove allow(missing_docs)

* Remove space

* Remove redundant into()s

* Blockchain Error source

* Bump impl version
2019-04-11 21:33:43 +02:00
Arkadiy Paronyan 0b2c3ae860 Apply negative rating on explicit ban (#2247)
* Apply negative rating on explicit ban

* Update core/network/src/service.rs

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>
2019-04-10 23:34:41 +02:00
Gregory Terzian ed3ae4ac39 remove clone bound on specialization in testnet factory (#2157) 2019-04-08 13:17:00 +02:00
Marek Kotewicz 1142bcde97 simplification of peerset api (#2123)
* Introduction of PeersetHandle

* integrate PeersetHandle with the rest of the codebase

* fix compilation errors

* more tests for peerset, fixed overwriting bug in add_reserved_peer

* Slots data structure and bugfixes for peerset

* bend to pressure

* updated lru-cache to 0.1.2 and updated linked-hash-map to 0.5.2

* peerset discovered list is now a LinkedHashMap

* fix review suggestions

* split back Peerset and PeersetHandle

* test for Peerset::discovered

* applied review suggestions

* fixes to peerset::incoming

* peerset disconnects are all instantaneous

* instantaneous drop in peerset finished

* Peerset::set_reserved_only can also reconnect nodes

* Peerset scores cache uses lru-cache

* remove redundant function call and comment from Peerset::on_set_reserved_only

* add_peer returns SlotState enum

* apply review suggestions

* is_reserved -> is_connected_and_reserved
2019-04-04 19:40:40 +02:00
Robert Habermeier bb95e7d6a2 Initial version of the polite-grandpa networking protocol (#2110)
* Consensus status packet

* Allow for repropagation after status

* More generic gossip

* add a basic view struct and gossip module

* move gossip stuff to the gossip module

* integrate view into gossip

* some reshuffling

* alter rules for keeping one commit at a time in view

* Allow sending addressed messages

* don't cast outgoing votes if we know that we voted before

* Handle one hop messages

* initial run at polite grandpa

* build WASM

* handle neighbor messages

* refactor validator's internals into an Inner struct

* gossip only knows to keep or discard messages. optimize should_send_to

* Periodic rebroadcast

* implement `should_send_to` and message_expired

* track peers' best received commit height

* Pass peer id to topic steam

* kill rebroadcasting network

* Notify about existing peers

* clean up network APIs a bunch

* implement gossip::send_message for direct messages

* refactor network trait

* implement gossip::send_message for direct messages

* get all non set-change tests passing

* treat unknown rebroadcasts as broadcasts

* get all other main tests passing

* remove unimplemented test

* everything compiles

* treat unknown rebroadcasts as broadcasts

* Rebradcast interval

* Apply suggestions from code review

Style

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Style

* some module docs

* address some grumbles + docs

* allow rebroadcast every few minutes

* send_topic && generic context

* some tests for view change

* more grumbles & tests

* use send_peer
2019-04-02 18:09:05 +02:00
Niklas Adolfsson 11da395553 style(on_demand): remove needless wildcards (#2171) 2019-04-02 16:54:02 +02:00
thiolliere 7e3b2d464a skip phantom for decl_event (#2158)
* use parity-codec-derive from parity-codec everywhere

* codec v3.2 -> v3.3

* skip phantom when deriving event

* fix decl_module call

* update impl version of node + builds
2019-04-02 12:04:23 +02:00
Stanislav Tkach cbfc36b39f Move authorities interface from Core to consensus (#1412)
* Move authorities interface from Core to consensus

f

* notify all caches of block insert + create with up-to-date best_fin

* merged authorities_are_cached from light_grandpa_import2

* Add ProvideCache trait

* Create helper function for 'get_cache'

* Fix some formatting

* Bump impl version

* Resolve wasm conflicts

* Apply review comments

* Use try_for_each

* Move authorities interface from Core to consensus

f

* notify all caches of block insert + create with up-to-date best_fin

* merged authorities_are_cached from light_grandpa_import2

* Add ProvideCache trait

* Create helper function for 'get_cache'

* Fix some formatting

* Bump impl version

* Resolve wasm conflicts

* Apply review comments

* Use try_for_each

* Move authorities interface from Core to consensus

f

* notify all caches of block insert + create with up-to-date best_fin

* merged authorities_are_cached from light_grandpa_import2

* Add ProvideCache trait

* Create helper function for 'get_cache'

* Fix some formatting

* Bump impl version

* Resolve wasm conflicts

* Apply review comments

* Use try_for_each

* Increment impl_version

* Update lib.rs
2019-03-29 17:41:22 +01:00
joe petrowski 0ddcbf747f Convert all UK spelling to US (#2138)
* all the ise

* forgot a misspelling

* a few more replacements

* bump impl

* rollback and fixes

* bump impl again

* Add aliases for RPC

* Update on_demand.rs
2019-03-29 14:11:45 +01:00
DemiMarie-parity a10e86ba5a Refactor DigestItem (#2108)
* Add `start_aura2`.

* .gitignore patch conflict files

and remove one that accidentally got committed

* Fix build

The tests still don’t work.

* Fix compilation errors

* Fix compile errors (again)

* Try (and fail) to fix tests

* Properly deserialize data

Previously, `DigestItem::Consensus` had no separate `DigestItemType`,
so it did not get properly serialized and deserialized.

* Add extra debug logging.  Always allow old seals.

A `RUST_LOG=substrate_aura_consensus cargo test --all -- --nocapture \
tests::authoring_blocks` revealed that old seals were being and
rejected, causing the test to hang.  As a temporary debug measure, allow
old seals unconditionally, so that CI can test if this fixes the
problem.

* Forcibly disable rejection of old seals

* Use old trait, but newer serialization

The old trait for `CompatibleDigestItem` actually worked.  By changing
its implementation, one can ensure that all *new* seals have the modern
form, but *legacy* seals are still decoded correctly.

* Bump impl version

* Squash spurious deprecation warning

`rustc` should not be emitting a deprecation warning in deprecated
code, but it does, so silence it.

* Rip out unused Cargo feature

* Move AURA to aura_primitives

* Respond to code review

* Wrap overly-long line

* Reduce logging verbosity and add target

* Add dependency on `sr-primitives` to `aura_primitives`

* Fix build

It failed with a message about Cargo.lock being out of date.

* core: aura: rename aura engine id const

* core: aura: remove superfluous logging

* core: primitives: add removed semicolons

* core: aura: remove unused import

* core: network: style fix

* runtime: update wasm blobs

* runtime: bump impl_version

* core: primitives: tag all DigestItemType variants explicitly
2019-03-29 12:15:23 +00:00
André Silva d17e34b3c8 core: network: reduce log level of peer drop messages (#2125) 2019-03-27 16:49:19 +01:00
Pierre Krieger 6e394464b8 Replace NodeIndex with PeerId everywhere (#2077)
* Replace NodeIndex with PeerId

* Fix tests

* More test fixing

* Whitespace
2019-03-23 10:34:28 +01:00
André Silva 25ec793e35 Fix handling of justifications (#2086)
* util: fork-tree: check predicate first while traversing tree

* core: sync: keep track of justifications sent to the import queue

* core: grandpa: verify authority set changes dependencies

* core: fork-tree: add more tests

* core: grandpa: extend enacts_standard_change tests
2019-03-22 19:07:24 +01:00
Arkadiy Paronyan 704d2a49c7 Don't sync on top of pruned blocks (#2079)
* Don't sync to pruned blocks

* Typo

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>
2019-03-22 16:27:50 +01:00
Pierre Krieger 90c6f85db5 Add a substrate-peerset crate (#2042)
* Add a substrate-peerset crate

* Some adjustements

* More adjustements

* Use a temporary libp2p branch

* Add back-off mechanism

* Fix RPC tests

* Some adjustements

* Another libp2p bugfix

* Do a round-robin in the peerset

* Use a real dependency instead of a patch for libp2p

* Initialize reserved nodes correctly

* Better diagnostic for no address

* Don't allocate slots if in reserved only

* Ban node on dial failure

* Fix indentation
2019-03-21 14:02:28 +01:00
Arkadiy Paronyan b31bcdfcc3 Keep peer info up to date (#2067)
* Keep peer info up to date

* Docs

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>
2019-03-21 12:55:30 +01:00
André Silva e56a5cd00b Minor gossip changes (#2038)
* core: gossip: don't expire messages based on time

* core: gossip: allow forcing resend of gossip messages

* core: grandpa: fix tests
2019-03-19 12:13:05 +01:00
Pierre Krieger 002143d0a2 Remove support for multiple network protocols (#2005)
* Remove support multiple network protocols

* Address concerns

* Add back debug_asserts
2019-03-19 11:56:56 +01:00
Roman Borschel 57387ef585 libp2p-0.5.0 (#1971)
* Update libp2p. Add support for ed25519 node (network) keys.

  * Update networking to the changes from https://github.com/libp2p/rust-libp2p/pull/972.
  * Add support for using ed25519 keys for libp2p networking.
  * Add support for reading libp2p secret keys from (external) files.

* Adapt to changes from https://github.com/libp2p/rust-libp2p/pull/992

* More tests.

* Cosmetics

* Deduplicate tests.

* Remove quickcheck from tests that don't use extra random inputs.

* Remove quickcheck.

* Swap new/default impls for NetworkConfiguration.

* Use libp2p-0.5.0 from crates.io.

* Post-rebase update.

* Remove unnecessary wildcard pattern.

* Combine two overlapping tests.
2019-03-19 11:16:25 +01:00
thiolliere d743a8b71f Implement parameterisable modules (#1800)
* first implementation

* remove done comment

* origin done

* impl log for instance

* impl inherent for instance

* Fix wasm build + full example build

this requires parity codec implements codec for core::marker::PhantomData

* patch parity-codec link to github branch

* improve internal names and fix instance prefix

* Fix in macros

* add test modules for support

this allow to test for construct_runtime as well.

The reason to have put that in another crate is:
* if we put test in `tests/` dir of srml/support then decl_storage fails to get
  srml-support access because it believes it is inside srml-support
  crate and so derive access to `quote!{ crate }` but this is wrong
  (and I don't see any way to prevent that, and it only bother us so I
  don't think that matters that much)
* if we put test inside lib.rs then contruct_runtime cannot be used
  because it call some macros that are defined with macros
  (decl_outer_event and decl_outer_origin) and thus rustc complains.

* defaultinstance to its own struct to avoid errors

* enforce <T, I> for Event and Config, impl test

* add origin, log, inherent to test

* test more code generation

* basic storage test

* fix typo

* rename a few imports and field

* delete wip test in example and runtime

* change default prefix to make it backward compatible with test

* rename Instance to I and Instantiable to Instance

note: the name of generic parameter I is only enforce by decl_module!
and this could be rewritten

* doc

* clean old TODOs

* update parity-codec to 3.2

* update node impl version + builds

* fix warning

* fix unrelated grandpa test

* refactor code
2019-03-15 19:25:18 +01:00