Commit Graph

281 Commits

Author SHA1 Message Date
André Silva 6876c50af8 sync: don't return on fork tree revert (#3444) 2019-08-20 20:32:26 +02:00
André Silva 68fd94028f grandpa: fix finalization race condition (#3437)
* network: handle safe fork_tree::revert errors

* grandpa: deal with race conditions on finality

* network: return on fork_tree revert
2019-08-19 13:40:36 +02:00
Svyatoslav Nikolsky 3825a21bac Fix light client synchronization on master (#3301)
* value ranges in consensus cache

* skip values in cache

* read epoch0 + epoch1 data from genesis in babe

* sync authorities + session validators at genesis

* removed some debug printlns

* fixed cache encoding

* Revert "skip values in cache"

This reverts commit ce451c32823aaa4b67d99ca5b58f1bf3984df4db.

* Revert "value ranges in consensus cache"

This reverts commit 9062f9434cddd14a01275ddbfcd904b04282e63b.

* get rid of cache::AUTHORITIES in Babe

* cleaning up

* cleaning up

* update spec version

* lost changes

* fixed tests

* Update node/runtime/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* fix once-per-block condition

* fix standalone babe + temp_storage in BuildGenesis

* fix benhes compilation

* fixed comment

* re-added light nodes to integration tests

* finalize_with_ancestors from extra_requests

* post-merge fix

* aaand removed debug code

* (another one)

* fix warn in logs (do not call ForkTree::finalize twice for the same block)

* sync digest.next_authorities with actual next authorities

* more docs

* reverting all commits affecting storage

* also remove keys from babe trait

* fixed warnings

* post-merge fixes

* reverted some redundant changes

* reverted more changes
2019-08-16 11:51:49 +02:00
André Silva 6bc72cea62 sync: fix peer common number on block announce (#3410) 2019-08-15 20:47:09 +02:00
Pierre Krieger be732a2d68 Update to libp2p 0.12 (#3408) 2019-08-15 16:41:39 +02:00
Robert Habermeier 6a7a222882 add some docs for consensus-gossip (#3405)
* add some docs for consensus-gossip

* formatting adjustments

* authorities -> nodes

* Update core/network/src/protocol/consensus_gossip.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>
2019-08-15 16:41:23 +02:00
Michael Müller 7f764dbd37 Replace config field with default value (srml/im-online) (#3388)
* Fix whitespacing

* Replace configurable fields with default values

* Bump version

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix more whitespacing

* Fix style
2019-08-13 20:16:02 +02:00
Max Inden 83052fe573 core/network: Surface peerset's set_priority_group in NetworkService (#3376)
`PeerSetHandle.set_priority_group` allows modifying a priority group by
group identifier. With this commit the function can be accessed through
`NetworkService`.

This is need in order for a validator to connect to as many other
validators as configured without reserving a specific connection slot
for them.
2019-08-12 19:23:59 +02:00
Robert Habermeier bf32626373 downgrade logging of malformed packets to debug (#3367) 2019-08-12 16:13:59 +02:00
Gavin Wood 14bb115193 Scale trait and move to u32 blocknumbers (#3357)
* Scale trait and move to u32 blocknumbers.

* Fixes

* Cleanups

* Update node/runtime/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix up some of the factory stuff.

* Update core/sr-primitives/src/traits.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Move Nonce/Index to u32 (#3361)

* Force a non-borked version of upstream crate

* Line lengths and runtime version bump
2019-08-11 15:03:06 +02:00
Pierre Krieger a29f5763a7 Add a small ban when we get disconnected from a node (#3325) 2019-08-09 16:25:28 +01:00
André Silva 59c88cb2f3 node: setup disabled grandpa properly (#3356) 2019-08-09 16:53:44 +02:00
Arkadiy Paronyan b29277239a Improve console output (#3326)
* Cache idle state

* Display import error by default
2019-08-08 09:34:09 +02:00
Pierre Krieger 6396901481 Reintroduce mplex (reverts #3312) (#3328)
This reverts commit f11291cd9a.
2019-08-08 09:32:07 +02:00
Arkadiy Paronyan c0fd256c01 Fixed test block conditions (#3332) 2019-08-08 01:01:16 +02:00
Pierre Krieger c32d9a17e0 Rename CustomProto to LegacyProto (#3313) 2019-08-07 21:22:27 +02:00
Pierre Krieger 97febf4c30 Make Verifier::verify mutable (#3165)
* Make Verifier::verify mutable

* Fix GrandPa tests

* Fix doctest

* Fix more doctests
2019-08-07 21:21:44 +02:00
Pierre Krieger f11291cd9a Remove support for mplex (#3312) 2019-08-07 21:08:21 +02:00
Gavin Wood 1a524b8207 Refactor key management (#3296)
* Add Call type to extensible transactions.

Cleanup some naming

* Merge Resource and BlockExhausted into just Exhausted

* Fix

* Another fix

* Call

* Some fixes

* Fix srml tests.

* Fix all tests.

* Refactor crypto so each application of it has its own type.

* Introduce new AuthorityProvider API into Aura

This will eventually allow for dynamic determination of authority
keys and avoid having to set them directly on CLI.

* Introduce authority determinator for Babe.

Experiment with modular consensus API.

* Work in progress to introduce KeyTypeId and avoid polluting API
with validator IDs

* Finish up drafting imonline

* Rework offchain workers API.

* Rework API implementation.

* Make it compile for wasm, simplify app_crypto.

* Fix compilation of im-online.

* Fix compilation of im-online.

* Fix more compilation errors.

* Make it compile.

* Fixing tests.

* Rewrite `keystore`

* Fix session tests

* Bring back `TryFrom`'s'

* Fix `srml-grandpa`

* Fix `srml-aura`

* Fix consensus babe

* More fixes

* Make service generate keys from dev_seed

* Build fixes

* Remove offchain tests

* More fixes and cleanups

* Fixes finality grandpa

* Fix `consensus-aura`

* Fix cli

* Fix `node-cli`

* Fix chain_spec builder

* Fix doc tests

* Add authority getter for grandpa.

* Test fix

* Fixes

* Make keystore accessible from the runtime

* Move app crypto to its own crate

* Update `Cargo.lock`

* Make the crypto stuff usable from the runtime

* Adds some runtime crypto tests

* Use last finalized block for grandpa authority

* Fix warning

* Adds `SessionKeys` runtime api

* Remove `FinalityPair` and `ConsensusPair`

* Minor governance tweaks to get it inline with docs.

* Make the governance be up to date with the docs.

* Build fixes.

* Generate the inital session keys

* Failing keystore is a hard error

* Make babe work again

* Fix grandpa

* Fix tests

* Disable `keystore` in consensus critical stuff

* Build fix.

* ImOnline supports multiple authorities at once.

* Update core/application-crypto/src/ed25519.rs

* Merge branch 'master' into gav-in-progress

* Remove unneeded code for now.

* Some `session` testing

* Support querying the public keys

* Cleanup offchain

* Remove warnings

* More cleanup

* Apply suggestions from code review

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* More cleanups

* JSONRPC API for setting keys.

Also, rename traits::KeyStore* -> traits::BareCryptoStore*

* Bad merge

* Fix integration tests

* Fix test build

* Test fix

* Fixes

* Warnings

* Another warning

* Bump version.
2019-08-07 20:47:48 +02:00
Pierre Krieger b5b1c2a4d8 Rename on_demand to light_dispatch and various minor changes (#3315)
* Rename on_demand to light_server

* Small docs improvement

* Rename on_block_announce to update_best_number

* More minor documentation

* Light server -> light dispatch

* is_light_rq_response -> is_light_response
2019-08-07 12:40:49 +02:00
thiolliere 4ed67e03a4 Update to parity-scale-codec (#3232)
* WIP: update codec

* WIP

* compiling

* WIP

* rename parity-scale-codec to codec

* WIP

* fix

* remove old comments

* use published crates

* fix expected error msg

* bump version

* fmt and fix

* remove old comment

* fix wrong decoding impl

* implement encode like for structures

* undo removal of old pending changes

* trailingzeroinput

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* update codec

* fmt

* version is 1.0.0

* show more error

* fmt
2019-08-06 19:36:23 +02:00
Max Inden c9c0ad4756 core/network: Fix bad import %s/runtime_primitives/sr_primitives/ (#3236)
* core/network: Fix bad import %s/runtime_primitives/sr_primitives/

* Cargo.lock: Update consensus-common-primitives parity-codec version

* core/consensus/common: Remove obsolete package path
2019-07-29 15:55:57 +02:00
Kian Paimani 79feb23a22 Refactor sr_primitives. (#3214)
* refactor sr_primitives.

* Fix try build error.

* Line-width

* Ui test.

* Final fixes.

* Fix build again.

* bring back ui test.

* Fix unsigned import.

* Another ui fix.

* Also refactor substrate-primitives

* Fix benchmarks.

* Fix doc test.

* fix doc tests
2019-07-29 14:43:53 +02:00
Pierre Krieger 8c919e031b Stop being generic over the message (#3186) 2019-07-29 09:29:06 +02:00
Pierre Krieger 9370a4a6b6 Clean up the basic-authorship crate (#3206)
* Switch consensus-common to new futures

* Fix tests

* More tests fixing

* Make Proposer, OnSlot and SyncOracle mut

* Make the Environment mut as well

* Fix test

* Fix Babe tests

* Babe fixes

* Fix CLI service tests

* Fix Babe tests

* Remove unnecessary trait bound

* Inline the code of BlockBuilder and AuthoringApi

* Remove warning lint

* Bounds simplification

* Imports simplification

* Don't panic on bad generated block

* Code style

* Add doc example

* Remove dependency on aura

* Order dependencies alphabetically

* Minor style
2019-07-27 20:24:51 -04:00
Svyatoslav Nikolsky 80472956f8 fixed on_demand test (#3200) 2019-07-25 11:49:30 +02:00
Pierre Krieger b31dcdf342 Switch consensus crates to new futures (#3146)
* Switch consensus-common to new futures

* Fix tests

* More tests fixing

* Fix Babe tests

* Fix Babe tests
2019-07-25 10:55:50 +08:00
Roman Borschel 343f4a2a50 libp2p-next (#3076)
* Changes for the next libp2p release:

  * Updates to the Kademlia APIs.
  * Updated imports due to the extracted libp2p-swarm crate.
  * ...

Still pending at least the following:

  * rust-libp2p/#1189
  * rust-libp2p/#1191
  * rust-libp2p/#1194

* Use Quorum::One.

The previous choice was apparently arbitrary.

* Use libp2p-0.11 from crates.io. Address feedback.

* Correct imports after merge.
2019-07-24 17:32:25 +02:00
Pierre Krieger c8dab27f35 Report to the PSM when a severe protocol error happens (#3161) 2019-07-24 00:43:50 -04:00
Pierre Krieger 8aaf0808dd Fix mDNS not working (#3181)
* Fix mDNS not working

* Fix WASM build
2019-07-23 14:03:21 +02:00
DemiMarie-parity f78a780790 BABE Epochs (#3028)
* Add `epoch` field to `SlotInfo`

* Add slot calculations

* More work on epochs in BABE

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Typo: `/` not `%` for division

* Delete useless `LastSlotInEpoch::put(false)`

* Bump `spec_version`

* Make test suite pass again

* Implement BABE epoch randomness signing

* Try to fix compilation

Currently causes a stack overflow in the compiler

* Fix rustc stack overflow

* Add missing `PartialEq` and `Eq` implementations

* Fix compile errors in test suite

* Another silly compile error

* Clone `epoch`

* Fix compile error in benchmarks

* Implement `clone` for `Epoch`

* Merge master

* AUTHORING TEST PASSES!!!

* Fix compilation

* Bump `spec_version`

* Fix compilation

* Fix compilation (again)

* Remove an outdated FIXME

* Fix run.sh and move it to scripts/

* Delete commented-out code

* Fix documentation

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Fix BABE initialization and refactor

* Respond to review

* typo

* Remove useless data in `CheckedHeader::Deferred`

* Remove `slot_number` from Epoch

It is not needed, and only served to waste space and cause confusion.

* Remove epoch from BABE digests

* Move digest.rs to primitives

* Fix incorrect warning names

* Fix compile error

* Consistent field naming for BABE digests

* More compiler error fixex

* Unbound variable

* more compile errors

* another compile error

* Fix compile errors in runtime

* another compile error

* Another compile error

* Fix wasm build

* missing import

* Fix more compile errors

* yet another compile error

* compile fix in test runtime

* Fix and simplify the BABE runtime

The BABE runtime was massively overcomplicated and also wrong.  It
assumed it needed to:

1. delay new authorities taking effect until the next epoch
2. not delay emitting `Consensus` digests to mark epoch changes

However, the first is handled by the `srml_session` crate, and the
second is flat-out incorrect: `Consensus` digests take effect
immediately.  Furthermore, `srml_babe` tried to duplicate the
functionality of `srml_session::PeriodicSession`, but did it both
clumsily and incorrectly.  Fortunately, the new code is simpler and far
more likely to be correct.

* Use `system` to get the test authorities

The genesis block used by tests defines no authorities.  Only the test
suite is affected.

* Fix test runtime impl for BabeApi::epoch() with std

* Fix compilation

* Cached authorities are in the form of an epoch

not a `Vec<AuthorityId>`.

* `slots_per_epoch` is not fixed in general

The BABE code previously assumed `slots_per_epoch` to be a constant,
but that assumption is false in general.  Furthermore, removing this
assumption also allows a lot of code to go away.

* fix compile error

* Implement epoch checker

* Fix runtime compilation

* fork-tree: add method for finding a node in the tree

* babe: register epoch transitions in fork tree and validate them

* fork-tree: add method for arbitrary pruning

* Expose the queued validator set to SRML modules

BABE needs to know not only what the current validator set is, but also
what the next validator set will be.  Expose this to clients of the
session module.

* Bump hex-literal

Hopefully this will fix the panic

* babe: prune epoch change fork tree on finality

* babe: validate epoch index on transition

* babe: persist epoch changes tree

* Fix compile error in tests

* Fix compile error in tests

* Another compile error in tests

* Fix compilation of tests

* core: move grandpa::is_descendent_of to client utils

* babe: use is_descendent_of from client utils

* babe: extract slot_number from pre_digest in import_block

* Move BABE testsuite to its own file

* Initial part of test code

* Missing `WeightMultiplierUpdate` in test-runtime

* bump `spec_version`

* Add a test that a very bogus is rejected

* Run the tests again

* Fix compiler diagnostics

* Bump `spec_version`

* Initial infrastructure for mutation testing

* Mutation testing of block import

* babe: revert epoch changes in case of block import error

* babe: fix logging target

* babe: BabeBlockImport doesn't box inner BlockImport

* babe: fix epoch check in block import

* babe: populate authorities cache on block authorship

* babe: remove unused functions

* babe: use RANDOMNESS_LENGTH const

* babe: remove unneeded config parameters

* core: revert change to hex dependency version

* cleanup gitignore

* babe: add docs to aux_schema

* babe: remove useless drops in tests

* babe: remove annoying macos smart quotes

* fork-tree: docs

* fork-tree: add tests

* babe: style

* babe: rename randomness config variable

* babe: remove randomness helper function

* babe: style fixes

* babe: add docs

* babe: fix tests

* node: bump spec_version

* babe: fix tests
2019-07-23 10:36:16 +02:00
Pierre Krieger 2edeef5825 ImportBlock -> BlockImportParams (#3158) 2019-07-21 18:43:02 -04:00
Michael Müller c70b81444a Introduce srml/im-online (#3079)
* Fix grammar and typo

* Extend network service

* Extend offchain API

* Support creating unsigned UncheckedExtrinsic

* Introduce srml/im-online

* Bump impl and spec version

* Fix web-wasm test

* Apply suggestions from code review

Remove parity-multiaddr dependency

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Replace transmute with from_raw_parts

* Replace PeerId.to_string() with .to_base58()

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update Cargo.lock

* Bump impl and spec version (again)

It was updated in master in the meantime.

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Address comments

* Add public function is_online_in_current_session()

* Bump spec_version

* Fix doc tests

* Improve comments

* Remove superfluous line

* Name parameters consistently

* Implement comments

* Switch From to TryFrom

* Use Vec instead of HashSet

* Fix tests

* Revert me: local testing

* Fix check if already sent during session

We gossip each session, hence we need to check
if already sent in this session (not era).

* Fix typos

* Consistent terminology

* Revert "Revert me: local testing"

This reverts commit 73fbc29ff3e5ed71d99436318260b4f007e837f4.

* Introduce IsMember trait

* Implement misc comments

* Remove unused function

* Fix test

* Fix external_addresses being written

* Fix test

* Add necessary trait bound

* Do not increment version

* Update lib.rs
2019-07-20 09:19:44 +08:00
Pierre Krieger 1e77717b26 Switch the network timers to new futures (#3117) 2019-07-18 18:52:19 -04:00
Pierre Krieger a22bb71029 Consistency with the Block trait name (#3129)
* Consistency with the Block trait name

* Line widths
2019-07-16 15:32:46 +01:00
Pierre Krieger babfc44736 Make mDNS part of DiscoveryBehaviour (#3116)
* Make mDNS part of DiscoveryBehaviour

* Fix tests

* Address concern
2019-07-16 08:49:44 +09:00
Max Inden 515a56af4a core/network: Remove self mutability on put and get value (#3118)
Make `NetworkService.{get,put}_value` only take a self reference instead
of a mutable self reference. When retrieving `NetworkService` via
`NetworkWorker`, one only gets an immutable reference.
2019-07-16 08:49:27 +09:00
Pierre Krieger 5bd806bd9b Improvements to the import queue (#3101)
* Remove block_imported

* Move blocks results processing to sync

* Remove methods from Link

* Better errors

* Allow cancelling the import queue

* Restore the import trace

* Fix network tests

* Line widths

* Use has_error instead

* Minor style
2019-07-14 13:22:32 +01:00
Pierre Krieger 7ae6556a02 Switch the peerset to new futures (#3108)
* Switch the peerset to new futures

* Fuse the peerset rx
2019-07-13 10:55:01 +02:00
Toralf Wittner 9ee79d5c5e Remove sync::Context trait. (#3105)
Instead of passing a context around to each method, thereby introducing
side-effecting I/O actions everywhere, with this PR `sync::ChainSync`
only contains state which is updated by invoking various callback
methods (`on_*`) and actionable items are returned as regular results
from method calls, often iterators yielding requests that should be
issued to peers. It is up to the caller to handle these in an
appropriate way, currently `protocol` will send those as messages.
2019-07-12 20:37:38 +02:00
Pierre Krieger bf2551a854 Switch the client to new futures (#3103)
* Switch the client to new futures

* No need for compat in the client

* Fix client tests

* Address review
2019-07-11 16:58:30 +02:00
Pierre Krieger efed2e3098 Remove set_finality_proof_request_builder (#3087)
* Remove set_finality_proof_request_builder

* Fix Babe

* Fix Grandpa

* Fix service doctests
2019-07-11 09:44:32 +02:00
André Silva 8f81bd90e7 network: don't announce genesis block (#3081) 2019-07-10 13:36:27 +02:00
Max Inden c2491ed20b core/network: Log when ignoring genesis block announcement (#3078)
Instead of logging the fact that a generic invalid block announcement is
ignored, log that the given block is the genesis block.
2019-07-10 10:12:36 +01:00
Pierre Krieger 01060df459 Some documentation for discovery.rs (#3024)
* Some documentation for discovery.rs

* Typos

* Address concerns

* Fix doc after imports
2019-07-10 10:01:56 +02:00
Pierre Krieger d7b6720663 Change the import queue traits to take &mut self instead of &self (#3058)
* SharedFinalityProofRequestBuilder -> BoxFinalityProofRequestBuilder

* SharedThings -> BoxThings

* Fix tests

* build_request_data now takes &mut self

* The other traits now also take &mut self

* More or less fix tests

* Fix tests

* Fix more tests

* Moar tests

* Don't call make_block_import multiple time

* Fix doctest
2019-07-09 17:11:25 +02:00
Pierre Krieger 459eb94c38 Hide the Kademlia warnings when offline (#3065) 2019-07-08 20:52:42 +01:00
Pierre Krieger 3a002a9100 Specialization::on_message now takes Vec<u8> (#3054) 2019-07-08 15:34:05 +02:00
Pierre Krieger 1e126eab2f Network crate cleanups (#3049)
* Remove useless internal messages

* Remove NetworkService::disconnect_peer

* Remove NetworkMsg altogether

* Rename ProtocolMsg ServerToWorkerMsg

* Remove useless code

* Add example for parse_str_addr

* Move parse_str_addr and ProtocolId to config

* Don't reexport the content of config

* Rework the imports

* More reexports rework

* Add documentation

* Move finalization report to network future

* Move on_block_imported to worker

* get_value/put_value no longer locking

* local_peer_id() no longer locks

* Remove FetchFuture

* Service imports cleanup

* Produce the network state in the network task

* Merge network task and RPC network task

* Move network methods to NetworkWorker

* Remove Arc peers system from network

* add_reserved_peer now goes through the channel

* Remove Mutex around network swarm

* Remove the FnOnce alias traits

* Replace is_offline with num_connected

* Improve style of poll()

* Fix network tests

* Some doc in service module

* Remove macro export

* Minor doc changes

* Remove the synchronized() method of the import queue

* Line width

* Line widths

* Fix import queue tests

* Fix CLI tests
2019-07-08 15:33:29 +02:00
Pierre Krieger 489a5ce9d7 Merge Protocol and ProtocolBehaviour (#2962)
* Pass the peerset config to ProtocolBehaviour

* Don't pass the protocol versions

* Move fields from protocol_behaviour.rs to protocol.rs

* Remove LocalNetworkOut

* Move CustomProtos from protocol_behaviour.rs to protocol.rs

* Remove ProtocolBehaviour

* Inline poll()

* Force Behaviour to use Protocol

* Don't even attempt to have working tests

* Remove NetworkOut trait

* Line widths
2019-07-05 16:06:07 +02:00