Commit Graph

107 Commits

Author SHA1 Message Date
André Silva 8f3a68db25 grandpa: fix --no-grandpa flag breaking block production (#3350) 2019-08-09 00:53:54 +01:00
André Silva 6565e1f8aa grandpa: track multiple live rounds in voter set state (#3298)
* grandpa: track multiple live rounds in voter set state

* grandpa: don't assume rounds are completed in-order

* grandpa: fix tests

* grandpa: don't assume round is being tracked on Environment::completed

* grandpa: fix missing import in test
2019-08-08 09:56:49 +02:00
Gavin Wood 1a524b8207 Refactor key management (#3296)
* Add Call type to extensible transactions.

Cleanup some naming

* Merge Resource and BlockExhausted into just Exhausted

* Fix

* Another fix

* Call

* Some fixes

* Fix srml tests.

* Fix all tests.

* Refactor crypto so each application of it has its own type.

* Introduce new AuthorityProvider API into Aura

This will eventually allow for dynamic determination of authority
keys and avoid having to set them directly on CLI.

* Introduce authority determinator for Babe.

Experiment with modular consensus API.

* Work in progress to introduce KeyTypeId and avoid polluting API
with validator IDs

* Finish up drafting imonline

* Rework offchain workers API.

* Rework API implementation.

* Make it compile for wasm, simplify app_crypto.

* Fix compilation of im-online.

* Fix compilation of im-online.

* Fix more compilation errors.

* Make it compile.

* Fixing tests.

* Rewrite `keystore`

* Fix session tests

* Bring back `TryFrom`'s'

* Fix `srml-grandpa`

* Fix `srml-aura`

* Fix consensus babe

* More fixes

* Make service generate keys from dev_seed

* Build fixes

* Remove offchain tests

* More fixes and cleanups

* Fixes finality grandpa

* Fix `consensus-aura`

* Fix cli

* Fix `node-cli`

* Fix chain_spec builder

* Fix doc tests

* Add authority getter for grandpa.

* Test fix

* Fixes

* Make keystore accessible from the runtime

* Move app crypto to its own crate

* Update `Cargo.lock`

* Make the crypto stuff usable from the runtime

* Adds some runtime crypto tests

* Use last finalized block for grandpa authority

* Fix warning

* Adds `SessionKeys` runtime api

* Remove `FinalityPair` and `ConsensusPair`

* Minor governance tweaks to get it inline with docs.

* Make the governance be up to date with the docs.

* Build fixes.

* Generate the inital session keys

* Failing keystore is a hard error

* Make babe work again

* Fix grandpa

* Fix tests

* Disable `keystore` in consensus critical stuff

* Build fix.

* ImOnline supports multiple authorities at once.

* Update core/application-crypto/src/ed25519.rs

* Merge branch 'master' into gav-in-progress

* Remove unneeded code for now.

* Some `session` testing

* Support querying the public keys

* Cleanup offchain

* Remove warnings

* More cleanup

* Apply suggestions from code review

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* More cleanups

* JSONRPC API for setting keys.

Also, rename traits::KeyStore* -> traits::BareCryptoStore*

* Bad merge

* Fix integration tests

* Fix test build

* Test fix

* Fixes

* Warnings

* Another warning

* Bump version.
2019-08-07 20:47:48 +02:00
thiolliere 4ed67e03a4 Update to parity-scale-codec (#3232)
* WIP: update codec

* WIP

* compiling

* WIP

* rename parity-scale-codec to codec

* WIP

* fix

* remove old comments

* use published crates

* fix expected error msg

* bump version

* fmt and fix

* remove old comment

* fix wrong decoding impl

* implement encode like for structures

* undo removal of old pending changes

* trailingzeroinput

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* update codec

* fmt

* version is 1.0.0

* show more error

* fmt
2019-08-06 19:36:23 +02:00
Kian Paimani 79feb23a22 Refactor sr_primitives. (#3214)
* refactor sr_primitives.

* Fix try build error.

* Line-width

* Ui test.

* Final fixes.

* Fix build again.

* bring back ui test.

* Fix unsigned import.

* Another ui fix.

* Also refactor substrate-primitives

* Fix benchmarks.

* Fix doc test.

* fix doc tests
2019-07-29 14:43:53 +02:00
Max Inden 6761ef1655 core/finality: Ref run_grandpa_voter instead of deprecated func in doc (#3035) 2019-07-05 15:43:16 +02:00
André Silva d5bc7325b9 grandpa: round catchup messages (#2801)
* grandpa: initial structure for catch up messages

* grandpa: answer catch up requests

* grandpa: inject catch up messages into global stream

* grandpa: keep track of pending catch up request

* grandpa: block catchup until all referenced blocks are imported

* grandpa: unify catch up and commit streams

* grandpa: simplify communication stream/sink types

* grandpa: note gossip validator on catch up message import

* grandpa: fix cost on catch up message validation

* grandpa: check signatures on catch up messages

* grandpa: clean up catch up request handling state

* grandpa: adjust costs on invalid catch up requests

* grandpa: release lock before pushing catch up message

* grandpa: validate catch up request against peer view

* grandpa: catch up docs

* grandpa: fix tests

* grandpa: until_imported: add tests for catch up messages

* grandpa: add tests for catch up message gossip validation

* grandpa: integrate HistoricalVotes changes

* grandpa: add test for neighbor packet triggering catch up

* grandpa: add test for full voter catch up

* grandpa: depend on finality-grandpa 0.8 from crates

* granda: use finality-grandpa test helpers

* grandpa: add PSM cost for answering catch up requests

* grandpa: code style fixes

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* grandpa: more trailing commas

* grandpa: lower cost of invalid catch up requests near set change

* grandpa: process catch up sending on import of neighbor message

* grandpa: add comments on HistoricalVotes

* grandpa: use finality-grandpa v0.8.1 from crates.io

* grandpa: fix test compilation
2019-07-04 20:40:16 +01:00
Pierre Krieger 1b73b6532a Remove tokio dependencies (#2935)
* Remove dependencies on tokio

* Make service not depend on tokio

* Fix service tests

* Manually poll the import queue if failed to start

* Spawn all tasks at the end

* Remove executor from TelemetryOnConnect

* Remove TaskExecutor from offchain workers

* Remove TaskExecutor from AuthoritySetup

* Remove TaskExecutor from service

* Remove tokio dependency from RPC

* Remove finality-grandpa from WASM checks

* Fix offchain tests

* Line widths

* Fix RPC tests

* Fix service tests

* Fix bad futures polling

* Address some concerns

* Better error handling

* Is it the connectivity test that's not passing? I don't know, let's try

* Revert "Is it the connectivity test that's not passing? I don't know, let's try"

This reverts commit 28bbe51f0e2e4885fe1f901e11078604604cb212.

* Fix test
2019-06-26 17:21:17 +02:00
Bastian Köcher e09045bde2 Fix several warnings and remove some forbid(warnings) (#2874) 2019-06-15 10:30:04 +02:00
Gavin Wood bda8641892 New sessions, kill consensus module (#2802)
* Draft of new sessions

* Reintroduce tuple impls

* Move staking module to new session API

* More work on staking and grandpa.

* Use iterator to avoid cloning and tuple macro

* Make runtime build again

* Polish the OpaqueKeys devex

* Move consensus logic into system & aura.

* Fix up system module

* Get build mostly going. Stuck at service.rs

* Building again

* Update srml/staking/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Refactoring out Consensus module, AuthorityIdOf, &c.

* Refactored out DigestItem::AuthoritiesChanged. Building.

* Remove tentative code

* Remove invalid comment

* Make Seal opaque and introduce nice methods for handling opaque items.

* Start to use proper digest for Aura authorities tracking.

* Fix up grandpa, remove system::Raw/Log

* Refactor Grandpa to use new logging infrastructure.

Also make authorityid/sessionkey static. Switch over to storing
authorities in a straight Vec.

* Building again

* Tidy up some AuthorityIds

* Expunge most of the rest of the AuthorityKey confusion.

Also, de-generify Babe and re-generify Aura.

* Remove cruft

* Untangle last of the `AuthorityId`s.

* Sort out finality_tracker

* Refactor median getting

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Session tests works

* Update core/sr-primitives/src/generic/digest.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Session tests works

* Fix for staking from @dvc94ch

* log an error

* fix test runtime build

* Some test fixes

* Staking mock update to new session api.

* Fix build.

* Move OpaqueKeys to primitives.

* Use on_initialize instead of check_rotate_session.

* Update tests to new staking api.

* fixup mock

* Fix bond_extra_and_withdraw_unbonded_works.

* Fix bond_with_little_staked_value_bounded_by_slot_stake.

* Fix bond_with_no_staked_value.

* Fix change_controller_works.

* Fix less_than_needed_candidates_works.

* Fix multi_era_reward_should_work.

* Fix nominating_and_rewards_should_work.

* Fix nominators_also_get_slashed.

* Fix phragmen_large_scale_test.

* Fix phragmen_poc_works.

* Fix phragmen_score_should_be_accurate_on_large_stakes.

* Fix phragmen_should_not_overflow.

* Fix reward_destination_works.

* Fix rewards_should_work.

* Fix sessions_and_eras_should_work.

* Fix slot_stake_is_least_staked_validator.

* Fix too_many_unbond_calls_should_not_work.

* Fix wrong_vote_is_null.

* Fix runtime.

* Fix wasm runtime build.

* Update Cargo.lock

* Fix warnings.

* Fix grandpa tests.

* Fix test-runtime build.

* Fix template node build.

* Fix stuff.

* Update Cargo.lock to fix CI

* Re-add missing AuRa logs

Runtimes are required to know about every digest they receive ― they
panic otherwise.  This re-adds support for AuRa pre-runtime digests.

* Update core/consensus/babe/src/digest.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Kill log trait and all that jazz.

* Refactor staking tests.

* Fix ci runtime wasm check.

* Line length 120.

* Make tests build again

* Remove trailing commas in function declarations

The `extern_functions!` macro doesn’t like them, perhaps due to a bug in
rustc.

* Fix type error

* Fix compilation errors

* Fix a test

* Another couple of fixes

* Fix another test

* More test fixes

* Another test fix

* Bump runtime.

* Wrap long line

* Fix build, remove redundant code.

* Issue to track TODO

* Leave the benchmark code alone.

* Fix missing `std::time::{Instant, Duration}`

* Indentation

* Aura ConsensusLog as enum
2019-06-14 16:34:34 +02:00
Robert Habermeier 67138b3103 polite-grandpa: Ensure only votes from a set's voters are propagated (#2720)
* make authorities available to gossip

* make local view optional until set

* address grumbles

* micro-optimize: encode without lock held
2019-06-09 11:38:41 +02:00
Pierre Krieger 5df89a8a6f Client::info() no longer returns a Result (#2776) 2019-06-04 16:09:46 +02:00
Benjamin Kampmann 22d3043917 Silence known deprecation warnings (#2651)
* Silence known deprecation warnings

1. Prefixes known instances of usages of client.backend and client.import_lock with `#[allow(deprecated)]` to silence the warnings. 2. Remove file-global `#![allow(deprecated)]` used in these cases. Both to prevent us overlooking externally caused deprecation messages.

* fixing missing ;

* fix missing test cases

* move deprecated markers to make CI happy

* move deprecated markers to make CI happy

* attempt to fix the test

* bumping impl_version of node runtime

* Minor cleanup
2019-05-28 10:34:59 +02:00
André Silva 10c1dfcffb Register previous rounds' votes with gossip service on startup (#2676)
* grandpa: register previous round votes with gossip service on startup

* gossip: fix tests

* grandpa: optionally register previous round votes on startup

* grandpa: fix tests
2019-05-26 23:05:02 +02:00
Tomasz Drwięga c162fc5ff1 Expunge error-chain (feat. tomaka) (#2662)
* Remove error_chain

* Expunge error-chain from rpc and service.

* Expunge from transaction pool.

* Expunge from node/cli

* Expunge from keystore.

* Remove some boilerplate.

* Fix remaining stuff.

* Improve on deprecation message.

* Fix issues.

* Fix trnsaction pool tests.

* Fix the rest.

* Fix borked merge.

* Update lock
2019-05-24 10:35:31 +01:00
Gavin Wood 3860d7c810 Remove As (#2602)
* Start to remove the `As` bound on `SimpleArtithmetic`

This just introduces standard numeric bounds, assuming a minimum of
`u32`. Also included is a saturating from/into trait allowing ergonomic
infallible conversion when you don't care if it saturates.

* Remove As from Balances trait

* Remove As from Aura module

* Remove As from Babe module

* Expunge `As` from contract

* Council module

* Democracy

* Finality tracker

* Grandpa

* First bit of indices

* indices

* Line lengths

* session

* system

* Staking

* Square up all other uses of As.

* RHD update

* Fix build/test

* Remove As trait

* line widths

* Remove final As ref

* Update srml/staking/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/client/src/cht.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/client/db/src/light.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* whitespace

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-Authored-By: André Silva <andre.beat@gmail.com>

* Bring back u32 check for number on CLI
2019-05-22 23:11:38 +01:00
Gavin Wood 0089ad4040 Revert "Allow warnings (#2627)" (#2648)
This reverts commit b9c39d771a.
2019-05-22 08:45:28 +01:00
Pierre Krieger b9c39d771a Allow warnings (#2627) 2019-05-20 17:14:02 -04:00
Svyatoslav Nikolsky 22586113ea Light GRANDPA import handler (#1669)
* GrandpaLightBlockImport

* extract authorities in AuraVerifier

* post-merge fix

* restore authorities cache

* license

* new finality proof draft

* generalized PendingJustifications

* finality proof messages

* fixed compilation

* pass verifier to import_finality_proof

* do not fetch remote proof from light import directly

* FinalityProofProvider

* fixed authorities cache test

* restored finality proof tests

* finality_proof docs

* use DB backend in test client

* justification_is_fetched_by_light_client_when_consensus_data_changes

* restore justification_is_fetched_by_light_client_when_consensus_data_changes

* some more tests

* added authorities-related TODO

* removed unneeded clear_finality_proof_requests field

* truncated some long lines

* more granular light import tests

* only provide finality proof if it is generated by the requested set

* post-merge fix

* finality_proof_is_none_if_first_justification_is_generated_by_unknown_set

* make light+grandpa test rely on finality proofs (instead of simple justifications)

* empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* missing trait method impl

* fixed proof-of-finality docs

* one more doc fix

* fix docs

* initialize authorities cache (post-merge fix)

* fixed cache initialization (post-merge fix)

* post-fix merge: fix light + GRANDPA tests (bad way)

* proper fix of empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* fixed easy grumbles

* import finality proofs in BlockImportWorker thread

* allow import of finality proofs for non-requested blocks

* limit number of fragments in finality proof

* GRANDPA post-merge fix

* BABE: pos-merge fix
2019-05-13 11:36:52 +02:00
Benjamin Kampmann 18ca0170c3 Allow for customisation of chain selection systems (#2240)
* move SelectChain trait out of client

* Extend SelectChain, move longest chain implementation into it

* Bring SelectChain into service

* implement LongestChain SelectChain

* implement longest chain for node

* update Cargo.lock's

* in between erroring tests

* deprecate ::backend and ::import_lock

* Remove unneded space

Co-Authored-By: gnunicorn <ben.kampmann@googlemail.com>

* Remove unneded space

Co-Authored-By: gnunicorn <ben.kampmann@googlemail.com>

* Fixes test compilation

* remove todo

* re-enable client test

* add doc

* fixing tests

* Clarify SelectChain Interface, intended implementation and usage

* minor components cleanups

* minor cleanups

* Update lock files

* Implement cleaner interface for SelectChain

* addressing comments

* Updating tests

* bump node runtime impl version

* address grumbles
2019-05-10 14:08:12 +02:00
DemiMarie-parity fb19684358 BABE signing and verification (#2270)
* Add basic BABE consensus type

* Update core/consensus/babe/slots/Cargo.toml

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Fix parameterization and run `rustfmt`

* Respond to review comments

* Update various Cargo.lock files

* Revert "Update various Cargo.lock files"

This reverts commit af53d7624752a744320e9cbb25749fdd8e6f46d2.

* `BabeSealSignature` → `BabeSeal`

* Move slot code to its own crate

This was highly non-trivial, due to cyclic dependencies.

* Remove redundancy between AuRa and BABE

Some of the code duplication was removed using a macro.

* Fix build error

* Avoid non-`#[doc(hidden)]` re-exports

Also, bump some library versions in `Cargo.toml`.

* Remove dead code in AuRa

* Remove impl_slot macro

It was more trouble than it was worth.

Also, delete useless dependencies on Serde.

* AuRa and BABE need different DB keys

* Bring back `aura::Network`, but deprecate it.

* Improve docs and add `slot_duration` inherent method

* Add docs to `substrate_consensus_aura::SlotDuration`

* Add missing documentation and #![forbid(missing_docs, unsafe_code)]

* Add a #![forbid(missing_docs)]

* Remove dependency of `test-runtime` on `slots`

* Update core/consensus/babe/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Remove wrongly added file

* Fix copyright notice

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Bump `impl_version` and `spec_version`

* Add more code to BABE

Most of it is copied from AuRa code, but at least the initial core is
there.

* Stuck on horrible compiler error message

* add missing files

* Spaces → tabs

* Simplify code

* Fix compilation

This involved fixing dependencies and adding a `Mutex`.

* More work on BABE

* Fix deprecation version

* Fix deprecation version; remove spurious carets

* Fix Cargo.toml

* Implement VRF signing logic

* The import queue code compiles, though it probably doesn’t work.

* Add VRF verification

* Update Cargo.lock

* Update dependencies

* Move test network to sr25519 authority keys

* Fix accidental build bustage

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Trying to get the tests to work

* Add logging messages and remove dead code

There seems to be a problem with the test network.  Since AuRa and BABE
are both affected, this is most likely due to the switch from ed25519 to
sr25519.

* Working testsuite at last!

The problem was with serialization and deserialization.  Normally,
those functions are generated automatically, but those for `BabeSeal`
had to be written manually.  The hand-written versions were not
correct, however, as shown by the decoder not being able to decode the
output of the encoder.

* Enable BabeSeal::Encode asserts in --release tests

* Bump runtime and dependency versions

* Fix wasm compilation

The wasm build was broken because of a typo in
`core/test-runtime/src/lib.rs`, and missing gates on the `std` feature
in `core/consensus/{aura,babe}/primitives/Cargo.toml`.  Additionally,
improve the quotation in the build scripts.

* Merge Cargo.lock

* Change expected JSON string

The test was also broken on `master`, so I suspect that the test was
incorrect.

* Responded to review

* Remove hard-coded threshold from production code

A hard-coded threshold is now only used in tests.

* Fix swapped doc comments

* Fix unused import warnings

* fix ci error

* fix typo

* Fix spacing in docs

* Minor changes suggested by @joepetrowski

on https://github.com/paritytech/substrate/pull/2372

* Remove unnecessary getters

* fix compile error

* Fix silly unused-variable error

* Improve documentation formatting

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Add issue links

* Revert excess verbosity and #![forbid(warnings)]

* Apply suggestions from code review

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Reformat some comments

* Threshold should depend on number of validators

Also, respond to code review

* Fix silly compilation errors

* Reduce logging verbosity

* Fix missing import
2019-05-02 11:16:54 -04:00
Michael Müller 87776e63bb Adjust consensus telemetry (#2198)
* Send high-level consensus telemetry by default

* Notify telemetry on finalized

* Send used authority set to telemetry

* Do not send commit message telemetry by default

* Fix typo

* Allow for notifications on telemetry connect

...and send the current authority set on each connect.

* Send authority set to telemetry on change

* Revert "Send used authority set to telemetry"

This reverts commit 1deceead52bb7443a02879ac8138afad9a6ca5ff.

* Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default'

Squashed commit of the following:

commit 19d77cbc23
Author: Xiliang Chen <xlchen1291@gmail.com>
Date:   Wed Apr 10 20:26:29 2019 +1200

    update authers for rest of the node-template cargo.toml files (#2242)

commit 0afc357a97
Author: Bastian Köcher <bkchr@users.noreply.github.com>
Date:   Tue Apr 9 10:31:18 2019 +0200

    Throw a compile error for `on_finalise` and `on_initialise` (#2236)

commit e57e54ab9c
Author: Pierre Krieger <pierre.krieger1708@gmail.com>
Date:   Tue Apr 9 05:30:43 2019 -0300

    Add warning when using default protocol ID (#2234)

    * Add warning when using default protocol ID

    * Update core/service/src/lib.rs

commit cb766e5f5d
Author: Xiliang Chen <xlchen1291@gmail.com>
Date:   Tue Apr 9 17:22:20 2019 +1200

    update name and authors to placeholder text for node-template (#2222)

    * update name and authors to placeholder text

    * revert package name change

commit a1e15ae55a
Author: André Silva <andre.beat@gmail.com>
Date:   Mon Apr 8 12:50:34 2019 +0100

    grandpa: Voter persistence and upgrade to finality-grandpa v0.7 (#2139)

    * core: grandpa: migrate to grandpa 0.7

    * core: grandpa: store current round votes and load them on startup

    * core: grandpa: resend old persisted votes for the current round

    * core: grandpa: store base and votes for last completed round

    * core: grandpa: fix latest grandpa 0.7 changes

    * core: grandpa: update to grandpa 0.7.1

    * core: grandpa: persist votes for last two completed rounds

    * core: grandpa: simplify VoterSetState usage

    * core: grandpa: use Environment::update_voter_set_state

    * core: grandpa: fix aux_schema test

    * core: grandpa: add docs

    * core: grandpa: add note about environment assumption

    * core: grandpa: don't update voter set state on ignored votes

    * core: grandpa: add test for v1 -> v2 aux_schema migration

    * core: grandpa: add test for voter vote persistence

    * core: grandpa: use grandpa 0.7.1 from crates.io

    * core: grandpa: use try_init in test

    * core: grandpa: add comment about block_import in test

    * core: grandpa: avoid cloning HasVoted

    * core: grandpa: add missing docs

    * core: grandpa: cleanup up can_propose/prevote/precommit

commit ed3ae4ac39
Author: Gregory Terzian <2792687+gterzian@users.noreply.github.com>
Date:   Mon Apr 8 13:17:00 2019 +0200

    remove clone bound on specialization in testnet factory (#2157)

commit 03f3fb1442
Author: Andrew Jones <ascjones@gmail.com>
Date:   Sat Apr 6 12:23:56 2019 +0100

    Contract import/export validation (#2203)

    * Reject validation of contract with unknown exports

    * Validate imports eagerly

    * Increment spec version

commit decddaab0f
Author: Pierre Krieger <pierre.krieger1708@gmail.com>
Date:   Fri Apr 5 14:07:09 2019 -0300

    Fix state inconsistency between handler and behaviour (#2220)

    * Fix state inconsistency between handler and behaviour

    * Fix the error! being in the wrong place

commit dce0b4ea49
Author: Bastian Köcher <bkchr@users.noreply.github.com>
Date:   Fri Apr 5 18:50:38 2019 +0200

    Use `storage_root` of newly calculated header (#2216)

    Instead of calculating the `storage_root` a second time, we just can
    take the `storage_root` from the new header.

commit b01136c90d
Author: Marek Kotewicz <marek.kotewicz@gmail.com>
Date:   Fri Apr 5 14:44:46 2019 +0200

    Peerset::discovered accepts many peer ids (#2213)

    * Peerset::discovered accepts many peer ids

    * Improve tracing in peerset

commit 1142bcde97
Author: Marek Kotewicz <marek.kotewicz@gmail.com>
Date:   Thu Apr 4 19:40:40 2019 +0200

    simplification of peerset api (#2123)

    * Introduction of PeersetHandle

    * integrate PeersetHandle with the rest of the codebase

    * fix compilation errors

    * more tests for peerset, fixed overwriting bug in add_reserved_peer

    * Slots data structure and bugfixes for peerset

    * bend to pressure

    * updated lru-cache to 0.1.2 and updated linked-hash-map to 0.5.2

    * peerset discovered list is now a LinkedHashMap

    * fix review suggestions

    * split back Peerset and PeersetHandle

    * test for Peerset::discovered

    * applied review suggestions

    * fixes to peerset::incoming

    * peerset disconnects are all instantaneous

    * instantaneous drop in peerset finished

    * Peerset::set_reserved_only can also reconnect nodes

    * Peerset scores cache uses lru-cache

    * remove redundant function call and comment from Peerset::on_set_reserved_only

    * add_peer returns SlotState enum

    * apply review suggestions

    * is_reserved -> is_connected_and_reserved

commit 301844dd56
Author: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Date:   Thu Apr 4 18:01:28 2019 +0200

    Disconnect on protocol timeout (#2212)

commit cb3c912b1a
Author: André Silva <andre.beat@gmail.com>
Date:   Thu Apr 4 15:56:49 2019 +0100

    core: grandpa: verify commit target in justification (#2201)

commit 6920b169cd
Author: Bastian Köcher <bkchr@users.noreply.github.com>
Date:   Thu Apr 4 16:56:16 2019 +0200

    Introduce `original_storage` and `original_storage_hash` (#2211)

    Both functions will ignore any overlayed changes and access the backend
    directly.

commit cb7a8161f5
Author: Xiliang Chen <xlchen1291@gmail.com>
Date:   Fri Apr 5 03:55:55 2019 +1300

    code cleanup (#2206)

commit acaf1fe625
Author: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Date:   Wed Apr 3 15:52:46 2019 +0200

    Emberic elm testnet (#2197)

* Make telemetry onconnect hoook optional

* Merge branch 'master' into 'cmichi-send-high-level-consensus-telemetry-by-default'

* Introduce GrandpaParams struct to condense parameters

* Remove debug statement

* Fix tests

* Rename parameter

* Fix tests

* Rename struct

* Do not send verbosity level

* Combine imports

* Implement comments

* Run cargo build --all

* Remove noisy telemetry

* Add docs for public items

* Unbox and support Clone trait

* Fix merge

* Fix merge

* Update core/finality-grandpa/src/lib.rs

Co-Authored-By: cmichi <mich@elmueller.net>
2019-04-23 15:49:28 +02:00
André Silva e31cd26a9e grandpa: observer (#2244)
* grandpa: initial implementation of minimal grandpa worker

* grandpa: extract grandpa observer future to function

* grandpa: add test for observer

* grandpa: start observer if no local key is defined

* grandpa: add minor comments

* grandpa: observer: log invalid commit

* grandpa: observer: persist voter set state on authority change and pause

* grandpa: observer: use commit processing callback

* grandpa: keep run_grandpa to avoid breaking public api

* grandpa: use grandpa::process_commit_validation_result

* grandpa: use finality-grandpa 0.7.2
2019-04-17 16:50:38 +02:00
Robert Habermeier 1aa6eb9fa8 Polite-grandpa improvements (#2229)
* send neighbor packets in more generic way

* integrate periodic neighbor-packet rebroadcaster

* integrate reporting

* attach callbacks to commit messages for rebroadcasting and reporting

* Tests for commit relay

* crunch up some nice warnings

* exit-scope sub-futures of grandpa

* address small grumbles

* some changes to commit handling
2019-04-16 15:25:46 +08:00
André Silva 2fad6d80da core: grandpa: persist voter set state on authority set change (#2249) 2019-04-15 12:17:08 +02:00
André Silva a1e15ae55a grandpa: Voter persistence and upgrade to finality-grandpa v0.7 (#2139)
* core: grandpa: migrate to grandpa 0.7

* core: grandpa: store current round votes and load them on startup

* core: grandpa: resend old persisted votes for the current round

* core: grandpa: store base and votes for last completed round

* core: grandpa: fix latest grandpa 0.7 changes

* core: grandpa: update to grandpa 0.7.1

* core: grandpa: persist votes for last two completed rounds

* core: grandpa: simplify VoterSetState usage

* core: grandpa: use Environment::update_voter_set_state

* core: grandpa: fix aux_schema test

* core: grandpa: add docs

* core: grandpa: add note about environment assumption

* core: grandpa: don't update voter set state on ignored votes

* core: grandpa: add test for v1 -> v2 aux_schema migration

* core: grandpa: add test for voter vote persistence

* core: grandpa: use grandpa 0.7.1 from crates.io

* core: grandpa: use try_init in test

* core: grandpa: add comment about block_import in test

* core: grandpa: avoid cloning HasVoted

* core: grandpa: add missing docs

* core: grandpa: cleanup up can_propose/prevote/precommit
2019-04-08 13:50:34 +02:00
Robert Habermeier bb95e7d6a2 Initial version of the polite-grandpa networking protocol (#2110)
* Consensus status packet

* Allow for repropagation after status

* More generic gossip

* add a basic view struct and gossip module

* move gossip stuff to the gossip module

* integrate view into gossip

* some reshuffling

* alter rules for keeping one commit at a time in view

* Allow sending addressed messages

* don't cast outgoing votes if we know that we voted before

* Handle one hop messages

* initial run at polite grandpa

* build WASM

* handle neighbor messages

* refactor validator's internals into an Inner struct

* gossip only knows to keep or discard messages. optimize should_send_to

* Periodic rebroadcast

* implement `should_send_to` and message_expired

* track peers' best received commit height

* Pass peer id to topic steam

* kill rebroadcasting network

* Notify about existing peers

* clean up network APIs a bunch

* implement gossip::send_message for direct messages

* refactor network trait

* implement gossip::send_message for direct messages

* get all non set-change tests passing

* treat unknown rebroadcasts as broadcasts

* get all other main tests passing

* remove unimplemented test

* everything compiles

* treat unknown rebroadcasts as broadcasts

* Rebradcast interval

* Apply suggestions from code review

Style

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Style

* some module docs

* address some grumbles + docs

* allow rebroadcast every few minutes

* send_topic && generic context

* some tests for view change

* more grumbles & tests

* use send_peer
2019-04-02 18:09:05 +02:00
DemiMarie-parity a10e86ba5a Refactor DigestItem (#2108)
* Add `start_aura2`.

* .gitignore patch conflict files

and remove one that accidentally got committed

* Fix build

The tests still don’t work.

* Fix compilation errors

* Fix compile errors (again)

* Try (and fail) to fix tests

* Properly deserialize data

Previously, `DigestItem::Consensus` had no separate `DigestItemType`,
so it did not get properly serialized and deserialized.

* Add extra debug logging.  Always allow old seals.

A `RUST_LOG=substrate_aura_consensus cargo test --all -- --nocapture \
tests::authoring_blocks` revealed that old seals were being and
rejected, causing the test to hang.  As a temporary debug measure, allow
old seals unconditionally, so that CI can test if this fixes the
problem.

* Forcibly disable rejection of old seals

* Use old trait, but newer serialization

The old trait for `CompatibleDigestItem` actually worked.  By changing
its implementation, one can ensure that all *new* seals have the modern
form, but *legacy* seals are still decoded correctly.

* Bump impl version

* Squash spurious deprecation warning

`rustc` should not be emitting a deprecation warning in deprecated
code, but it does, so silence it.

* Rip out unused Cargo feature

* Move AURA to aura_primitives

* Respond to code review

* Wrap overly-long line

* Reduce logging verbosity and add target

* Add dependency on `sr-primitives` to `aura_primitives`

* Fix build

It failed with a message about Cargo.lock being out of date.

* core: aura: rename aura engine id const

* core: aura: remove superfluous logging

* core: primitives: add removed semicolons

* core: aura: remove unused import

* core: network: style fix

* runtime: update wasm blobs

* runtime: bump impl_version

* core: primitives: tag all DigestItemType variants explicitly
2019-03-29 12:15:23 +00:00
André Silva e56a5cd00b Minor gossip changes (#2038)
* core: gossip: don't expire messages based on time

* core: gossip: allow forcing resend of gossip messages

* core: grandpa: fix tests
2019-03-19 12:13:05 +01:00
Michael Müller c52c528ee8 Add grandpa telemetry (+ other minor improvements) (#1997)
* Support skipping the interactive purge prompt

Skip it via `cargo run -- purge-chain -y`.

* Fix typos

* Add Grandpa telemetry

* Address comments

* Revert unnecessary version bump

* Version bump to make CI run

* Remove unnecessary cast

* Do not bump version
2019-03-15 15:04:01 +01:00
Gav Wood d7fcf5dc9d Overhaul crypto (Schnorr/Ristretto, HDKD, BIP39) (#1795)
* Rijig to Ristretto

* Rebuild wasm

* adds compatibility test with the wasm module

* Add Ed25519-BIP39 support

* Bump subkey version

* Update CLI output

* New keys.

* Standard phrase/password/path keys.

* Subkey uses S-URI for secrets

* Move everything to use new HDKD crypto.

* Test fixes

* Ignore old test vector.

* fix the ^^ old test vector.

* Fix tests

* Test fixes

* Cleanups

* Fix broken key conversion logic in grandpa

CC @rphmeier

* Remove legacy Keyring usage

* Traitify `Pair`

* Replace Ed25519AuthorityId with ed25519::Public

* Expunge Ed25519AuthorityId type!

* Replace Sr25519AuthorityId with sr25519::Public

* Remove dodgy crypto type-punning conversions

* Fix some tests

* Avoid trait

* Deduplicate DeriveJunction string decode

* Remove cruft code

* Fix test

* Minor removals

* Build fix

* Subkey supports sign and verify

* Inspect works for public key URIs

* Remove more crypto type-punning

* Fix typo

* Fix tests
2019-03-13 14:08:31 +01:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Robert Habermeier dfb48a2405 Offline fallback for GRANDPA (#1619)
Co-authored-by: André Silva <andre.beat@gmail.com>

* skeleton for finality tracker

* dispatch events when nothing finalized for a long time

* begin integrating finality tracker into grandpa

* add delay field to pending change

* add has_api_with function to sr_version for querying APIs

* partially integrate new force changes into grandpa

* implement forced changes

* get srml-grandpa compiling

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* remove explicit dependence on CoreApi

* increase node runtime version

* integrate grandpa forced changes into node runtime

* add some tests to finality-tracker

* integrate finality tracking into node-runtime

* test forced-change logic

* test forced changes in the authority-set handler

* kill some unneeded bounds in client

* test forced-changes in finality-grandpa and fix logic

* build wasm and finality-tracker is no-std

* restart voter on forced change

* allow returning custom error type from lock_import_and_run

* extract out most DB logic to aux_schema and use atomic client ops

* unify authority set writing

* implement set pausing

* bump runtime version

* note on DB when we pause.

* core: grandpa: integrate forced changes with multiple pending standard changes

* core: grandpa: fix AuthoritySet tests

* runtime: bump impl_version

* core: clear pending justification requests after forced change import

* srml: finality-tracker: use FinalizedInherentData

* core: log requests for clearing justification requests

* core, node: update runtimes

* core: grandpa: fix tests

* core: grandpa: remove todos and add comments

* core: grandpa: use has_api_with from ApiExt

* core: fix tests

* core: grandpa: remove unnecessary mut modifier

* core: replace PostImportActions bitflags with struct

* core: grandpa: restrict genesis on forced authority set change

* core: grandpa: add more docs

* core: grandpa: prevent safety violations in Environment::finalize_block

* core: grandpa: register finality tracker inherent data provider

* core: grandpa: fix tests

* node: update runtime blobs

* core: grandpa: remove outdated todo

* core: aura: fix typo in log message

* core: grandpa: check re-finalization is on canonical chain

* srml: finality-tracker: fix initialization

* node: update runtime wasm

* srml: finality-tracker: don't re-initialize config keys
2019-03-05 15:41:35 +00:00
Hero Bird 128d164f2b Make use of parity-codec "derive" crate feature (#1919)
* Make use of parity-codec "derive" crate feature

* Bump impl_version

* Update wasm files and Cargo.lock files

* Transferred to parity_codec derive feature for the rest of the crates

* Update wasm binaries and lock files

* Update core and node wasm binaries
2019-03-05 09:00:37 +01:00
Michael Müller c485eb8fdc Get rid of wildcard imports (#1916)
* Get rid of wildcard imports

* Update core/service/src/lib.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Update core/client/src/client.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Import by namespace
2019-03-04 14:00:46 +01:00
Robert Habermeier 4c8fec17fc Some gossip improvements (#1892)
* queue messages in future

* use new gossip API in GRANDPA

* implement message_expired for grandpa

* fix indent
2019-02-28 20:19:42 +01:00
Michael Müller 90e5c5ddfb Telemetry improvements (#1886)
* Fix typo

* Support multiple telemetry endpoints and verbosity levels

* Bump substrate-telemetry version

* Telemetrify Aura consensus

* Telemetrify Grandpa

* Fix CI version conflicts

* Implement style remarks

* Fix fixture

* Implement style remarks

* Clone only when necessary

* Get rid of Arc for URL

* Handle connection issues better
2019-02-28 12:22:05 +01:00
Arkadiy Paronyan b3eae17f65 Gossip refactoring (#1811)
* First part of gossip protocol refactoring

* Message validation in GRANDPA

* Reverted to time-based expiration for future round messages

* Removed collect_garbage_for_topic

* Use consensus engine id instead of kind

* Use try_init

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Comment

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Added expiration check on broadcast

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Apply suggestions from code review

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>

* Style

* Style
2019-02-20 12:40:03 +01:00
André Silva 21779b8cf2 Import multiple authority set change blocks (#1808)
* core: implement logic for tracking dag of possible pending changes

* core: move pending justifications dag to its own crate

* core: remove unnecessary clone bounds on dag

* core: request justifications in-order from the dag

* core: dag: rename changes variables to node

* core: dag: allow finalizing blocks not part of dag

* core: dag: track best finalized number

* core: dag: add more tests

* core: sync: clean up pending justifications dag

* core: dag: derive codec decode encode

* core: dag: better error support

* core: dag: add finalization guarded by predicate

* core: grandpa: track multiple authority set changes in dag

* core: dag: add pre-order iterator

* core: grandpa: request justifications on startup

* core: dag: rearrange order of definitions

* core: rename util/dag to util/fork_tree

* core: fork_tree: add docs

* core: fork_tree: add more tests

* core: fork_tree: fix issues found in tests

* core: grandpa: fix authorities tests

* core: grandpa: add docs for is_descendent_of

* core: sync: add docs for PendingJustifications

* core: sync: add test for justification requests across forks

* core: sync: don't resend import or finality notifications in tests

* core: grandpa: add test for importing multiple change blocks

* core: grandpa: fix logic for checking if a block enacts a change

* core: grandpa: fix authorities tests
2019-02-20 00:08:43 +01:00
Stanislav Tkach 34e3487035 Migrate finality-grandpa to the 2018 edition (#1797) 2019-02-15 16:06:18 -03:00
André Silva f8246ebcdf core: grandpa: collect garbage for topic (#1780)
* core: grandpa: collect garbage for topic

* core: grandpa: use collect_garbage_for_topic in tests
2019-02-13 15:37:44 +01:00
Gregory Terzian 6122f7d6b6 Remove wait on future in network bridge (#1765)
* remove wait on future in network bridge

* nit

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>

* nit

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>

* nit

* propagate error

* nit once more

* nit

Co-Authored-By: gterzian <2792687+gterzian@users.noreply.github.com>
2019-02-12 16:18:53 +01:00
Stanislav Tkach e60be1ad12 Migrate network, primitives and rpc to the 2018 edition (#1710) 2019-02-06 19:07:48 +01:00
Gregory Terzian a2d2ed69ab Rewrite network protocol/service to use channels (#1340)
* rewrite network protocol/service to use channels

* remove use of unwrap

* re-introduce with_spec

* remove unnecessary mut

* remove unused param

* improve with_spec, add with_gossip

* rename job to task

* style: re-add comma

* remove extra string allocs

* rename use of channel

* turn TODO into FIXME

* remove mut in match

* remove Self in new

* pass headers by value to network service

* remove network sender from service

* remove TODO

* better expect

* rationalize use of network sender in ondemand
2019-02-06 12:54:02 +01:00
André Silva 57ac31a0aa Split substrate-finality-grandpa (#1675)
* core: grandpa: move consensus_changes to its own submodule

* core: grandpa: move justification to its own submodule

* core: grandpa: move environment to its own submodule

* core: grandpa: move block import to its own submodule
2019-02-04 13:54:14 -03:00
André Silva 4983f113e6 Reduce consensus spam (#1658)
* core: fix predicate for dropping grandpa round messages

* core: grandpa: drop commits topic on authority set change

* core: gossip: only drop known messages based on expiration time

* core: grandpa: don't broadcast commit messages

* core: gossip: don't assume topics are header hashes

* core: gossip: expire messages more agressively

* core: grandpa: fix test environment

* core: gossip: fix tests

* core: gossip: track dead topics (and ignore messages)

* core: gossip: test dead topic pruning
2019-02-01 14:25:07 -03:00
Benjamin Kampmann 15ae7cfef6 Document more TODOs as tickets (#1418)
Went through the TODOs, removed a bunch, which are outdated or nothing more than a regular comment, documented a bunch more as actual tickets and made them FIXMEs and unified their structure (`FIXME #TICKETNO DESC` for local tickets, `FIXME: DESC LINK` for external tickets) for easier in-editor support. Further more remove unnecessary remarks and related old code that I noticed in that instance.
2019-01-30 10:29:48 +01:00
Robert Habermeier d027123059 GRANDPA: announce blocks we vote on to peers (#1593)
* announce blocks we vote on to peers (missing network impl)

* Implemented 'announce'

* Fixed test

* improve announce docs and add logging

* Track recently announced blocks
2019-01-28 16:41:36 +00:00
thiolliere 0078927ac5 update codec and grandpa (#1582)
* update codec

* use new codec

* update grandpa code

* build .lock and binaries
2019-01-28 14:16:17 +01:00
Robert Habermeier 86c5f78805 more logging around pending GRANDPA changes (#1587) 2019-01-27 14:53:57 +01:00