Commit Graph

69 Commits

Author SHA1 Message Date
Arkadiy Paronyan c6addc9bd5 CLI option to enable authoring when offline (#2104)
* CLI option to enable authoring when offline

* Update core/service/src/config.rs

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>
2019-03-25 11:28:35 +01:00
Pierre Krieger 6e394464b8 Replace NodeIndex with PeerId everywhere (#2077)
* Replace NodeIndex with PeerId

* Fix tests

* More test fixing

* Whitespace
2019-03-23 10:34:28 +01:00
André Silva 25ec793e35 Fix handling of justifications (#2086)
* util: fork-tree: check predicate first while traversing tree

* core: sync: keep track of justifications sent to the import queue

* core: grandpa: verify authority set changes dependencies

* core: fork-tree: add more tests

* core: grandpa: extend enacts_standard_change tests
2019-03-22 19:07:24 +01:00
DemiMarie-parity 000e7a8002 Make AuRa generic over cryptography (#2062)
As is so often the case in Rust, most of the work was fixing compiler
errors and removing spurious type annotations.
2019-03-21 17:26:00 -04:00
Robert Habermeier e83a39fee3 remove aura inherent-checking compatibility shim (#2044) 2019-03-19 18:13:23 +01:00
thiolliere d743a8b71f Implement parameterisable modules (#1800)
* first implementation

* remove done comment

* origin done

* impl log for instance

* impl inherent for instance

* Fix wasm build + full example build

this requires parity codec implements codec for core::marker::PhantomData

* patch parity-codec link to github branch

* improve internal names and fix instance prefix

* Fix in macros

* add test modules for support

this allow to test for construct_runtime as well.

The reason to have put that in another crate is:
* if we put test in `tests/` dir of srml/support then decl_storage fails to get
  srml-support access because it believes it is inside srml-support
  crate and so derive access to `quote!{ crate }` but this is wrong
  (and I don't see any way to prevent that, and it only bother us so I
  don't think that matters that much)
* if we put test inside lib.rs then contruct_runtime cannot be used
  because it call some macros that are defined with macros
  (decl_outer_event and decl_outer_origin) and thus rustc complains.

* defaultinstance to its own struct to avoid errors

* enforce <T, I> for Event and Config, impl test

* add origin, log, inherent to test

* test more code generation

* basic storage test

* fix typo

* rename a few imports and field

* delete wip test in example and runtime

* change default prefix to make it backward compatible with test

* rename Instance to I and Instantiable to Instance

note: the name of generic parameter I is only enforce by decl_module!
and this could be rewritten

* doc

* clean old TODOs

* update parity-codec to 3.2

* update node impl version + builds

* fix warning

* fix unrelated grandpa test

* refactor code
2019-03-15 19:25:18 +01:00
Marcio Diaz 534863d6d8 Import blocks with bad justification (#1977)
* init version

* chore: improve code

* fix test

* fix: log error resulting of bad justification

* fix: add test to check for disconnected peer
2019-03-14 12:24:29 +01:00
Gav Wood d7fcf5dc9d Overhaul crypto (Schnorr/Ristretto, HDKD, BIP39) (#1795)
* Rijig to Ristretto

* Rebuild wasm

* adds compatibility test with the wasm module

* Add Ed25519-BIP39 support

* Bump subkey version

* Update CLI output

* New keys.

* Standard phrase/password/path keys.

* Subkey uses S-URI for secrets

* Move everything to use new HDKD crypto.

* Test fixes

* Ignore old test vector.

* fix the ^^ old test vector.

* Fix tests

* Test fixes

* Cleanups

* Fix broken key conversion logic in grandpa

CC @rphmeier

* Remove legacy Keyring usage

* Traitify `Pair`

* Replace Ed25519AuthorityId with ed25519::Public

* Expunge Ed25519AuthorityId type!

* Replace Sr25519AuthorityId with sr25519::Public

* Remove dodgy crypto type-punning conversions

* Fix some tests

* Avoid trait

* Deduplicate DeriveJunction string decode

* Remove cruft code

* Fix test

* Minor removals

* Build fix

* Subkey supports sign and verify

* Inspect works for public key URIs

* Remove more crypto type-punning

* Fix typo

* Fix tests
2019-03-13 14:08:31 +01:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
renlulu 46541ec73c fix typo (#1928) 2019-03-06 08:39:49 +01:00
Robert Habermeier dfb48a2405 Offline fallback for GRANDPA (#1619)
Co-authored-by: André Silva <andre.beat@gmail.com>

* skeleton for finality tracker

* dispatch events when nothing finalized for a long time

* begin integrating finality tracker into grandpa

* add delay field to pending change

* add has_api_with function to sr_version for querying APIs

* partially integrate new force changes into grandpa

* implement forced changes

* get srml-grandpa compiling

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* remove explicit dependence on CoreApi

* increase node runtime version

* integrate grandpa forced changes into node runtime

* add some tests to finality-tracker

* integrate finality tracking into node-runtime

* test forced-change logic

* test forced changes in the authority-set handler

* kill some unneeded bounds in client

* test forced-changes in finality-grandpa and fix logic

* build wasm and finality-tracker is no-std

* restart voter on forced change

* allow returning custom error type from lock_import_and_run

* extract out most DB logic to aux_schema and use atomic client ops

* unify authority set writing

* implement set pausing

* bump runtime version

* note on DB when we pause.

* core: grandpa: integrate forced changes with multiple pending standard changes

* core: grandpa: fix AuthoritySet tests

* runtime: bump impl_version

* core: clear pending justification requests after forced change import

* srml: finality-tracker: use FinalizedInherentData

* core: log requests for clearing justification requests

* core, node: update runtimes

* core: grandpa: fix tests

* core: grandpa: remove todos and add comments

* core: grandpa: use has_api_with from ApiExt

* core: fix tests

* core: grandpa: remove unnecessary mut modifier

* core: replace PostImportActions bitflags with struct

* core: grandpa: restrict genesis on forced authority set change

* core: grandpa: add more docs

* core: grandpa: prevent safety violations in Environment::finalize_block

* core: grandpa: register finality tracker inherent data provider

* core: grandpa: fix tests

* node: update runtime blobs

* core: grandpa: remove outdated todo

* core: aura: fix typo in log message

* core: grandpa: check re-finalization is on canonical chain

* srml: finality-tracker: fix initialization

* node: update runtime wasm

* srml: finality-tracker: don't re-initialize config keys
2019-03-05 15:41:35 +00:00
Hero Bird 128d164f2b Make use of parity-codec "derive" crate feature (#1919)
* Make use of parity-codec "derive" crate feature

* Bump impl_version

* Update wasm files and Cargo.lock files

* Transferred to parity_codec derive feature for the rest of the crates

* Update wasm binaries and lock files

* Update core and node wasm binaries
2019-03-05 09:00:37 +01:00
Michael Müller c485eb8fdc Get rid of wildcard imports (#1916)
* Get rid of wildcard imports

* Update core/service/src/lib.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Update core/client/src/client.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Import by namespace
2019-03-04 14:00:46 +01:00
Bastian Köcher 2e9b2be8fa Update parity-codec/-derive to 3.1 (#1900)
* Update parity-codec/-derive to 3.1

* Update wasm stuff
2019-03-01 09:04:53 +01:00
Michael Müller 90e5c5ddfb Telemetry improvements (#1886)
* Fix typo

* Support multiple telemetry endpoints and verbosity levels

* Bump substrate-telemetry version

* Telemetrify Aura consensus

* Telemetrify Grandpa

* Fix CI version conflicts

* Implement style remarks

* Fix fixture

* Implement style remarks

* Clone only when necessary

* Get rid of Arc for URL

* Handle connection issues better
2019-02-28 12:22:05 +01:00
Gregory Terzian 8a72abffdd Remove blocking sync -> import-queue operations (#1818)
* remove blocking sync -> import-queue operations

add specialization to testnet

remove add peer default impl on TestNetFactory

* remove empty brackets from dummy specialization

* nits

* make mut_peers take an fn once

* add SpecializationFactory trait in test

* remove add_peer imple in grandpa, fix typo

* use cmp::max for best importing number comparison

* remove import of non-existent create_peer

* add sender to start message
2019-02-28 12:11:18 +01:00
Gregory Terzian 5282615416 Use bounded channels for network -> sync -> import (#1874)
* use bounded channels for network -> sync -> import

* bound at 4

* indent

* use return value of handle_network_msg
2019-02-27 09:19:57 +01:00
Gregory Terzian 72bb8ef4c5 Rewrite the BasiQueue using channels (#1327)
* use channels to implement basic import queue

* async justification import

* better conditional for is_done in tests

* reword the test for presence of link

* fix conditional

* trace instead of panic when no link present

* reword expectations when sending to importers

* fix

* debug justification import error

* update expectations

* use NumberFor

* nits

* add general description

* move error handling into closure
2019-02-17 10:13:14 +01:00
Stanislav Tkach ff5e4ca87e Migrate everything to the 2018 edition (#1758) 2019-02-13 11:45:59 +01:00
Arkadiy Paronyan 1dc15be5bd Check for block parent before verification. (#1714)
* Treat verification errors more seriously

* Track obsolete requests

* Check block parent before verification

* Style
2019-02-07 11:58:27 +01:00
Bastian Köcher 567122fab5 Moves node-template into substrate repo (#1637)
* Make runtime macros work without required `macro_use`

* Adds node-template

* Adds node-template-release tool

* Fixes building `node-template` and improve the release

* Add `profile.release` by release script to remove warning

* Adds script for releasing the node template

* Fixes compilation after master merge

* Port node-template to edition 2018

* Remove license

* Fixes compilation after master merge

* Add `node-template-release.sh` into the CI

* WIP Ci integrate node template (#1701)

* copy artifacts to s3 bucket latest path

* typo

* bucket name

* Update wasm files
2019-02-06 17:15:12 +01:00
Gregory Terzian a2d2ed69ab Rewrite network protocol/service to use channels (#1340)
* rewrite network protocol/service to use channels

* remove use of unwrap

* re-introduce with_spec

* remove unnecessary mut

* remove unused param

* improve with_spec, add with_gossip

* rename job to task

* style: re-add comma

* remove extra string allocs

* rename use of channel

* turn TODO into FIXME

* remove mut in match

* remove Self in new

* pass headers by value to network service

* remove network sender from service

* remove TODO

* better expect

* rationalize use of network sender in ondemand
2019-02-06 12:54:02 +01:00
Bastian Köcher 190393d476 Make API backwards compatible with CC (#1697)
* Rework how a runtime api calls into the runtime

Now we generate a default implementation for each api call that calls
a generated method `method_runtime_api_impl`. This newly generated
method is the one that will be implemented by the `impl_runtime_apis`
macro in the runtime for the client side.

* Support `changed_in` to change runtime api function signatures

* Update documentation

* Fixes tests

* Implement checking the api version with a predicate

* Make the implementation backwards compatible with CC

* Update wasm files after merge

* Check for wasm runtime differences by building master and current branch

* Update spec_version and wasm files

* Fixes

* Revert my changes

* Remove `patch.crates-io` from test-runtime
2019-02-06 11:47:47 +01:00
Wei Tang 1a3fa9eb68 Refactor aura and generalize slot worker (#1632)
* Refactor aura and generalize slot worker

* Fix bound issues

* Change copyright to 2019

* Write deps minor version

* Don't pass inherent_data_providers

* Revert "Write deps minor version"

This reverts commit 6d886702a899bfbdbbc609c0536e8abab2bc012c.

* Fix merge issues
2019-02-04 11:32:21 +01:00
Arkadiy Paronyan 641bb7cb46 Stop authoring blocks when offline (#1655)
* Don't author blocks when offline

* Increased canonicalization delay

* Fixed test
2019-02-01 17:17:53 +01:00
Tomasz Drwięga ecdd33a367 Allow both consensus and runtime to limit block building (#1581)
* Limit block size in runtime,

* Add test for basic authorship.

* Store length of extrinsics instead of computing it.

* Don't rely on note_extrinsic

* Use hashed version of storage and write test.

* Recompile runtime.
2019-02-01 13:58:23 +01:00
thiolliere 9de35fa7d5 update rhododendron trie-bench for parity-codec 3.0 (#1613)
* update rhd and tri

* .lock and wasm build
2019-01-31 11:04:41 +01:00
Benjamin Kampmann 15ae7cfef6 Document more TODOs as tickets (#1418)
Went through the TODOs, removed a bunch, which are outdated or nothing more than a regular comment, documented a bunch more as actual tickets and made them FIXMEs and unified their structure (`FIXME #TICKETNO DESC` for local tickets, `FIXME: DESC LINK` for external tickets) for easier in-editor support. Further more remove unnecessary remarks and related old code that I noticed in that instance.
2019-01-30 10:29:48 +01:00
thiolliere 0078927ac5 update codec and grandpa (#1582)
* update codec

* use new codec

* update grandpa code

* build .lock and binaries
2019-01-28 14:16:17 +01:00
Benjamin Kampmann ab4c70b379 Update dependencies (#1527)
* update to latest env_logger

* roundabout dependency update

* Update to latest vengen

* Update rand to 0.6

* Update to latest rng shuffle API

* Roundabout update of dependencies

* Update rustc_hex

* Update wasm

* Fix logging setup in tests

* revert to fix test
2019-01-25 12:44:41 +01:00
Bastian Köcher b187695af7 Aura: Remove unused structures and adds SlotDuration::get() (#1565) 2019-01-25 10:56:46 +00:00
Stanislav Tkach 7f05bd959f Migrate consensus (common, aura-primitives, aura, rhd) to 2018 edition (#1548) 2019-01-24 15:51:33 +00:00
Wei Tang dd88dc6cd6 Split BlockImport and JustificationImport (#1521)
* Split BlockImport and JustificationImport

* Remove unused trait impl

* Fix compile

* Fix grandpa tests

* Fix network tests
2019-01-23 12:22:08 -03:00
Bastian Köcher 70b1af7b1e Rewrite Inherent data (#1488)
* Implement new inherent data

* Fixes compilation on wasm

* Fixes after rebase

* Switch back to generate inherent stuff by macro

* Update after rebase

* Apply suggestions from code review

Co-Authored-By: bkchr <bkchr@users.noreply.github.com>

* Fix compilation after rebase

* Address grumbles

* Remove `InherentDataProviders` from `Client`

* Update wasm files after rebase

* Address grumbles

* Fixes compilation after latest merge

* Last fix
2019-01-22 17:52:08 +01:00
Bastian Köcher 010e63116f Make runtime api calls native when possible (#1302)
* Add simple benchmark for the runtime api

* Make the executor support native calls

* Some documentation

* Hide behind `feature = "std"`

* Rework the native calls

* Make all tests compile again

* Make every parameter using the Block serialized/deserialized in the native call

* Forward `UnwindSafe` requirement

* Remove debug stuff

* Add some documentation

* Fixes warnings

* Fixes errors after master rebase

* Fixes compilation after master rebase

* Fixes compilation after rebase
2019-01-21 14:32:53 +01:00
André Silva 399cee310a Sync block justifications (#1410)
* core: sync protocol for justifications

* core: basic test for justification sync

* core: pass block number with justification

* grandpa: request justifications when importing change blocks

* core: pass finality notifications to chain sync

* core: require justifications for pending change blocks on start

* core: avoid requesting justifications from previous failed peers

* core: timeout block justification requests

* core: add some docs

* core: fix unused variables warning

* core: tick pending justifications fetch periodically

* grandpa: add test for syncing justifications

* core: early exit dispatch of pending justifications

* core: style fix

* core: grandpa: change logging level

* core: sync: add missing docs

* core: network: report peer on bad justification

* core: replace mem::replace with Option::take

* core: revert authority set changes on failed block finalization

* core: grandpa: add docs to import_justification

* core: warn on re-finalization of last finalized block

* core: only notify sync with last finality notification

* core: style fix

* core: add docs for PendingJustifications

* core: network: use BlockRequest messages for justification requests

* core: reference issues in todo comments

* core: grandpa: revert authority set changes on db

* core: grandpa: remove inconsistent state warning
2019-01-21 07:04:01 +01:00
Robert Habermeier a7a0121b09 mark GRANDPA and consensus stuff as unstable (#1469) 2019-01-17 22:07:54 +01:00
thiolliere beeacf9cfa impl HasCompact for Perbill and Permill (#1411)
* impl Compact<> and HasCompact for Permill Perbill

* update parity-codec to 2.2

* add Cargo.lock

* add lock and build for runtime

* rebuild Cargo.lock after rebase
2019-01-17 11:25:48 +01:00
Benjamin Kampmann 7610f64a85 Properly log sealing and the newly created proposal hash from aura (#1397)
* Properly log sealing and the newly created proposal hash from aura

* Update lib.rs
2019-01-11 15:20:40 +01:00
Shawn Tabrizi c4010edb75 Updating parking_lot dependencies to 0.7.1 (#1361) 2019-01-08 11:15:42 +01:00
Wei Tang 71d889b692 Make AuthorityId generic (#1296)
* BlockAuthorityId convenience type

* Rename AuthorityId -> Ed25519AuthorityId to make it more precise

* Generalize AuthorityId up to substrate-client

* Fix in client-db

* rename: BlockAuthorityId -> AuthorityIdFor

* typo: should be digest item

* Fix test-runtime authorityId mismatch

One states that AuthorityId is u64 while the other states that it's Ed25519AuthorityId.

* Fix more u64 - Ed25519AuthorityId mismatch

* Fix compile of most of the srml modules

* Continue to pin aura and grandpa with ed25519 and fix compile

* Add MaybeHash trait

* Fix node-runtime compile

* Fix network tests
2019-01-08 11:14:18 +01:00
Wei Tang ac1be0665e Support custom fork choice rule (#1339)
* Support custom fork choice rule

* Remove unneeded reexport

* Fix network compile
2019-01-04 20:34:58 +01:00
Gregory Terzian 3add75910a move import queue to consensus-common (#1282) 2018-12-31 12:33:21 +01:00
Robert Habermeier 35ba22cdd1 refactor aura slot timers a bit, add additional guards (#1303) 2018-12-29 15:29:09 +01:00
André Silva f8f932d123 node: fix shutdown (#1308)
* node: remove grandpa authority flags

* node: exit-guard grandpa and aura spawned futures

* node: wait for futures to stop running on shutdown

* core: run connectivity tests on same ports

* core: pass on_exit future when starting aura and grandpa

* node: add issue number to todo

* core: fix aura and grandpa tests
2018-12-21 17:30:45 +01:00
Simon Littlejohns 1c104e0568 Consensus data added with generic use in service/src/consensus.rs. Th… (#1274)
* Consensus data added with generic use in service/src/consensus.rs. This may need to change.

* refactor consensus service a bit
2018-12-17 18:51:40 +01:00
Robert Habermeier 20782baeef add on_exit parameter to aura thread builder (#1249) 2018-12-11 13:43:53 +01:00
Robert Habermeier 6299b42a4d Punish offline validators, aura-style (#1216)
* make offline-reporting infrastructure more generic

* add a listener-trait for watching when the timestamp has been set

* prevent inclusion of empty offline reports

* add test for exclusion

* generate aura-offline reports

* ability to slash many times for being offline "multiple" times

* Logic for punishing validators for missing aura steps

* stub tests

* pave way for verification of timestamp vs slot

* alter aura import queue to wait for timestamp

* check timestamp matches seal

* do inherent check properly

* service compiles

* all tests compile

* test srml-aura logic

* aura tests pass

* everything builds

* some more final tweaks to block authorship for aura

* switch to manual delays before step

* restore substrate-consensus-aura to always std and address grumbles

* update some state roots in executor tests

* node-executor tests pass

* get most tests passing

* address grumbles
2018-12-10 18:37:08 +01:00
Tomasz Drwięga 507c13e31c Move pool maintainance to a background task. (#1236)
* Move pool maintainance to a background task.

* Remove commented code.

* Add issue number to TODOs

* Fix warnings and tests.
2018-12-10 14:13:48 +01:00
André Silva e779eeb2ec Grandpa validator set handoff justification (#1190)
* core: make block justification optional

* runtime: update wasm binaries

* core: optionally pass justification on finalize_block

* finality-grandpa: add channel to trigger authority set changes

this will allow the `BlockImport` to trigger an authority set change when
importing a change block that provides a justification (when syncing)

* finality-grandpa: move finalize_block to free function

* finality-grandpa: add GrandpaOracle for auth set liveness checking

this will be used by `BlockImport` to check whether the authority set for a
given block is still live, if the authority set isn't live then importing a
change block requires a justification.

* finality-grandpa: store justification on finalized transition blocks

* finality-grandpa: check justification on authority set change blocks

* finality-grandpa: poll grandpa liveness oracle every 10 seconds

* finality-grandpa: spawn grandpa oracle in service setup

* core: support multiple subscriptions per consensus gossip topic

* finality-grandpa: create and verify justifications

* finality-grandpa: update to local branch of grandpa

* finality-grandpa: update to finality-grandpa v0.5.0

* finality-grandpa: move grandpa oracle code

* finality-grandpa: fix canonality check

* finality-grandpa: clean up error handling

* finality-grandpa: fix canonical_at_height

* finality-grandpa: fix tests

* runtime: update wasm binaries

* core: add tests for finalizing block with justification

* finality-grandpa: improve validation of justifications

* core: remove unused IncompleteJustification block import error

* core: test multiple subscribers for same consensus gossip topic

* Revert "finality-grandpa: improve validation of justifications"

This reverts commit 51eb2c58c2219801e876af6d6c9371bdd9ff2477.

* finality-grandpa: fix commit validation

* finality-grandpa: fix commit ancestry validation

* finality-grandpa: use grandpa v0.5.1

* finality-grandpa: add docs

* finality-grandpa: fix failing test

* finality-grandpa: only allow a pending authority set change per fork

* finality-grandpa: fix validator set transition test
2018-12-08 13:34:59 +08:00