Commit Graph

101 Commits

Author SHA1 Message Date
Robert Habermeier 6299b42a4d Punish offline validators, aura-style (#1216)
* make offline-reporting infrastructure more generic

* add a listener-trait for watching when the timestamp has been set

* prevent inclusion of empty offline reports

* add test for exclusion

* generate aura-offline reports

* ability to slash many times for being offline "multiple" times

* Logic for punishing validators for missing aura steps

* stub tests

* pave way for verification of timestamp vs slot

* alter aura import queue to wait for timestamp

* check timestamp matches seal

* do inherent check properly

* service compiles

* all tests compile

* test srml-aura logic

* aura tests pass

* everything builds

* some more final tweaks to block authorship for aura

* switch to manual delays before step

* restore substrate-consensus-aura to always std and address grumbles

* update some state roots in executor tests

* node-executor tests pass

* get most tests passing

* address grumbles
2018-12-10 18:37:08 +01:00
Tomasz Drwięga 507c13e31c Move pool maintainance to a background task. (#1236)
* Move pool maintainance to a background task.

* Remove commented code.

* Add issue number to TODOs

* Fix warnings and tests.
2018-12-10 14:13:48 +01:00
Bastian Köcher 1f6719346f Make runtime api generate version and identifier information (#1226)
* Make `decl_runtime_apis!` implement `RuntimeApiInfo` for all runtime
apis

* Make the runtime side generate the info constants as well

* Make `RuntimeApiInfo` implementation use the correct generics

* Adds a test for the runtime api info stuff

* Remove duplicated code by using block from `test-client`

* Adds `compile_fail` tests for `api_version`

* Adds documentation for `api_version`

* Make `impl_runtime_apis!` generate `RUNTIME_API_VERSIONS`

* Update documentation and tests for `RUNTIME_API_VERSIONS`

* Implement `has_api` by using the `RuntimeApiInfo`

* Make `impl_runtime_apis` check that trait identifiers are unique

* Prefix all runtime api function with the corresponding trait

So `execute_block` will be called `Core_execute_block`.
This makes it possible to have traits implement a function with the
same name.

* Rebase master

* Update after master rebase
2018-12-10 11:48:07 +01:00
André Silva e779eeb2ec Grandpa validator set handoff justification (#1190)
* core: make block justification optional

* runtime: update wasm binaries

* core: optionally pass justification on finalize_block

* finality-grandpa: add channel to trigger authority set changes

this will allow the `BlockImport` to trigger an authority set change when
importing a change block that provides a justification (when syncing)

* finality-grandpa: move finalize_block to free function

* finality-grandpa: add GrandpaOracle for auth set liveness checking

this will be used by `BlockImport` to check whether the authority set for a
given block is still live, if the authority set isn't live then importing a
change block requires a justification.

* finality-grandpa: store justification on finalized transition blocks

* finality-grandpa: check justification on authority set change blocks

* finality-grandpa: poll grandpa liveness oracle every 10 seconds

* finality-grandpa: spawn grandpa oracle in service setup

* core: support multiple subscriptions per consensus gossip topic

* finality-grandpa: create and verify justifications

* finality-grandpa: update to local branch of grandpa

* finality-grandpa: update to finality-grandpa v0.5.0

* finality-grandpa: move grandpa oracle code

* finality-grandpa: fix canonality check

* finality-grandpa: clean up error handling

* finality-grandpa: fix canonical_at_height

* finality-grandpa: fix tests

* runtime: update wasm binaries

* core: add tests for finalizing block with justification

* finality-grandpa: improve validation of justifications

* core: remove unused IncompleteJustification block import error

* core: test multiple subscribers for same consensus gossip topic

* Revert "finality-grandpa: improve validation of justifications"

This reverts commit 51eb2c58c2219801e876af6d6c9371bdd9ff2477.

* finality-grandpa: fix commit validation

* finality-grandpa: fix commit ancestry validation

* finality-grandpa: use grandpa v0.5.1

* finality-grandpa: add docs

* finality-grandpa: fix failing test

* finality-grandpa: only allow a pending authority set change per fork

* finality-grandpa: fix validator set transition test
2018-12-08 13:34:59 +08:00
Bastian Köcher ed421c56ee Rewrite impl_runtime_apis! and decl_runtime_apis! as proc-macro (#1174)
* Rewrites `impl_runtime_apis!` macro as `proc-macro`

* Adds some documentation

* Require the `impl_runtime_apis` to use a path for accessing the trait

* Make the runtime implement `GetNodeBlockType`

* Moves first chunk of runtime api code into the `impl_runtime_apis` macro

This also renames `ClientWithApi` into `RuntimeApi`.

* Make `impl_runtime_apis` use `runtime` api version automatically

* `decl_runtime_apis` automatically adds `Block: BlockT` as generic parameter

* Remove function generic arguments in block builder api

* Remove some unnused stuff from the `decl_runtime_apis` macro

* Make `InherentData` working again

* Make `impl_runtime_apis!` implement the `RuntimeApi` side as well

* Make it compile again after rebasing with master

* Split `sr-api-macros` into multiple files

* Reimplement `decl_runtime_apis!` as proc_macro

* Use `decl_runtime_apis!` for `Core` as well and improve error reporting

* Adds documentation for `decl_runtime_apis!` and `impl_runtime_apis!`

* Move some code

* Adds compile fail tests

* Adds a test and fixes some bugs

* Make `impl_runtime_apis!` support `_` as parameter name

* Fixes build errors with wasm

* Wasm rebuild after master rebase

* Apply suggestions from code review

Co-Authored-By: bkchr <bkchr@users.noreply.github.com>

* Addresses some grumbles

* Adds test to ensure that method signatures need to match

* New wasm files
2018-11-30 11:42:46 +01:00
Guanghua Guo 9ee6bdd203 Misspelled words (#1169) 2018-11-27 10:56:54 +01:00
Benjamin Kampmann dbe31230c3 Simplify consensus message sending (#1166)
* Create gossip_consensus_message helper on service, simplify consensus_gossip access
2018-11-27 09:06:58 +01:00
Robert Habermeier 59af4de4fd GRANDPA: add commit messages (#1151)
* get compiling with latest version of grandpa

* generalize UntilImported to prepare for waiting for commit message targets

* extract until_imported out to own module

* logic for blocking commits until enough blocks imported

* add tests for commit message blocking logic

* pass through commit mesage round number as well

* extract communication streams to own module

* add Error implementation for ExitOrError

* introduce stream adapter for checking commit messages

* output sink for commits

* implement the unimplemented

* remove extra line

* update to latest version of grandpa api

* update finality-grandpa to 0.4.0

* Use filter_map earlier when checking incoming commits messages

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* address some grumbles
2018-11-26 16:24:26 +01:00
Robert Habermeier 11fe84a742 Minimal switch of substrate-node to GRANDPA /Aura (#1128)
* add beginnings of SRML grandpa library

* get srml-grandpa compiling

* tests for srml-grandpa

* add optional session integration to grandpa SRML

* start integration into node runtime

* Allow extracting pending change from header digest

* Make it compile on wasm

* make tests compile again

* Move Authority Key fetching into service, simplify service factory construction

* Generalize Authority Consensus Setup system

* Add Authority Setup Docs

* Allow CLI params to be extensible

 - move params to structopts
 - split parsing and default command execution
 - add custom config to node
 - extended parsing of custom config
 - extending params via structop's flatten

* Minor fixes on cli extension params:
 - added docs
 - re-add actual app name, rather than node-name
 - make strategy and subcommand optional

* better cli params

* synchronize GRANDPA and normal node authorities

* Implement grandpa::network for gossip consensus

* run_grandpa in Node

* Fix missed merge error

* Integrate grandpa import queue

* more specific type def

* link up linkhalf and import block

* make grandpa future send

* get compiling

* Fix new params convention and license header

* get it running

* rebuild node runtime WASM

* change logging level

* Update node/cli/src/params.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/cli/src/params.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/cli/src/lib.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/runtime/src/lib.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update node/cli/src/lib.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Clean up and Fixme for mutable config

* Move GrandpaService Integration into grandpa, feature gated but on per default

* Fixing grandpa runtime module test

* Update wasm runtime hashes for tests

* GRANDPA: use post-header hash when logging scheduled changes

* add an extra bit of logging to authorities

* fixing missing constrain

* remove old code

* move `NewAuthorities` to an event in srml-grandpa

* fix node-executor tests to use grandpa log

* Remove GossipConsensus from tests, use newly provided sync-feature, fixes tests

* Update to latest wasm runtimes

* address grumbles

* address grumbles

* only derive deserialize when using std

* Clean up use of Deserialize
2018-11-21 18:42:50 +01:00
Robert Habermeier 5d5bbcccfb rename fg-primitives to more verbose name (#1123) 2018-11-16 08:36:04 +01:00
Bastian Köcher 276ec5f85d Try to get grandpa tests compiling again (#1121)
* Revert "start to refactor block import wrapper a bit"

This reverts commit b919de49e1a82422a0870b66100832d531174771.

* Refactor grandpa stuff to get tests almost compiling

* Fixes after rebase

* Fixes merge compile errors
2018-11-16 07:44:51 +01:00
Robert Habermeier 1b202e1ffb start to refactor block import wrapper a bit 2018-11-15 18:36:16 +01:00
Robert Habermeier d2e928a39f more issues due to core API reshuffle 2018-11-15 18:09:03 +01:00
Robert Habermeier 913513d983 Merge branch 'rh-grandpa-dynamic2' of github.com:paritytech/substrate 2018-11-15 17:49:36 +01:00
Bastian Köcher 3cd2739f66 Fixes GrandpaApi implementation and usage 2018-11-15 17:34:55 +01:00
Robert Habermeier 4307a55505 special case equality in ancestry function 2018-11-15 17:03:39 +01:00
Robert Habermeier 1cb0bbc48b start addressing basti comments 2018-11-15 16:36:28 +01:00
Robert Habermeier 7e088f6342 address grumbles 2018-11-14 14:34:32 +01:00
Robert Habermeier e37ab01482 Merge branch 'master' into rh-grandpa-dynamic2 2018-11-14 14:32:30 +01:00
Bastian Köcher 9063d1acae Implement runtime api client side directly in the runtime (#1094)
* Move `initialise_block` into `Core` trait as it is crucial calling the API functions

* Switch to first version of new runtime API implementation

* Fixes bug in tests

* Reenable asserts

* Directly use the `TestAPI` in the tests

* Start improving the api traits

:100644 100644 898aadc7 49217199 M	Cargo.lock
:100644 100644 61570436 465ed664 M	core/client/src/backend.rs
:100644 100644 5d0c886b 64d710fd M	core/client/src/block_builder.rs
:100644 100644 c447855e 5ecbe474 M	core/client/src/client.rs
:100644 100644 139cef13 f90dbf3d M	core/client/src/error.rs
:100644 100644 2800c503 3298e66a M	core/client/src/runtime_api.rs
:100644 100644 affa1c5c 809b08bc M	core/primitives/src/lib.rs
:100644 100644 2877dfa9 d5547413 M	core/sr-api/Cargo.toml
:100644 100644 9a49784d 6a625a03 M	core/sr-api/src/lib.rs
:100644 100644 7c28e1c7 a1a444a9 M	core/sr-primitives/src/traits.rs
:100644 100644 2e113ab6 dcc01a6d M	srml/metadata/Cargo.toml
:100644 100644 ea722a70 0809531a M	srml/metadata/src/lib.rs

* Refactoring

* Move `sr-api` into client and more refactoring

* Fixes tests

* Some documentation and cleanup

* Fixes compilation after rebase

* More refactoring and more documentation

* Makes `substrate-client` compilable on `wasm`

On `wasm` it basically just exports the runtime api stuff.

* Fixes grumbles

* Updates wasm files after rebasing the master

* Remove TODO comment

* Remove whitespaces

* Fixes after rebasing master

* Another rebase, another fix commit
2018-11-13 13:33:28 +01:00
Luke Schoen 037f9dde10 docs: Extends #999. Remove disused package readmes. Rename to sr-metadata. Fix Readme Rustdoc names (#1016)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme

* docs: Remove link to old docs

* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names

* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory

* Change example to use package name that opens all packages in RustDocs

* Fix list of packages names that are available to open in RustDocs

* fix typo

* fix ordering of CLI options and add missing backslash
2018-11-12 21:18:06 +01:00
Robert Habermeier 876969d6f0 add some docs 2018-11-05 11:27:01 +01:00
Robert Habermeier dc59431474 API id module 2018-11-03 16:45:18 +01:00
Robert Habermeier 21f291a607 Merge branch 'master' into rh-grandpa-dynamic2 2018-11-01 20:31:13 +01:00
Robert Habermeier 5b2a93821c use published finality-grandpa 2018-11-01 20:25:59 +01:00
Robert Habermeier 6704f15e99 finish GRANDPA test: dynamic authority sets 2018-11-01 20:15:51 +01:00
Wei Tang b21de8a0b5 Support multi trie in genesis generation (#958)
* Support multi trie in genesis generation

* Fix merge issues
2018-11-01 09:30:03 +01:00
Robert Habermeier c64ff78776 initial test to ensure auxiliary data is generated correctly 2018-10-31 12:39:27 +01:00
Robert Habermeier fe6dd131f0 tests compile after changes 2018-10-30 23:46:27 +01:00
Robert Habermeier afbe00db75 extract out tests to file 2018-10-29 17:05:42 +01:00
Robert Habermeier 6d4acb053d tests compile using new test framework 2018-10-29 17:03:08 +01:00
Robert Habermeier e2ee6a2b90 read authority set from DB on startup 2018-10-29 02:31:51 +01:00
Robert Habermeier 3317fee0a1 write authority set to DB 2018-10-29 02:05:47 +01:00
Robert Habermeier 55ca13ab09 call into chain WASM to check for changes 2018-10-28 20:50:31 +01:00
Robert Habermeier 442e300163 add substrate finality grandpa primitives for WASM 2018-10-28 20:37:09 +01:00
Robert Habermeier 66dafbcb0b minimize public interface 2018-10-28 18:45:42 +01:00
Robert Habermeier ab6c343aaf Merge branch 'master' into rh-grandpa-dynamic2 2018-10-27 18:12:33 +02:00
Robert Habermeier 1a3fe5657d restart voter with new authority set as necessary 2018-10-27 17:32:42 +02:00
Robert Habermeier 25984d8dc8 add some tests for authority set logic 2018-10-27 16:18:01 +02:00
Robert Habermeier e33a8dde4a compiles 2018-10-27 15:43:03 +02:00
Robert Habermeier 5f6dc6bc6a apply authority set changes 2018-10-26 19:19:12 +02:00
Luke Schoen f923e38cc7 docs: Relates to Issue #995. Fix links in existing documentation (#999)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core
2018-10-26 11:09:48 +02:00
Robert Habermeier a422a14b52 use generic block number in GRANDPA 2018-10-25 18:00:46 +02:00
Robert Habermeier 08890f2bc1 note future changes when importing block 2018-10-25 16:48:51 +02:00
Robert Habermeier 515153fa85 localize messages to set-index and add pending changes 2018-10-21 11:11:53 +02:00
Robert Habermeier 1c75f8059a round-localized signatures on GRANDPA messages 2018-10-17 14:52:08 -07:00
Robert Habermeier b255f8f987 grandpa-compatible digest items 2018-10-17 14:51:54 -07:00
Gav Wood 8bc5242c92 Compact format for block number, nonce, balance (#910)
* Try to make everything compact

* Make `Clone` work

* Fix

* Move contracts and balances over to compact encoding

* Session and timestamp are compact

* Sataking uses compact numbers

* Treasury now compact

* Compact Democracy

* Council is compact

* Fix
2018-10-16 22:47:30 +02:00
Benjamin Kampmann ac4bcf879f Generalize the Consensus Infrastructure (#883)
* Split out Consensus
* Supply ImportQueue through network-service
  - simplify ImportQueue.import_blocks
  - remove Deadlock on import_block
  - Adding Verifier-Trait
  - Implement import_queue provisioning in service; allow cli to import
* Allow to actually customize import queue
* Consensus Gossip: Cache Message hash per Topic
2018-10-16 13:40:33 +02:00
Robert Habermeier 02f8897648 Fix grumbles from previous GRANDPA PR (#880)
* implement grandpa client

* consensus gossip with arbitrary topics

* defer GRANDPA messages until referenced blocks imported

* set up communication for voter in a transparent way

* instantiate GRANDPA voter

* keep last round state on disk

* switch back to crates.io finality-grandpa

* update cargo.lock

* use new `collect_garbage` API

* update sync test framework and make public

* test that observers can observe

* fix warning

* use more idiomatic predicate for collecting garbage in gossip

* kill spaces

* fix date

* fatal error when unable to complete

* rename run_voter to run_grandpa

* switch back to crates.io
2018-10-05 14:26:28 -04:00