Commit Graph

60 Commits

Author SHA1 Message Date
Ashley 1baee72d75 use pdqselect for median_algorithm selection (#3636)
* use pdqselect for median_algorithm selection instead of sorting the whole vec

* Make use of pqdselect clearer

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Make use of pqdselect clearer

Co-Authored-By: André Silva <andre.beat@gmail.com>
2019-09-23 22:37:53 +02:00
Robert Habermeier c200ce757b Fixing BABE epochs to change between blocks (#3583)
* always fetch epoch from runtime

* node integration tests don't test light nodes

* give stand-in full node a FULL role

* rejig babe APIs

* introduce next-epoch-descriptor type

* overhaul srml-BABE epoch logic

* ensure VRF outputs end up in the right epoch-randomness

* rewrite `do_initialize` to remove unnecessary loop

* begin accounting for next epoch in epoch function

* slots passes header to epoch_data

* pass slot_number to SlotWorker::epoch_data

* begin extracting epoch-change logic into its own module

* aux methods for block weight

* aux methods for genesis configuration

* comment-out most, refactor header-check pipeline

* mostly flesh out verifier again

* reinstantiate babe BlockImport implementation

* reinstate import-queue instantiation

* reintroduce slot-worker implementation

* reinstate pretty much all the rest

* move fork-choice logic to BlockImport

* fix some, but not all errors

* patch test-runtime

* make is_descendent of slightly more generic

* get skeleton compiling when passing is_descendent_of

* make descendent-of-builder more succinct

* restore ordering of authority_index / slot_number

* start fiddling with tests

* fix warnings

* improve initialization architecture and handle genesis

* tests use correct block-import

* fix BABE tests

* fix some compiler errors

* fix node-cli compilation

* all crates compile

* bump runtime versions and fix some warnings

* tweak fork-tree search implementation

* do backtracking search in fork-tree

* node-cli integration tests now work

* fix broken assumption in test_connectivity

* babe tests fail for the right reasons.

* test genesis epoch logic for epoch_changes

* test that epochs can change between blocks

* First BABE SRML test

* Testing infrastructure for BABE

Also includes a trivial additional test.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* A little more test progress

* More work on BABE testing

* Try to get the tests working

* Implement `UintAuthorityId`-based test mocks

* Fix compilation errors

* Adjust to upstream changes

* Block numbers are ignored in BABE epoch calculation

* authority_index() should ignore invalid authorities

* Fix compile error

* Add tests that session transitions happen

* Check if BABE produces logs

It currently does not.

* Fix test suite

This was really nasty, due to a type confusion that showed up as an
off-by-1 buffer error.

* Add additional tests

Most of these were derived from the current output, so they are only
useful to guard against regressions.

* Make the tests more readable

Also bump impl_version.

* Fix excessive line width

* Remove unused imports

* Update srml/babe/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* try to fix imports

* Fix build errors in test suite

* tests did not pass

* Try to get at least one digest to be output

Currently, the code emits either no digests (if I don’t call
`Session::rotate_session()` or two digests (if I do), which is wrong.

* More tests

They still don’t work, but this should help debugging.

* fix silly error

* Don’t even try to compile a broken test

* remove broken check_epoch test and add one for genesis epoch

* Check that the length of the pre-digests is correct

* Bump `impl_version`

* use epoch_for_descendent_of even for genesis

* account for competing block 1s

* finish srml-babe docs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* address grumbles
2019-09-23 16:03:05 +02:00
Svyatoslav Nikolsky eba6dd73c6 Blockchain cache pruning strategy (#3395)
* blockchain cache pruning strategy

* added some internal docs to cache_pruning_strategy

* Update core/client/db/src/cache/mod.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
2019-09-20 09:49:15 +03:00
Weiliang Li d105d3f3a1 fn import_block -> fn block_import_params in SimpleSlotWorker (#3647) 2019-09-19 15:49:24 +01:00
Svyatoslav Nikolsky 010395e620 Never panic during execution proof check (#3504)
* do not panic during execution proof check

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* BackendTrustLevel enum

* up runtime version

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* fixed some grumbles

* Update core/state-machine/src/testing.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update core/state-machine/src/lib.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* mov where

* spaces -> tabs (to restart build)
2019-09-11 10:27:32 +03:00
Benjamin Kampmann 0cae7217d8 Remove client.backend (#2960)
* generalize tree_root to remove client.backend dependency

* replace client.backend.blockchain.header with client.header

* move used_state_cache_size into client info

* Create intermediate Setup State. Fixes #1134

* remove client.backend from finality proof

* update node-template

* move memory backend into test helper mode

* move test helper into client

* starting the big refactor, remove unused functions

* apply_finality

* apply_finality

* replacing more .backend from environment with client directly

* remove .backend from grandpa by using traits

* remove .backend from babe

* remove .backend from tests where it is not needed

* remove .backend from tests

* fixing tests

* fixing tests

* fixing more tests

* fixing tests

* fix all forks test

* fix style

* fixing unnecessary allocation

* remove old test.

* fix service docs

* apply suggestion

* minor clean ups

* turns out the test-helper features actually is being used!

* fixing line length.

* fix line length

* minor cleaning

* Apply suggestions from code review

thanks, @Basti

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* address grumbles

* simplify finalize block on client

* move block back into inner function

* Apply suggestions from code review

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* use as.ref instead of match

* Update core/client/src/backend.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
2019-08-30 02:20:26 +02:00
Svyatoslav Nikolsky 6fb721e861 Value lifetime is returned from blockchain cache (#3403)
* value range in blockchain cache

* revert me (testing for spurious failure)

* Revert "revert me (testing for spurious failure)"

This reverts commit 21a4a3cf5ee14e003541b779c41351e4f5e1122a.
2019-08-19 13:44:16 +02:00
André Silva cb7527d2b2 babe: Introduce secondary slots (Aurababeous) (#3380)
* babe: initial implementation of secondary slots

* babe: validate secondary slot author

* babe: implement weight based fork choice

* babe: remove unused

* aura: cleanup unused imports

* babe: pass in parent weight when authoring and verifying

* babe: use epoch randomness for picking secondary slot authors

* babe: fix tests

* babe: fix wasm build

* babe: node-side code for disabling secondary slots

* babe: allow enabling/disabling secondary slots from runtime

* babe: fix test

* babe: use blake2_256 for secondary slot assignment

* babe: run block initialization in should_end_session

* node: increase slot duration to 6s

* babe: add docs

* node: bump spec_version

* Apply suggestions from code review

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* babe: simplify secondary slot assignment calculation

* babe: remove unnecessary comment

* node: bump spec_version

* babe: fix bad merge
2019-08-16 15:47:53 +02:00
Svyatoslav Nikolsky 3825a21bac Fix light client synchronization on master (#3301)
* value ranges in consensus cache

* skip values in cache

* read epoch0 + epoch1 data from genesis in babe

* sync authorities + session validators at genesis

* removed some debug printlns

* fixed cache encoding

* Revert "skip values in cache"

This reverts commit ce451c32823aaa4b67d99ca5b58f1bf3984df4db.

* Revert "value ranges in consensus cache"

This reverts commit 9062f9434cddd14a01275ddbfcd904b04282e63b.

* get rid of cache::AUTHORITIES in Babe

* cleaning up

* cleaning up

* update spec version

* lost changes

* fixed tests

* Update node/runtime/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* fix once-per-block condition

* fix standalone babe + temp_storage in BuildGenesis

* fix benhes compilation

* fixed comment

* re-added light nodes to integration tests

* finalize_with_ancestors from extra_requests

* post-merge fix

* aaand removed debug code

* (another one)

* fix warn in logs (do not call ForkTree::finalize twice for the same block)

* sync digest.next_authorities with actual next authorities

* more docs

* reverting all commits affecting storage

* also remove keys from babe trait

* fixed warnings

* post-merge fixes

* reverted some redundant changes

* reverted more changes
2019-08-16 11:51:49 +02:00
André Silva 6058207935 consensus: refactor aura and babe proposer (#3377) 2019-08-15 08:20:39 +02:00
Marcio Diaz 7b45130115 Add transaction pool to Aura and Babe import queue (#3225)
* Add transaction pool to babe import queue

* Add transaction pool to Babe check header

* Fix tests

* Add tx pool to Aura import_queue

* Fix tests, node-template

* Add comments regarding unused _transaction_pool

* Make tx pool optional in check_header
2019-08-13 10:44:00 +01:00
Gavin Wood 14bb115193 Scale trait and move to u32 blocknumbers (#3357)
* Scale trait and move to u32 blocknumbers.

* Fixes

* Cleanups

* Update node/runtime/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix up some of the factory stuff.

* Update core/sr-primitives/src/traits.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Move Nonce/Index to u32 (#3361)

* Force a non-borked version of upstream crate

* Line lengths and runtime version bump
2019-08-11 15:03:06 +02:00
Arkadiy Paronyan 58bd0d4c05 Set uncles inherent (#3317)
* Include uncles

* Filter missing uncles

* Moved inherent registration to a new crate

* Ignore invalid inherent encoding
2019-08-08 00:56:29 +02:00
Pierre Krieger 97febf4c30 Make Verifier::verify mutable (#3165)
* Make Verifier::verify mutable

* Fix GrandPa tests

* Fix doctest

* Fix more doctests
2019-08-07 21:21:44 +02:00
Gavin Wood 1a524b8207 Refactor key management (#3296)
* Add Call type to extensible transactions.

Cleanup some naming

* Merge Resource and BlockExhausted into just Exhausted

* Fix

* Another fix

* Call

* Some fixes

* Fix srml tests.

* Fix all tests.

* Refactor crypto so each application of it has its own type.

* Introduce new AuthorityProvider API into Aura

This will eventually allow for dynamic determination of authority
keys and avoid having to set them directly on CLI.

* Introduce authority determinator for Babe.

Experiment with modular consensus API.

* Work in progress to introduce KeyTypeId and avoid polluting API
with validator IDs

* Finish up drafting imonline

* Rework offchain workers API.

* Rework API implementation.

* Make it compile for wasm, simplify app_crypto.

* Fix compilation of im-online.

* Fix compilation of im-online.

* Fix more compilation errors.

* Make it compile.

* Fixing tests.

* Rewrite `keystore`

* Fix session tests

* Bring back `TryFrom`'s'

* Fix `srml-grandpa`

* Fix `srml-aura`

* Fix consensus babe

* More fixes

* Make service generate keys from dev_seed

* Build fixes

* Remove offchain tests

* More fixes and cleanups

* Fixes finality grandpa

* Fix `consensus-aura`

* Fix cli

* Fix `node-cli`

* Fix chain_spec builder

* Fix doc tests

* Add authority getter for grandpa.

* Test fix

* Fixes

* Make keystore accessible from the runtime

* Move app crypto to its own crate

* Update `Cargo.lock`

* Make the crypto stuff usable from the runtime

* Adds some runtime crypto tests

* Use last finalized block for grandpa authority

* Fix warning

* Adds `SessionKeys` runtime api

* Remove `FinalityPair` and `ConsensusPair`

* Minor governance tweaks to get it inline with docs.

* Make the governance be up to date with the docs.

* Build fixes.

* Generate the inital session keys

* Failing keystore is a hard error

* Make babe work again

* Fix grandpa

* Fix tests

* Disable `keystore` in consensus critical stuff

* Build fix.

* ImOnline supports multiple authorities at once.

* Update core/application-crypto/src/ed25519.rs

* Merge branch 'master' into gav-in-progress

* Remove unneeded code for now.

* Some `session` testing

* Support querying the public keys

* Cleanup offchain

* Remove warnings

* More cleanup

* Apply suggestions from code review

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* More cleanups

* JSONRPC API for setting keys.

Also, rename traits::KeyStore* -> traits::BareCryptoStore*

* Bad merge

* Fix integration tests

* Fix test build

* Test fix

* Fixes

* Warnings

* Another warning

* Bump version.
2019-08-07 20:47:48 +02:00
Bastian Köcher 394eff6c2f Update to schnorrkel 0.8.0 (#3267)
* Update to schnorrkel `0.8.0`

* Increase `spec_version`

* Bump schnorrkel to 0.8.3 (#3283)

* Schnorrkel 0.8.1 (builds on bkchr upgrade branch)

* Add tests for known hard/soft derivation values

* Bump all schnorrkel versions

* Flatten for easier inspection

* 0.8.2

* 0.8.3

* Update subkey/Cargo.toml

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update `Cargo.lock`

* 0.8.4

* Fix cargo lock file.

* Adding an error message for `NotMarkedSchnorrkel` as that was missing.
Also fixing a typo, strage -> stage.
2019-08-07 12:59:54 +02:00
thiolliere 4ed67e03a4 Update to parity-scale-codec (#3232)
* WIP: update codec

* WIP

* compiling

* WIP

* rename parity-scale-codec to codec

* WIP

* fix

* remove old comments

* use published crates

* fix expected error msg

* bump version

* fmt and fix

* remove old comment

* fix wrong decoding impl

* implement encode like for structures

* undo removal of old pending changes

* trailingzeroinput

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* update codec

* fmt

* version is 1.0.0

* show more error

* fmt
2019-08-06 19:36:23 +02:00
André Silva aa86185648 node: use BABE predigest data to find authorship (#3293)
* node: use BABE predigest data to find authorship

* node: bump spec_version

* babe: use u32 for authority index
2019-08-02 19:52:36 +02:00
Tomasz Drwięga 04c115a6e9 Revert "Update schnorrkel to 0.7.0 (#3261)" (#3264)
This reverts commit a5fa1ddd99.
2019-07-31 00:19:32 +02:00
Bastian Köcher a5fa1ddd99 Update schnorrkel to 0.7.0 (#3261)
* Update schnorrkel to `0.7.0`

* Apply suggestions from code review

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Increase `spec_version`
2019-07-30 17:46:05 -04:00
André Silva bd052d8da4 babe: Disable unused median calculation (#3251)
* babe: don't execute median algorithm

* babe: checked sub in median calculation

* node: bump impl_version to make ci happy

* babe: remove forbid dead_code
2019-07-30 22:08:20 +01:00
Kian Paimani 79feb23a22 Refactor sr_primitives. (#3214)
* refactor sr_primitives.

* Fix try build error.

* Line-width

* Ui test.

* Final fixes.

* Fix build again.

* bring back ui test.

* Fix unsigned import.

* Another ui fix.

* Also refactor substrate-primitives

* Fix benchmarks.

* Fix doc test.

* fix doc tests
2019-07-29 14:43:53 +02:00
Pierre Krieger 9370a4a6b6 Clean up the basic-authorship crate (#3206)
* Switch consensus-common to new futures

* Fix tests

* More tests fixing

* Make Proposer, OnSlot and SyncOracle mut

* Make the Environment mut as well

* Fix test

* Fix Babe tests

* Babe fixes

* Fix CLI service tests

* Fix Babe tests

* Remove unnecessary trait bound

* Inline the code of BlockBuilder and AuthoringApi

* Remove warning lint

* Bounds simplification

* Imports simplification

* Don't panic on bad generated block

* Code style

* Add doc example

* Remove dependency on aura

* Order dependencies alphabetically

* Minor style
2019-07-27 20:24:51 -04:00
Pierre Krieger b31dcdf342 Switch consensus crates to new futures (#3146)
* Switch consensus-common to new futures

* Fix tests

* More tests fixing

* Fix Babe tests

* Fix Babe tests
2019-07-25 10:55:50 +08:00
André Silva 9f50c8fce4 Use BABE instead of AuRa in node (#3171)
* babe: add babe module trait

* babe: track current slot and epoch start slot

* babe: implement ShouldEndSession based on epochs

* babe: rename weight type to avoid ambiguities

* babe: expose epoch start slot in Epoch digest

* babe: use epoch start for validating epoch transitions

* babe: make the epoch duration a parameter type

* babe: remove unused fields from config

* node: update runtime to use babe instead of aura

* node: use babe instead of aura

* core: generate sr25519 keys from seed and add to keystore

* core: remove AuthorityKeyring

* node: remove unused primitive types related to babe crypto

* uniform babe primitives crate import name

* wrap long lines

* babe: fix find_epoch_digest

* fork-tree: fix find_node_where

* node: set babe epoch duration to "10 minutes"

* babe: cleanup import key cache if authorities don't change

* node: make integration test compile (but fail)

* node: bump spec_version

* node: fix import

* babe: don't use constants in storage fields array sizes

* babe: account for first epoch slot way in the past

* babe: signal next epoch change (not current)

* babe: calculate next epoch randomness with next epoch index

* babe: track next epoch in node

* babe: cache current epoch and authorities separately

* babe: generate valid babe vrf proofs in integration test

* babe: cleanup claim_slot

* babe: perform threshold calculation according to spec

* babe: compute relative weight in threshold

* babe: more precise threshold calculation

* babe: use floats for threshold exponent calculation

* babe: update constant c
2019-07-24 15:53:04 -04:00
Bastian Köcher 6e4066b62d Bkchr check all benches (#3180)
* Make sure CI checks all benchmarks

* Fix babe
2019-07-23 13:57:32 +02:00
DemiMarie-parity f78a780790 BABE Epochs (#3028)
* Add `epoch` field to `SlotInfo`

* Add slot calculations

* More work on epochs in BABE

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Typo: `/` not `%` for division

* Delete useless `LastSlotInEpoch::put(false)`

* Bump `spec_version`

* Make test suite pass again

* Implement BABE epoch randomness signing

* Try to fix compilation

Currently causes a stack overflow in the compiler

* Fix rustc stack overflow

* Add missing `PartialEq` and `Eq` implementations

* Fix compile errors in test suite

* Another silly compile error

* Clone `epoch`

* Fix compile error in benchmarks

* Implement `clone` for `Epoch`

* Merge master

* AUTHORING TEST PASSES!!!

* Fix compilation

* Bump `spec_version`

* Fix compilation

* Fix compilation (again)

* Remove an outdated FIXME

* Fix run.sh and move it to scripts/

* Delete commented-out code

* Fix documentation

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Fix BABE initialization and refactor

* Respond to review

* typo

* Remove useless data in `CheckedHeader::Deferred`

* Remove `slot_number` from Epoch

It is not needed, and only served to waste space and cause confusion.

* Remove epoch from BABE digests

* Move digest.rs to primitives

* Fix incorrect warning names

* Fix compile error

* Consistent field naming for BABE digests

* More compiler error fixex

* Unbound variable

* more compile errors

* another compile error

* Fix compile errors in runtime

* another compile error

* Another compile error

* Fix wasm build

* missing import

* Fix more compile errors

* yet another compile error

* compile fix in test runtime

* Fix and simplify the BABE runtime

The BABE runtime was massively overcomplicated and also wrong.  It
assumed it needed to:

1. delay new authorities taking effect until the next epoch
2. not delay emitting `Consensus` digests to mark epoch changes

However, the first is handled by the `srml_session` crate, and the
second is flat-out incorrect: `Consensus` digests take effect
immediately.  Furthermore, `srml_babe` tried to duplicate the
functionality of `srml_session::PeriodicSession`, but did it both
clumsily and incorrectly.  Fortunately, the new code is simpler and far
more likely to be correct.

* Use `system` to get the test authorities

The genesis block used by tests defines no authorities.  Only the test
suite is affected.

* Fix test runtime impl for BabeApi::epoch() with std

* Fix compilation

* Cached authorities are in the form of an epoch

not a `Vec<AuthorityId>`.

* `slots_per_epoch` is not fixed in general

The BABE code previously assumed `slots_per_epoch` to be a constant,
but that assumption is false in general.  Furthermore, removing this
assumption also allows a lot of code to go away.

* fix compile error

* Implement epoch checker

* Fix runtime compilation

* fork-tree: add method for finding a node in the tree

* babe: register epoch transitions in fork tree and validate them

* fork-tree: add method for arbitrary pruning

* Expose the queued validator set to SRML modules

BABE needs to know not only what the current validator set is, but also
what the next validator set will be.  Expose this to clients of the
session module.

* Bump hex-literal

Hopefully this will fix the panic

* babe: prune epoch change fork tree on finality

* babe: validate epoch index on transition

* babe: persist epoch changes tree

* Fix compile error in tests

* Fix compile error in tests

* Another compile error in tests

* Fix compilation of tests

* core: move grandpa::is_descendent_of to client utils

* babe: use is_descendent_of from client utils

* babe: extract slot_number from pre_digest in import_block

* Move BABE testsuite to its own file

* Initial part of test code

* Missing `WeightMultiplierUpdate` in test-runtime

* bump `spec_version`

* Add a test that a very bogus is rejected

* Run the tests again

* Fix compiler diagnostics

* Bump `spec_version`

* Initial infrastructure for mutation testing

* Mutation testing of block import

* babe: revert epoch changes in case of block import error

* babe: fix logging target

* babe: BabeBlockImport doesn't box inner BlockImport

* babe: fix epoch check in block import

* babe: populate authorities cache on block authorship

* babe: remove unused functions

* babe: use RANDOMNESS_LENGTH const

* babe: remove unneeded config parameters

* core: revert change to hex dependency version

* cleanup gitignore

* babe: add docs to aux_schema

* babe: remove useless drops in tests

* babe: remove annoying macos smart quotes

* fork-tree: docs

* fork-tree: add tests

* babe: style

* babe: rename randomness config variable

* babe: remove randomness helper function

* babe: style fixes

* babe: add docs

* babe: fix tests

* node: bump spec_version

* babe: fix tests
2019-07-23 10:36:16 +02:00
Pierre Krieger 2edeef5825 ImportBlock -> BlockImportParams (#3158) 2019-07-21 18:43:02 -04:00
Pierre Krieger a22bb71029 Consistency with the Block trait name (#3129)
* Consistency with the Block trait name

* Line widths
2019-07-16 15:32:46 +01:00
Pierre Krieger bf2551a854 Switch the client to new futures (#3103)
* Switch the client to new futures

* No need for compat in the client

* Fix client tests

* Address review
2019-07-11 16:58:30 +02:00
Pierre Krieger efed2e3098 Remove set_finality_proof_request_builder (#3087)
* Remove set_finality_proof_request_builder

* Fix Babe

* Fix Grandpa

* Fix service doctests
2019-07-11 09:44:32 +02:00
Pierre Krieger d7b6720663 Change the import queue traits to take &mut self instead of &self (#3058)
* SharedFinalityProofRequestBuilder -> BoxFinalityProofRequestBuilder

* SharedThings -> BoxThings

* Fix tests

* build_request_data now takes &mut self

* The other traits now also take &mut self

* More or less fix tests

* Fix tests

* Fix more tests

* Moar tests

* Don't call make_block_import multiple time

* Fix doctest
2019-07-09 17:11:25 +02:00
Robert Habermeier 13b9e49688 Dispatch on-disabled digests from consensus modules (#3055)
* on-disable primitives for engines

* dispatch on-disabled digests from SRML consensus

* bump runtime versions

* use find_map
2019-07-08 16:22:15 +02:00
Pierre Krieger 1e126eab2f Network crate cleanups (#3049)
* Remove useless internal messages

* Remove NetworkService::disconnect_peer

* Remove NetworkMsg altogether

* Rename ProtocolMsg ServerToWorkerMsg

* Remove useless code

* Add example for parse_str_addr

* Move parse_str_addr and ProtocolId to config

* Don't reexport the content of config

* Rework the imports

* More reexports rework

* Add documentation

* Move finalization report to network future

* Move on_block_imported to worker

* get_value/put_value no longer locking

* local_peer_id() no longer locks

* Remove FetchFuture

* Service imports cleanup

* Produce the network state in the network task

* Merge network task and RPC network task

* Move network methods to NetworkWorker

* Remove Arc peers system from network

* add_reserved_peer now goes through the channel

* Remove Mutex around network swarm

* Remove the FnOnce alias traits

* Replace is_offline with num_connected

* Improve style of poll()

* Fix network tests

* Some doc in service module

* Remove macro export

* Minor doc changes

* Remove the synchronized() method of the import queue

* Line width

* Line widths

* Fix import queue tests

* Fix CLI tests
2019-07-08 15:33:29 +02:00
Pierre Krieger 22ec13cf65 Rewrite the network TestNet (#3016)
* Add a memory-only option for the network

* Tests cleanup

* Make grandpa/aura/babe compile

* Aura and Babe tests now passing

* More work on tests rewrite

* Attempt to fix grandpa

* Make more grandpa tests pass

* More grandpa tests work

* Work on sync tests

* More work

* light_peer_imports_header_from_announce passes

* can_sync_small_non_best_forks passes

* syncing_node_not_major_syncing_when_disconnected passes

* blocks_are_not_announced_by_light_nodes passing

* All sync tests passing 🎉

* Some TestNet cleanup

* Work on grandpa tests

* More grandpa work

* GrandPa work

* Add check about block_import

* Remove the temporarily added Sync

* Fix network tests warnings

* voter_persists_its_votes passes

* Fix imports in network tests

* Fix service tests

* Call on_block_imported 🤷

* Add shortcut

* Finish using shortcut
2019-07-05 11:19:03 +01:00
David Craven 51e345c901 Generic keystore (#3008)
* Add KeyTypeId.

* Implement clone for sr25519::Pair.

* Extend Pair with to_raw_vec.

* Implement TypedKey for Signature and Pair.

* Add trait Public.

* Make keystore generic.

* Fixup clone.

* Fix tests.

* Update service.

* Fix imports.

* Fix build.

* Fix babe build.

* Fix subkey build.

* Make authority setup generic.

* Update node-template.

* Fix build.

* Remove unsafe code.

* Fix tests.
2019-07-04 13:14:55 +02:00
DemiMarie-parity 81d8a5d01d BABE Randomness using PreRuntime digests (#2929)
* Initial work on exposing pre-runtime digests

This provides the primitive API, as well as exposing it from BABE.

* Initial work on using pre-digests in runtimes

This includes both code to expose them from `srml_system`, as well as
using it in (currently dead) code in `srml_babe`.

* Bump `{spec,impl}_version`

* Add `u64_backend` feature to curve25519-dalek

Otherwise, it errors out at compile-time.

* Bump `Cargo.lock`

* Do not depend on the schnorrkel crate in the runtime

The schnorrkel crate does not work on `#![no_std]`, but the runtime only
needs constants from it.  This adds our own definitions of those
constants, and checks them for correctness at compile-time.

* Actually implement storage of VRF outputs

* Trivial formatting change

* Provide a `hash_randomness` function in BABE

for processing VRF outputs.

* Implement a basic randomness generating function

It just XORs the VRF outputs together.

* Actually implement on-chain randomness

Blake2b is used for hashing.

* Update dependencies

* Run `cargo update` where needed

* Re-add a newline at EOF

* Remove broken and unsafe code

XOR is not a hash function, and must not be used as such.  The
implementation was also needlessly unsafe.

* Run `cargo update` where needed

* Remove spurious dependency

* Document security guarantees of BABE randomness

* Add a `RandomnessBeacon` trait

* Document `RandomnessBeacon::random`

* Fix silly compile error (unexpected type arguments)

* Fix BABE randomness

* Implement `FindAuthor` for `babe::Module`

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Respond to suggestions from code review and fix bugs

* Store an authority index, not the authority itself.
* Avoid unnecessary decoding.
* Implement relative slots and BABE randomness fully and correctly.

* Remove spurious dependency

* Fix error reported by rust-analyzer

* Update Cargo.lock files

* `wrapping_add` → `checked_add`

The epoch index will not overflow.  Panic if it does.

* Move randomness documentation to trait

* Fix compile error in test suite

* Explain 2^64 limit

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>
2019-07-03 14:49:07 +02:00
kaichao 3e2c77aa21 typo fix in core (#2987) 2019-07-01 19:33:38 +02:00
DemiMarie-parity 48aa32bece Relative slots (#2820)
* Initial work on relative slots for BABE

* More work

* Update core/consensus/babe/src/lib.rs

`Aura` → `Babe`

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* More work on relative slots

* Add missing field in test-runtime

* Bump `impl_version` and `authoring_version`

* Fix compile errors and warnings

* Upgrade dependencies

* Update dependencies more

* Revert some updates to dependencies

Somehow, those broke the build

* Fix compilation errors

* `Duration` → `u128` in calculations

* `slot_duration` is in milleseconds, not seconds

* Median algorithm: ignore blocks with slot_num < sl

* Fix silly compile error

* Store a duration, rather than an instant

It is more useful

* Fix compilation errors

* `INVERSE_NANO` → `NANOS_PER_SEC`

Also: `1000_000_000` → `1_000_000_000`

Suggested-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Un-bump `authoring_version`

* Disable median algorithm when `median_required_blocks` is 0

Otherwise it would panic.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Simplify panic

* Fix build error

* Create `SignedDuration` struct

for signed `Duration` values.

Suggested-by: Bastian Köcher

* Refactor median algorithm into separate function

* Add issues for FIXMEs and respond to code review

* Fix minor warnings
2019-06-22 12:21:29 -04:00
Bastian Köcher e09045bde2 Fix several warnings and remove some forbid(warnings) (#2874) 2019-06-15 10:30:04 +02:00
Gavin Wood bda8641892 New sessions, kill consensus module (#2802)
* Draft of new sessions

* Reintroduce tuple impls

* Move staking module to new session API

* More work on staking and grandpa.

* Use iterator to avoid cloning and tuple macro

* Make runtime build again

* Polish the OpaqueKeys devex

* Move consensus logic into system & aura.

* Fix up system module

* Get build mostly going. Stuck at service.rs

* Building again

* Update srml/staking/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Refactoring out Consensus module, AuthorityIdOf, &c.

* Refactored out DigestItem::AuthoritiesChanged. Building.

* Remove tentative code

* Remove invalid comment

* Make Seal opaque and introduce nice methods for handling opaque items.

* Start to use proper digest for Aura authorities tracking.

* Fix up grandpa, remove system::Raw/Log

* Refactor Grandpa to use new logging infrastructure.

Also make authorityid/sessionkey static. Switch over to storing
authorities in a straight Vec.

* Building again

* Tidy up some AuthorityIds

* Expunge most of the rest of the AuthorityKey confusion.

Also, de-generify Babe and re-generify Aura.

* Remove cruft

* Untangle last of the `AuthorityId`s.

* Sort out finality_tracker

* Refactor median getting

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Session tests works

* Update core/sr-primitives/src/generic/digest.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Session tests works

* Fix for staking from @dvc94ch

* log an error

* fix test runtime build

* Some test fixes

* Staking mock update to new session api.

* Fix build.

* Move OpaqueKeys to primitives.

* Use on_initialize instead of check_rotate_session.

* Update tests to new staking api.

* fixup mock

* Fix bond_extra_and_withdraw_unbonded_works.

* Fix bond_with_little_staked_value_bounded_by_slot_stake.

* Fix bond_with_no_staked_value.

* Fix change_controller_works.

* Fix less_than_needed_candidates_works.

* Fix multi_era_reward_should_work.

* Fix nominating_and_rewards_should_work.

* Fix nominators_also_get_slashed.

* Fix phragmen_large_scale_test.

* Fix phragmen_poc_works.

* Fix phragmen_score_should_be_accurate_on_large_stakes.

* Fix phragmen_should_not_overflow.

* Fix reward_destination_works.

* Fix rewards_should_work.

* Fix sessions_and_eras_should_work.

* Fix slot_stake_is_least_staked_validator.

* Fix too_many_unbond_calls_should_not_work.

* Fix wrong_vote_is_null.

* Fix runtime.

* Fix wasm runtime build.

* Update Cargo.lock

* Fix warnings.

* Fix grandpa tests.

* Fix test-runtime build.

* Fix template node build.

* Fix stuff.

* Update Cargo.lock to fix CI

* Re-add missing AuRa logs

Runtimes are required to know about every digest they receive ― they
panic otherwise.  This re-adds support for AuRa pre-runtime digests.

* Update core/consensus/babe/src/digest.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Kill log trait and all that jazz.

* Refactor staking tests.

* Fix ci runtime wasm check.

* Line length 120.

* Make tests build again

* Remove trailing commas in function declarations

The `extern_functions!` macro doesn’t like them, perhaps due to a bug in
rustc.

* Fix type error

* Fix compilation errors

* Fix a test

* Another couple of fixes

* Fix another test

* More test fixes

* Another test fix

* Bump runtime.

* Wrap long line

* Fix build, remove redundant code.

* Issue to track TODO

* Leave the benchmark code alone.

* Fix missing `std::time::{Instant, Duration}`

* Indentation

* Aura ConsensusLog as enum
2019-06-14 16:34:34 +02:00
Gavin Wood f4afdd2f0b Subkey supports 24-word phrases (#2827)
* Revamp crypto API and make seeds work better in subkey

* Final tweaks

* Update tests

* line spacing

* Avoid escapes in hex constants

* Fix build

* Another fix

* More fixes

* Minor nits
2019-06-13 11:06:30 +02:00
Pierre Krieger ce302390dd Refactoring: Remove ExtraVerification in the import queue (#2844)
* Remove ExtraVerification in the import queue

* Fix Babe

* Fix node-template
2019-06-12 14:56:31 +02:00
Pierre Krieger ac895023be More crate compiling for WASM-browser (#2825)
* More crate compiling for WASM-browser

* Update runtime Cargo.lock
2019-06-07 16:06:15 +02:00
Pierre Krieger 67bdfc7d8e Small slots refactor (#2780)
* Deprecate SlotWorker::on_start

* start_slot_worker no longer needs an Arc

* start_slot_worker now always succeeds

* Removed on_exit parameter from start_*_worker

* Minor doc

* Fix node-template
2019-06-05 10:31:20 -04:00
Benjamin Kampmann eaa0ab014a Move import lock onto backend (#2797)
* Drop import_lock from client, move it into backend, impl default version via static mutex

* still need to allow depcretion because of client.backend

* additional docs

* Remove default impl of get_import_lock, impl on instances

* Bump parking_lot to 0.8.0 accross the board
2019-06-05 15:46:01 +02:00
Bastian Köcher 6142f95611 Fix tons of warnings in newest nightly (#2784)
* Fix tons of warnings in newest nightly

* Fix sr-api-macro doc tests
2019-06-04 20:09:49 +02:00
Pierre Krieger 5df89a8a6f Client::info() no longer returns a Result (#2776) 2019-06-04 16:09:46 +02:00
André Silva 7e591a8f4d core: import equivocated aura and babe blocks (#2709)
* core: import equivocated aura and babe blocks

* core: cleanup check_equivocation handling

* fix: use map_err on Aura

* core: slots: remove unneeded Arc and minimize cloning

* core: fix slots equivocation tests

* core: slots: remove unused import

* core: remove unnecessary comments
2019-05-29 15:25:02 +02:00
DemiMarie-parity c7d1204ce5 Introduce inherent digests (#2466)
* Introduce inherent digests

* Implement inherent digests

* fix silly error

* Implementation of inherent digests in BABE

All tests pass. There are still limitations:

1. The runtime strips out inherent digests, so BABE must re-add them.
2. The test runtime checks that it can re-compute all digests.  It
   can’t, so I had to comment out that test.

* Fix compilation and seal import

Seals were not imported correctly: the pre-digest was imported twice,
instead of both it and the seal being imported.  Also, other parts of
the code did not compile due to incomplete refactoring.

* Remove bogus assertion

* Fix testsuite compilation

* Remove unused import

* Fix compiler diagnostics

* Add inherent digest parameters to block constructors

This enforces that inherent digests are added first.

* Fixup Cargo.lock

* Fix build errors

* Re-add an incorrectly removed import

* Bump primitive-types version

* Update Cargo.lock

* Refactoring

* Use inherent digests for AuRa

They do reach the runtime, but get stripped.  I have not figured out
where.

* Fix compilation errors

* Fix compilation errors due to incorrect types

* Fix whitespace

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Add preamble

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Fix silly compile error

* Refactor pre-digest finding code into a separate function

* Remove unwanted assertion

It is too likely to bring down the entire blockchain.

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Use `find_pre_digest` after runtime, too

Also, use `Member` trait rather than rolling our own requirements.

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Fix various warnings

mostly due to upgrading the dependency on `error_chain`.

* Pre-digests nearly complete

This nearly completes the implementation of pre-runtime digests.

* `Seal2` → `Seal` and fix test suite

* Try to fix the storage error

* Try to fix storage (again)

* Fix tests

* Hopefully finish pre-runtime digests

The key is to pass *only* the pre-runtime digests to the runtime.  The
others must be stripped out by `initialize_block`.

* Fix silly typo

* Fix another silly mistake

* Remove unnecessary filtering of BABE pre-digests

We no longer get duplicate BABE pre-digests, so if they appear, the
header should be rejected outright.

* Update Cargo.lock files

* Reformatting

* Fix silly typo in inherent digest code

Also, revert `error.rs` files that contained calls to the `error_chain!`
macro.

* Try to keep the runtime from stripping pre-digests

Currently runs into the “Storage root must match that calculated”
assertion.

* Don’t compute storage root until storage changes are done.

Also, fix a compilation error.

* Fix compile-time error

* Fix compilation errors

* Fix more compile errors

* Hopefully it compiles this time…

* Fix compilation and add docs

* Prevent BABE from adding duplicate pre-runtime digests

Found by comparing with the AuRa code.  I also did some refactoring.

* Respond to review and fix some warnings

* Delete some dead code introduced earlier

* More dead code goes away

* `ref mut` → `&mut`

* Respond to review and fix some warnings

* Fix compilation error

* Remove unneeded `HashT` type parameter

Suggested-by: Robert Habermeier <robert@parity.io>

* Remove spurious #[allow(deprecated)]

* Document inherent digest parameter to `build_block`

* Delete `Simple` trait

It wasn’t needed

* delete wrongly added files

* Fix trait bounds

* Digest serialization tests

I also did some reformatting and cleanup.

* Apply suggestions from code review

Reformatting

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Swap two arguments to `propose` and `propose_with`

Also, remove some needless unsafe code.

* Remove bogus `#![allow(deprecated)]` annotations

With the removal of the deprecated `Seal` variant, these are not needed.

* Add a missing `#[allow(deprecated)]` in the AuRa tests

* Fix silly compile error

* Fix silly compiler error

RLS did not tell me that I hadn’t fixed `babe/lib.rs`, so I missed it.

* Fixes made automatically by Cargo
2019-05-29 14:13:38 +02:00