Commit Graph

66 Commits

Author SHA1 Message Date
Rakan Alhneiti 2577dde3d9 Use sign_with for signing grandpa's outgoing message (#6178)
* Use sign_with and stop using `Pair`

* PR feedback

* Remove clone

* Transfer ownership of public to sign_message

* Use Option

* Simplify code

* Fix error message

* Pass keystore as ref

* Pass keystore properly

* Fix tests
2020-06-09 07:39:30 +00:00
Bastian Köcher 663cd09be9 Fix transaction pruning in tx-pool (#6276)
The `tree_route` generated by the import notification is only from the
old best block to the new best parent. This means, it does not contain
the new best block in `enacted()`. We need to prune the transactions of
the new best block "manually" to fix this bug.

Besides that, this pr also changed the `id` parameter of the `NewBlock`
chain event to `hash`. The hash of a block is unique in contrast to the
block number. (Block id can either be number or hash)
2020-06-08 12:38:19 +02:00
Bastian Köcher d2846e2b9a Make transaction pool prune transactions only of canonical blocks (#6123)
* Make tx pool aware of retracted fork blocks

* Make it compile

* Update client/transaction-pool/src/lib.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Fix doc test

* Simplify the implementation

* Send tree route as arc to prevent heavy clones

* Switch to use `ExtrinsicHash` to make it more clear

* Fix benchmark

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-06-05 23:12:00 +02:00
Nikolay Volf 8f7bec3519 remover ancient code (#6157) 2020-05-27 13:10:04 +02:00
André Silva 0ddd5cc278 babe: treat epoch_authorship RPC method as unsafe (#6069)
* service: pass DenyUnsafe to rpc extensions

* node: add DenyUnsafe to rpc full node dependencies

* client: fix whitespace in rpc policy file

* babe: treat epochAuthorship rpc method as unsafe

* babe: add test for unsafe rpc method

* babe: rename babe rpc handler

* service: traitify rpc extension builder

* service: make the rpc extensions builder api non-breaking

* service: revert changes from light node rpc extensions builder

* node: remove unnecessary type in service creation

* service: cleanup with_rpc_extensions implementation

* service: add missing docs to RpcExtensionBuilder
2020-05-21 13:58:04 +02:00
Nikolay Volf dd573374e0 Add block construction prometheus metrics (#6030)
* Add basic authorship metrics

* fixes

* no arc

* move to crate

* Update client/proposer-metrics/Cargo.toml

Co-authored-by: Max Inden <mail@max-inden.de>

* remove prefix

* use HistogramTimer

* Update client/proposer-metrics/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/proposer-metrics/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/proposer-metrics/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-05-18 13:48:44 +00:00
Nikolay Volf d5411969ac fix whitespace (#6062) 2020-05-18 13:51:28 +02:00
Benjamin Kampmann efc4849f1a Meter block import results via prometheus (#6025) 2020-05-15 17:25:51 +02:00
Benjamin Kampmann be8c96adec Relax substrate licensing scheme (#5947) 2020-05-15 13:21:56 +02:00
Demi Obenour ddea306044 Add ‘transaction_version’ to the signed transaction (#5979)
* Add ‘transaction_version’ to the signed transaction

This allows hardware wallets to know which transactions they can safely
sign.  To reduce transaction size, I reduced it to a ‘u8’ from a ‘u32’.

Fixes #5951.

* Restore transaction_version to a u32

* Fix comments

`transaction_version` is not part of a tx, but is still signed.

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix the test suite

I had forgotten to change the production of transactions in the test
code.

* Fix benchmarks

* Improve docs for `CheckTxVersion` in `frame_system`

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Remove spurious cast

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-05-15 00:01:56 +02:00
André Silva fbd2ac8f3b grandpa: report equivocations (#3868)
* session: runtime api for generating session membership proofs

* grandpa: add runtime api for creating equivocation report txs

* grandpa: submit signed equivocation report transactions

* grandpa: use proper equivocation report type

* grandpa: report equivocations

* grandpa: validate equivocation proof

* grandpa: update to finality-grandpa 0.9.1

* grandpa: fix encoding of session membership proof

* grandpa: initialize set id session mapping for genesis session

* grandpa: fix bug in set_id session validation

* fix compilation

* cleanup from merge conflicts

* cleanup crate tomls

* grandpa: refactor equivocation handling to separate trait

* node-template: fix compilation

* fix test compilation

* bump finality-grandpa to v0.10.2

* rpc: fix runtime version test

* CHERRY-PICK #4200: Add documentation to SubmitSignedTransaction and actually make it work

Squashed commit of the following:

commit 4f2cb0b1c588a06f2f3b478bb4b28b5cb29d54b9
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Tue Dec 3 16:29:33 2019 +0100

    Split the method to avoid confusing type error message.

commit c5bf24eeaaf902add89ed1b046b22c4a4aaeb2cd
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Tue Dec 3 16:19:55 2019 +0100

    Make accounts optional, fix logic.

commit 97db1ef556e023cf6847e5ffdb036c0e3ea6fb0a
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Tue Dec 3 10:06:20 2019 +0100

    Remove warning.

commit 535f5c116d1a2e826eaf90c3f7e6798e443d61d8
Merge: 516257217 0f1a5f651
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Tue Dec 3 07:08:05 2019 +0100

    Merge branch 'master' into td-signed-transactions

commit 516257217bac89fcebd083712f4ea68b7b23b55a
Merge: ac98248c6 2e68c80c2
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Mon Dec 2 13:57:25 2019 +0100

    Merge branch 'master' into td-signed-transactions

commit ac98248c6c56cff381130645a82a13d29933cf83
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Mon Nov 25 17:34:52 2019 +0100

    Forgotten import.

commit 67a3c19031506c28e31c6bc4a90fff62d467dd58
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Mon Nov 25 17:32:10 2019 +0100

    Fix naming and bounds.

commit 93e768ea9df97a4629fca1f9bc4b108fdb33f876
Author: Tomasz Drwięga <tomasz@parity.io>
Date:   Mon Nov 25 17:01:05 2019 +0100

    Add documentation to signed transactions and actually make them work.

* grandpa: skip block initialization on report submission method

* primitives: allow transaction pool access by default for offchain calls

* grandpa: unused parameters

* grandpa: remove unused method

* grandpa: enable equivocation reporting

* grandpa: add workaround for parameter encoding

* grandpa: fix localized_payload calls in tests

* fix submit_report_equivocation_extrinsic in runtimes

* node: fix submit transaction test compilation

* node: bump spec_version

* rpc: fix api version test

* grandpa: allow custom equivocation offence type

* grandpa: add test for authorities::next_change_height

* grandpa: cleanup report_equivocation function

* node: move reporting app crypto to node-primitives

* grandpa: move equivocation traits to own module

* grandpa: rename app-crypto crate import

* grandpa: export equivocation types

* node: bump spec_version

* grandpa: rename EquivocationReport to EquivocationProof

* grandpa: add missing docs to primitives

* grandpa: add missing docs to equivocation

* node: fix compilation

* grandpa: add missing docs to pallet

* node: bump spec_version

* fix whitespace

* grandpa: return error on offence reporting

* grandpa: expose session and validator count in proofs through traits

* grandpa: use strong key in module KeyOwnerProofSystem

* grandpa: move key ownership proof to grandpa runtime api

* grandpa: remove unnecessary cloning when checking equivocation proof

* grandpa: make report_equivocation a method in Environment

* support: implement KeyOwnerProofSystem for ()

* grandpa: move KeyOwnerProofSystem to module trait

* test-utils: fix runtime compilation

* grandpa: fix test compilation

* grandpa: fix test compilation after merge

* grandpa: simplify transaction submission types

* grandpa: validate equivocation report in signed extension

* client: fix test

* node: use ValidateEquivocationReport signed extension

* grandpa: expose key ownership proof under opaque type

* grandpa: better docs on key ownership proofs

* grandpa: add note about signed extension

* grandpa: add ValidateEquivocationReport::new

* grandpa: remove skip_initialize_block from runtime api

* grandpa: use new offchain transaction submission API

* grandpa: take set_id in generate_key_ownership_proof

* grandpa: update to finality-grandpa v0.12.2

* grandpa: cleanup usages of AuthoritySet::current

* grandpa: fix test

* grandpa: add mocking utilities for equivocation reporting

* grandpa: add test for equivocation reporting

* grandpa: move SetIdSession initialization

* grandpa: add more tests

* node: enable historical session manager

* node: bump spec_version

* node: use strong key types in KeyOwnerProofSystem definitions

* grandpa: export GrandpaEquivocationOffence type
2020-05-06 17:25:51 +01:00
Jon Häggblad 1f7f8abb33 Expose GRANDPA round state through RPC (#5375)
* grandpa: wire up basic RPC call

* grandpa: make it compile against GRANDPA with expose round state

* grandpa: use shared voter state to expose RPC endpoint

* grandpa: restructure into nested structs

* grandpa: return background rounds too

* grandpa: return error when endpoint not ready

* grandpa: collect grandpa rpc deps

* grandpa: decide to use concrete AuthorityId in finality-grandpa-rpc

* grandpa: remove unncessary type annotation

* grandpa: move error code to const

* grandpa: remove unnecessary WIP comment

* grandpa: remove Id type parameter for SharedVoterState

* grandpa: update tests to add shared_voter_state in parameters

* grandpa: remove old deprecated test

* grandpa: fix getting the correct set_id

* grandpa: make SharedVoterState a struct

* grandpa: wrap shared_voter_state in rpc_setup

* grandpa: replace spaces with tabs

* grandpa: limit RwLock write attempt to 1 sec

* grandpa: add missing doc comments and remove some pub

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-Authored-By: Hernando Castano <HCastano@users.noreply.github.com>

* grandpa: update function name call after change in finality-grandpa

* grandpa: group pub use and only export voter::report

* grandpa: add missing docs

* grandpa: extract out structs used for json serialization

* grandpa: stick to u32 for fields intended for js

* grandpa: move Error type to its own file

* grandpa: group pub use better

* Apply code review suggestion

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* grandpa: use correct version of finality-granpda in rpc crate

* grandpa: add back basic rpc unit test

* grandpa: replace SharedVoterState::new() with empty()

* node: cleanup grandpa::SharedVoterState usage in macro

* grandpa: remove VoterState error variant

* grandpa: enable missing futures compat feature

* grandpa: fix typo in error variant

* grandpa: remove test_utils

* grandpa: allow mocking rpc handler components

* grandpa: rename serialized to report in rpc module

* grandpa: add proper test for RPC

* grandpa: update to finality-grandpa v0.12.1

Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-05-04 20:37:22 +01:00
Bastian Köcher 9c5536e01a Expose that BasicQueue expects blocking spawn (#5860)
* Expose that `BasicQueue` expects blocking spawn

Up to now `BasicQueue` expected a closure that to spawn a `Future`.
This was expected to be a closure that spawns a blocking future.
However, this wasn't documented anywhere. This pr introduces a new trait
`SpawnBlocking` that exposes this requirement to the outside.

* Feedback
2020-05-04 19:40:29 +02:00
pscott 0f401e4699 Move spawning tasks from thread pools to Service's TaskManager for block importing (#5647)
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-04-29 18:46:39 +02:00
Seun Lanlege 4fa5941f44 Move sc-client into sc-service (#5502)
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api

* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa

* drop sc-service from sc-rpc

* drop sc-service from sc-consensus-aura

* drop sc-client from manual-seal and babe

* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli

* drop sc-client from bin/node and bin/node-template

* drop sc-client

* fix tests

* remove check -p sc-client from gitlab.yml

* fix warnings

* fixes ui test

* fix light client tests

* adds associated Client type to AbstractService

* adds UsageProvider to Client

* fixed ui test, again

* tried and failed to get node-cli to compile for wasm

* thanks to tomaka for helping me get node-cli to compile for wasmm

* ui test pls pas 🙏🏾

* all tests passing 🪄

* no_run documentation code

* rm -f documentation code

* ClientProvider

* fix mega trait

* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client

* adds license to sc-consensus

Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-04-28 11:59:31 +00:00
Alexander Theißen 000c924b62 Integrate pallet_contracts gas with the weight system (#5712)
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
2020-04-24 10:48:13 +02:00
Nikolay Volf fea626ca84 Add prometheus registry to transaction pool, with couple of initial metrics (#5657)
* make new contructor

* add metrics to txpool

* fix review

* fix doc comment

* change to counters

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* use dedicated wrapper

Co-authored-by: Max Inden <mail@max-inden.de>
2020-04-17 11:02:45 +02:00
Max Inden d742e88e79 client/authority-discovery: Allow to be run by sentry node (#5568)
* client/authority-discovery: Allow to be run by sentry node

When run as a sentry node, the authority discovery module does not
publish any addresses to the dht, but still discovers validators and
sentry nodes of validators.

* client/authority-discovery/src/lib: Wrap lines at 100 characters

* client/authority-discovery: Remove TODO and unused import

* client/authority-discovery: Pass role to new unit tests

* client/authority-discovery: Apply suggestions

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* bin/node/cli/src/service: Use expressions instead of statements

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-04-16 18:11:26 +01:00
Pierre Krieger 1e1b0e2767 Give names to channels (#5626)
* Give names to channels

* Fix

* A couple more changes

* More minor tweaks

* Fix test
2020-04-14 14:49:41 +02:00
Kian Paimani e7e7e89e71 Collection of enhancement for staking/phragmen. (#5534)
* Collection of enhancement for staking phragmen.

* Ditch signed extension

* Fix build

* Update frame/staking/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Update frame/staking/src/tests.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Fix reward limits

* Disallow payout_stakers

* Remove unused import

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2020-04-09 11:29:11 +02:00
Cecile Tonglet 3da069e359 CLI: refactoring: remove Options from sc_service::Configuration's fields (#5271)
* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Rename IntoConfiguration to CliConfiguration

* Renamed into_configuration to create_configuration

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Move keystore params to its own module

* Use in-memory keystore even for build-spec

* Enforce proper value for node name

* dev_key_seed

* Telemetry endpoints

* rustfmt

* Converted all RunCmd

* rustfmt

* Added export-blocks

* Missed something

* Removed config_path in NetworkConfiguration (not used)

* Fixed warnings

* public_addresses is used but never set, keeping it

* Merge Configuration.node and NetworkConfiguration.node_name

...because they are the same thing

* Added: import-blocks

* Adding a proc_macro to help impl SubstrateCli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Re-export spec_factory from sc_cli

* Re-added all the commands

* Refactored node_key_params

* Fixed previous refucktoring

* Clean-up and removed full_version()

* Renamed get_is_dev to not confuse with Configuration field

* Fixed sc-cli-derive example

* Fixing tests

* Fixing tests and removing some (will re-add later)

* Fixing more tests

* Removes the need of type parameter

* Converting bin/node and simplifying API

* Converting more

* Converting last command

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fixing tests and added default for WasmExecutionMethod

* Fixing stuff

* Fixed something I broke oops

* Update Cargo.lock

* Moving things around

* Convert everything to Result

* Added new macros to simplify the impl of CliConfiguration

* Added a macro to generate CliConfiguration automatically for subcommands

* Revert... too many macros (this one is not really useful)

This reverts commit 9c516dd38b40fbc420b02c1f8e61d5b2b1a4e434.

* Renamed is_dev to get_is_dev

Good enough for now

* Fixed name roles (this is plural, not singular)

* Clean-up

* Re-export NodeKeyConfig and TelemetryEndpoints from sc_service

* Improve styling/formatting

* Added copyrights

* Added doc and fixed warnings

* Added myself to code owners

* Yes it is needed according to the history

* Revert formatting

* Fixing conflict

* Updated build.rs

* Cargo.lock

* Clean-up

* Update client/cli-derive/Cargo.toml

Co-Authored-By: Seun Lanlege <seunlanlege@gmail.com>

* Fail if using proc_macro and build.rs is not set properly

* Dropped all get_ in front of methods

* Clean-up

* Fixing proc macro missing env var

* Get the configuration inside the Runtime (needed for polkadot)

* Clean-up

* Get is_dev from argument like the others

* Get chain ID instead of chain spec from shared params

* &self is passed to spec_factory/load_spec

* Wrong text

* Fix example

* Officialize macro and made a cool doc

* Renamed spec_factory to load_spec (substrate_cli_configuration)

* Removed not so useful ChainSpec

* Renamed SubstrateCLI to SubstrateCli

* Added changelog for impl_version being full now

* Renamed Runtime to Runner

* Update changelog to show example

* Removed option on database cache size

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fix on removal of option

* typo

* Clean-up imports

* Added info in Cargo.toml

* typo

* remarks

* Moved function for build.rs to substrate-build-script-utils

* Fixed example & test of cli-derive

* Moved function for build.rs to substrate-build-script-utils

* Renamed substrate_cli_configuration to substrate_cli oops

It implements SubstrateCli not CliConfiguration!

* Added documentation and wrapper macro

* Removed option on database cache size

* Removed option on database cache size

* Clean-up

* Reduce risk of errors due to typos

* Removed option on database cache size

* Added NOTE as suggested

* Added doc as suggested

* Fixed test

* typo

* renamed runtime to runner

* Fixed weird argument

* More commas

* Moved client/cli-derive to client/cli/derive

* Added 7 tests for the macros

* Improve error message

* Upgrade assert_cmd

* Fixing missing stuff

* Fixed unused import

* Improve SubstrateCli doc

* Applied suggestions

* Fix and clean-up imports

* Started replacing macros WIP

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Started removing substrate_cli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* fixed bug introduced while refactoring

* Renamed NetworkConfigurationParams to NetworkParams for consistency sake

* Fixed test

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/check_block_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 5906776953392c02beac6bc0bf50f8cbe1a12a01.

* Revert "Update client/cli/src/commands/check_block_cmd.rs"

This reverts commit f705f42b7f3d732be001141afee210fe46a1ef47.

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 8d57c0550164449e6eb2d3bacb04c750c714fcea.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 93e74cf5d2e1c0dc49cdff8608d59fc40fc59338.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 11d527ba345c0d79f0d3b5b071933d95474d0614.

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/import_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Changed ::sc_cli to $crate in the macro

* fixed tests

* fixed conflicts

* Fixing test

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove comment as suggested

* Apply suggestion

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/node_key_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/network_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/config.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added doc

* Fixed error introduced after applying suggestion

* Revert "Update client/cli/src/params/pruning_params.rs"

This reverts commit 0574d06a4f1efd86e94c1214420a12e7a4be0099.

* Print error

* Apply suggestions from code review

* Remove useless Results

* Fixed CI failing on polkadot approval

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-07 11:38:07 +02:00
Pierre Krieger 8c03a4fcef Split the Roles in three types (#5520)
* Split the Roles bitfield in three

* Forgot to include some changes

* Fix cli test

* More test fixes

* Oh God, merging master broke other tests

* Didn't run the doctests

* Address review

* I'm trying to fix the build blindly because it's taking a good hour to compile on my machine

* Address some review

* Also update the peerset's API to make sense

* Fix peerset tests

* Fix browser node

* client: distinguish between local and network authority

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-04-03 19:08:14 +02:00
Kian Paimani 970c5f94f2 Offchain Phragmén BREAKING. (#4517)
* Initial skeleton for offchain phragmen

* Basic compact encoding decoding for results

* add compact files

* Bring back Self::ensure_storage_upgraded();

* Make staking use compact stuff.

* First seemingly working version of reduce, full of todos

* Everything phragmen related works again.

* Signing made easier, still issues.

* Signing from offchain compile fine 😎

* make compact work with staked asssignment

* Evaluation basics are in place.

* Move reduce into crate. Document stuff

* move reduce into no_std

* Add files

* Remove other std deps. Runtime compiles

* Seemingly it is al stable; cycle implemented but not integrated.

* Add fuzzing code.

* Cleanup reduce a bit more.

* a metric ton of tests for staking; wip 🔨

* Implement a lot more of the tests.

* wip getting the unsigned stuff to work

* A bit gleanup for unsigned debug

* Clean and finalize compact code.

* Document reduce.

* Still problems with signing

* We officaly duct taped the transaction submission stuff. 🤓

* Deadlock with keys again

* Runtime builds

* Unsigned test works 🙌

* Some cleanups

* Make all the tests compile and stuff

* Minor cleanup

* fix more merge stuff

* Most tests work again.

* a very nasty bug in reduce

* Fix all integrations

* Fix more todos

* Revamp everything and everything

* Remove bogus test

* Some review grumbles.

* Some fixes

* Fix doc test

* loop for submission

* Fix cli, keyring etc.

* some cleanup

* Fix staking tests again

* fix per-things; bring patches from benchmarking

* better score prediction

* Add fuzzer, more patches.

* Some fixes

* More docs

* Remove unused generics

* Remove max-nominator footgun

* Better fuzzer

* Disable it 

* Bump.

* Another round of self-review

* Refactor a lot

* More major fixes in perThing

* Add new fuzz file

* Update lock

* fix fuzzing code.

* Fix nominator retain test

* Add slashing check

* Update frame/staking/src/tests.rs

Co-Authored-By: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>

* Some formatting nits

* Review comments.

* Fix cargo file

* Almost all tests work again

* Update frame/staking/src/tests.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Fix review comments

* More review stuff

* Some nits

* Fix new staking / session / babe relation

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Update primitives/phragmen/compact/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Some doc updates to slashing

* Fix derive

* Remove imports

* Remove unimplemented tests

* nits

* Remove dbg

* Better fuzzing params

* Remove unused pref map

* Deferred Slashing/Offence for offchain Phragmen  (#5151)

* Some boilerplate

* Add test

* One more test

* Review comments

* Fix build

* review comments

* fix more

* fix build

* Some cleanups and self-reviews

* More minor self reviews

* Final nits

* Some merge fixes.

* opt comment

* Fix build

* Fix build again.

* Update frame/staking/fuzz/fuzz_targets/submit_solution.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update frame/staking/src/slashing.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update frame/staking/src/offchain_election.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Fix review comments

* fix test

* === 🔑 Revamp without staking key.

* final round of changes.

* Fix cargo-deny

* Update frame/staking/src/lib.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
2020-03-26 15:37:40 +01:00
Nikolay Volf db86094b03 Produce block always on updated transaction pool state (#5227)
* make sure return ready iterator once state is updated

* update sc_basic_authorship tests

* update node tests

* fix manual seal

* actually fix service test

* add tests

* Update client/basic-authorship/src/basic_authorship.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* helper function

* review suggestions

* warning and continue

* add debug log

* use futures::chennel::oneshot

* use declaration bound

* no option for updated_at

* no allocation

* ready_at / ready

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-17 16:24:04 +01:00
Wei Tang 846a9ce8c6 epoch-changes: separate epoch header with epoch data (#4881)
* fork-tree: prune returns all pruned node data

* epoch-changes: split EpochHeader vs epoch data

* EpochChanges::viable_epoch and add missing comments

* Incoperate the new epoch_changes interface for BABE

* Fix BABE tests

* Fix fork-tree pruning issue

* Fix tests

* Fix pruning algorithm

* fork-tree: implement map function for mapping one value type to another

* Add migration script for new epoch changes scheme

* Update utils/fork-tree/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update client/consensus/slots/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Remove authorities_len.is_none check, which is duplicate of unwrap_or(false)

* Update client/consensus/epochs/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update client/consensus/epochs/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* No trailing ; for return statement

* Use VERSION_KEY for migration

* Fix issues that removed nodes are not properly added into removed list

* Add comments indicating end_slot is non-inclusive

* fork-tree: use &mut F for map type declaration

* Add tests for v0 epoch_changes migration

* Fix babe RPC tests

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-03-16 17:06:13 -04:00
Arkadiy Paronyan dc0bf210fb ChainSpec trait (#5185)
* ChainSpec trait

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added docs

* Fixed build

* Fixed build

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-12 00:00:10 +01:00
Max Inden d2345e8d5c client/authority-discovery: Instrument code with Prometheus (#5195)
* client/authority-discovery: Instrument code with Prometheus

Introduce Prometheus metrics into the authority discovery module
enabling one to observe:

- authority_discovery_times_published_total

- authority_discovery_amount_external_addresses_last_published

- authority_discovery_times_requested_total

- authority_discovery_dht_event_received

* client/authority-discovery/src/lib.rs: Reword metric help texts

Co-Authored-By: Ashley <ashley.ruglys@gmail.com>

* client/authority-discovery/src/lib.rs: Reword metric help text

Co-Authored-By: Ashley <ashley.ruglys@gmail.com>

* client/authority-discovery/tests: Fix struct instantiation + basic test

Co-authored-by: Ashley <ashley.ruglys@gmail.com>
2020-03-11 23:59:33 +01:00
Benjamin Kampmann 99ae5342eb removes use of sc_client::Client from sc-rpc (#5063)
* removes use of sc_client::Client from sc-rpc

* remove Client impl from sc-finality-benches

* remove client impl from sc-finality-grandpa

* read_proof accepts iterator

* remove generic Executor param from ExecutorProvider

* fix long ass line

* code style changes

* merge with master

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
2020-03-05 16:41:10 +01:00
Gavin Wood ab58a08a53 remove on_exit from grandpa (#5098) 2020-03-05 13:38:16 +01:00
Ashley 883ddfc897 Add more metrics to prometheus (#5034)
* Add a few things

* Add finality_grandpa_round

* fix fg tests

* Nitpicks

* Nitpicks

* Fix name of prometheus crate
2020-03-03 11:36:58 +01:00
Seun Lanlege f26f703ad4 removes use of sc_client::Client from sc_basic_authorship (#5050)
* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path
2020-02-27 17:12:00 +01:00
Seun Lanlege 5907b0d902 removes use of sc_client::Client from sc_finality_grandpa (#5030)
* removes use of sc_client::Client from sc_finality_grandpa

* code formatting

* code formatting

* removes use of sc_client::Client from sc_finality_grandpa
2020-02-27 12:19:07 +01:00
Robert Habermeier 0090fe979b Remove NetworkSpecialization (#4665)
* remove networkspecialization

* Fix most of the fallout

* get all tests compiling

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-02-21 14:02:12 +01:00
Seun Lanlege 832b2ef642 Removes use of sc_client::Client from sc_consensus_babe (#5014)
* removes use of sc_client::Client from sc_consensus_babe

* Update client/consensus/babe/src/lib.rs

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
2020-02-21 12:13:16 +01:00
André Silva f0043055cc node, node-template: disable GRANDPA observer (#5011)
* node-template: disable grandpa observer

* node: disable grandpa observer

* node: add doc about grandpa-voter infallible task

* grandpa: remove grandpa observer from public API

* grandpa: ignore observer_enabled field in config
2020-02-21 10:50:42 +01:00
Stanislav Tkach 504914b0a6 Remove deprecated API (#4993)
* Remove deprecated API

* Remove (some) allow(dead_code)

* Bump spec_version

* Fix import, remove allow dead code.

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-02-20 17:03:33 +01:00
Seun Lanlege 5567d18b6a Adds babe rpc support (#4729)
* babe_epochAuthorship
remove test-helpers from sp-keyring, bump spec_version, impl_version

* bump Cargo.lock

* add BabeRPC to node-rpc

* rename to BabeApi, remove err_derive

* pass &ServiceBuilder to with_rpc_extensions callback

* sc-consensus-babe-rpc

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Better docs, code style chanegs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* new line at the end of Cargo.toml

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-02-17 18:05:30 +00:00
Nikolay Volf 86ab0cb4d9 Revalidation queue for transaction pool (#4781)
* Revalidation queeue.

* add docs and license

* move test

* refactor worker to async/await

* address review

* fix warnings

* update Cargo.lock

* move background task to service

* use tomusdrw loop

* naming

* return From::from

* add doc comment

* add more doc comments

* fix merge bug

* add doc comment for test function

* Update client/transaction-pool/src/testing/pool.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* more review fixes

* refactor to allow service keep background tasks from isntantiated subsystems

* use const delay

* fix fallout

* remove fallout

* remove already moved test

* fix doc test

* add valid_at helper

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-02-17 16:48:24 +03:00
Wei Tang c7b09b642a Refactor BlockImportParams to be non_exhaustive (#4936)
* Refactor BlockImportParams to be non_exhaustive

* Fix cargo check compile
2020-02-17 10:18:53 +01:00
Max Inden 3f3910ccaf client/network-gossip: Integrate GossipEngine tasks into Future impl (#4767)
`GossipEngine` spawns two tasks, one for a periodic tick, one to forward
messages from the network to subscribers. These tasks hold an `Arc` to a
`GossipEngineInner`.

To reduce the amount of shared ownership (locking) this patch integrates
the two tasks into a `Future` implementation on the `GossipEngine`
struct. This `Future` implementation can now be called from a single
owner, e.g. the `finality-grandpa` `NetworkBridge`.

As a side effect this removes the requirement on the `network-gossip`
crate to spawn tasks and thereby removes the requirement on the
`finality-grandpa` crate to spawn any tasks.

This is part of a greater effort to reduce the number of owners of
components within `finality-grandpa`, `network` and `network-gossip` as
well as to reduce the amount of unbounded channels. For details see
d9837d7dd, 5f80929dc and 597c0a6c4.
2020-02-12 13:15:26 +01:00
Wei Tang 14b0c9a746 babe: pass epoch data via intermediates (#4807)
* babe: pass epoch data via intermediates

* Switch to use Box<dyn Any> for intermediates

* Set intermediate.epoch to be Option

* Fix proposer should put out an empty intermediate

* Remove unnecessary encode/decode

* Add EpochData to block_import_params in slot worker

* Fix aura compile

* Fix integration test
2020-02-06 17:52:30 +00:00
Bastian Köcher 709a899f9d Cleanup transaction pool deps (#4782)
* Cleanup transaction pool deps

* Fix it properly

* Fix doc test
2020-01-31 12:40:32 +01:00
Cecile Tonglet 605f643eed CLI API refactoring and improvement (#4692)
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).

This touches a few important things:
 - the startup of the async task with tokei:
    This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
 - increased the version of structopt and tokei
 - no more use of structopt internal's API
 - less use of generics

Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want

Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
2020-01-30 11:40:08 +01:00
Pierre Krieger b452867eb7 Add diagnostics to tasks (#4752) 2020-01-29 11:46:39 +01:00
Ashley c2c429877e Switch GrandPa to std futures (replaces #3909) (#4612)
* Switch GrandPa to new futures

* Work on making tests work

* until_imported tests working again

* Work on switching tests to stable futures

* Modifications

* Re-add test as #[ignore]

* Don't ignore

* Add manual unpins

* Remove Header import

* Return concrete Sink type

* Switch to crates.io finality-grandpa version

* Remove use statement that slipped in

* Fix some nitpicks

* Remove unpin from i

* Fixed typo

* Move futures01 to dev-deps

* Fix nitpicks

* Update client/finality-grandpa/src/communication/mod.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* nitpicking

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-01-24 13:34:42 +01:00
Nikolay Volf 14e95f3398 Refactor tx-pool maintenance and other high-level api (#4629)
* Reduction.

* Reformation.

* add locked timer stuff

* fix issues and introduce full pool

* arrange together

* fix benches

* fix new_light

* Add revalidation test case

* review fixes

* review fixes

* use just ready future

* address review
2020-01-24 04:21:24 -08:00
Wei Tang 4b2e6a5be2 Default fork choice value and intermediates for block import parameters (#4652)
* consensus, pow: intermediate separation and fail

* Fix compiles

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Document what None means for `fork_choice` in block import params

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2020-01-23 00:03:38 -08:00
Tomasz Drwięga 437772be9e Rename basic-authority to basic-authorship (#4640)
* Rename basic-authority to basic-authorship

* Revert weird tab.

* Fix rename.
2020-01-16 13:57:41 +01:00
Robert Habermeier c7069de044 Make Proposer instantiation potentially async. (#4630)
* Make Proposer instantiation potentially async.

* fix node-service test

* fix basic-authority doc-test

* only block once on futures in test

* use async/await
2020-01-15 21:09:27 +01:00
Ashley 3219be2508 Update the service to std futures (#4447)
* Switch service to futures03

* Fix tests

* Fix service test and cli

* Re-add Executor trait to SpawnTaskHandle

* Fix node-service

* Update babe

* Fix browser node

* Update aura

* Revert back to tokio-executor to fix runtime panic

* Add todo item

* Fix service tests again

* Timeout test futures

* Fix tests

* nits

* Fix service test

* Remove zstd patch

* Re-add futures01 to aura and babe tests as a dev-dep

* Change failing test to tee

* Fix node

* Upgrade tokio

* fix society

* Start switching grandpa to stable futures

* Revert "Start switching grandpa to stable futures"

This reverts commit 9c1976346237637effc07c13f7d0403daf5e71cf.

* Fix utils

* Revert substrate service test

* Revert gitlab

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2020-01-14 15:43:45 +01:00