Commit Graph

63 Commits

Author SHA1 Message Date
Bastian Köcher 7b56ab15b4 Run cargo fmt on the whole code base (#9394)
* Run cargo fmt on the whole code base

* Second run

* Add CI check

* Fix compilation

* More unnecessary braces

* Handle weights

* Use --all

* Use correct attributes...

* Fix UI tests

* AHHHHHHHHH

* 🤦

* Docs

* Fix compilation

* 🤷

* Please stop

* 🤦 x 2

* More

* make rustfmt.toml consistent with polkadot

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-07-21 14:32:32 +00:00
Kian Paimani 4d6802c530 Make it possible to override maximum payload of RPC (#9019)
* Make it possible to override maximum payload of RPC

* Finish it.

* remove todo.

* Update client/cli/src/commands/run_cmd.rs

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Incorporate suggestions

* Thread rpc_max_payload from configuration to trace_block

* Try obey line gitlab/check_line_width.sh

* update state rpc tests

* Improve readbility

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
2021-06-17 14:37:43 +00:00
tgmichel a4bfd40a1b rpc-http-threads cli arg (#8890)
* Add optional `rpc-http-threads` cli arg

* Update `http::ServerBuilder`threads
2021-06-08 11:18:57 +00:00
Bastian Köcher 37ea3958d9 Let the clone wars begin! (#8731)
Sprinkle some `Clone` onto the cli commands.
2021-05-04 21:57:18 +00:00
Dan Shields 3cd9cfd57a Change to use the same subcommand syntax as subkey (#8678)
* Change to use the same subcommand syntax as subkey

* Update client/cli/src/commands/inspect_key.rs

* revert to InspectKeyCmd struct
2021-05-04 11:50:31 +02:00
Tomasz Drwięga 46095af452 subkey: display SS58 encoding of public key (#8674)
* Add SS58 public key encoding.

* [Companion] Update Cargo.toml subkey version, readme to reflect new output (#8694)

* Update Cargo.toml

* update cargo, readme for subkey

Co-authored-by: Dan Shields <danwshields@gmail.com>

Co-authored-by: Dan Shields <35669742+NukeManDan@users.noreply.github.com>
Co-authored-by: Dan Shields <danwshields@gmail.com>
2021-04-29 15:23:03 +00:00
Pierre Krieger f6273f7914 Remove all code related to sentry nodes (#8079)
* Remove all code related to sentry nodes

* More fixing
2021-02-15 08:45:03 +00:00
Cecile Tonglet 970cc25cef Telemetry per node (#7463) 2021-01-20 12:28:56 +01:00
Bastian Köcher e3e651f72c Happy new year (#7814)
* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL
2021-01-04 09:03:13 +00:00
Bastian Köcher 05ae24d90a Fix base-path handling in key insert (#7775)
This fixes the handling of base-path when using `key insert`. Before
the base-path wasn't setup correctly, as done when starting a node. This
resulted in putting the keys into the wrong directory. This pr fixes
this by creating the correct base-path/config dir for the keystore.
Besides that it also removes the insert command from `subkey` as it
doesn't make that much sense. If requested, we could bring it back later.
2020-12-28 14:08:30 +00:00
Guillaume Thiolliere 8814d793c7 Fix docs (#7710)
* fix docs

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
2020-12-10 14:38:56 +01:00
Benjamin Kampmann 5ce8c33368 Expand remote keystore interface to allow for hybrid mode (#7628)
* update to latest master

* updates on docs, license, meta

* hide ssrs behind feature flag

* implement remaining functions on the server

* sign server line length fix

* fix tests

* fixup in-memory-keystore

* adding failsafe

* skipping ecdsa test for now

* remote keystore param

* remote sign urls made available

* integrating keystore remotes features

* don't forget the dependency

* remove old cruft

* reset local keystore

* applying suggestions

* Switch to single remote, minor grumbles

* minor grumbles, docs
2020-12-09 10:52:56 +01:00
Bernhard Schuster f373ecbcf0 improve error handling in cli (#7436)
* other error variant should carry a dyn Error

* introduce thiserror for error derive, add explicit error variants, cleanup dependencies

* cleanup handle dev-deps of sc-cli too

* Update client/cli/src/error.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-10-27 18:50:04 +01:00
Bastian Köcher 5808f467c5 Make Substrate compile with latest nightly (#7381)
* Try to get it compiling

* One more

* Make stable happy

* Make stable even more happy ;)

* Update shell.nix
2020-10-23 21:08:04 +02:00
Bastian Köcher 158fdecc0b Adds support for inspecting a hex encoded public key (#7344)
* Adds support for inspecting a hex encoded public key

This adds support for inspecting hex encoded public keys to subkey. The
command looks like:

`subkey inspect --public 0xPUBLICHEX`

* Update client/cli/src/commands/utils.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-10-17 09:43:46 +02:00
Ashley d1ef8507d7 Add missing fields to the light sync state (#7225)
* Initial draft

* Add an iterator that helps us get most items

* Revert changes to grandpa

* Change fields to just be the grandpa authority set and babe epoch changes

* Only use the fields we need from the shared authority set

* Switch to RPC call

* Revert "Only use the fields we need from the shared authority set"

This reverts commit 6ede87b0c5fe53f251d7cb45951006a7dc8f9b83.

* Add babe_finalized_block_weight from `ashley-improve-sync-state-WIP-loading`

* Fix rpc test

* Move sync state rpc stuff into sc-sync-state-rpc

* Remove as_json_value and remove unwraps from sc-sync-state-rpc

* Add clone_inner to SharedAuthoritySet
2020-10-15 10:10:43 +00:00
Rakan Alhneiti 3aa4bfacfc Async keystore + Authority-Discovery async/await (#7000)
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 20:56:35 +00:00
Bastian Köcher a00f0bf81c Fix vanity command by taking the network into account (#7192)
* Fix vanity command by taking the network into account

* Delete empty line

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Change test

* Stupid me

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-09-23 16:30:12 +00:00
Ashley ae2de4c2b2 Fix build sync spec (#7086) 2020-09-11 13:54:03 +00:00
Ashley 614c6a743f Add a build-sync-spec subcommand and remove the CHT roots from the light sync state. (#6999)
* Move subcommands from sc-cli to nodes

* Add --build-sync-spec subcommand

* Remove CHTs from snapshots

* Keep ProvideChtRoots
2020-09-11 14:50:12 +02:00
Bastian Köcher ae0f7e58b0 Make --file optional for generate-node-key (#7043)
This pr makes the `--file` argument optional to `generate-node-key`.
If the argument is not given, the secret node key will be printed to
`stdout`. The public node key will always be printed to `stderr`.
2020-09-08 10:32:37 +02:00
Ashley 57dd0facbb Move subcommands from sc-cli to nodes (#6948) 2020-09-01 09:06:22 +00:00
Bastian Köcher e7f25ebda8 Fix printing of subkey when using the --network override (#6932)
* Fix printing of subkey when using the `--network` override

This fixes a bug where `--network` did not printed the account ss58 address for the
requested network. Basically we now always print all account ss58 addresses using the
requested network.

* Review comments

* Fixes test

* Update client/cli/src/commands/inspect.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/cli/src/commands/utils.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix more tests

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-08-22 12:53:39 +00:00
Seun Lanlege 6963272451 Merge Subkey into sc-cli (#4954)
* draft

* revert

* WIP

* all that remains is tests

* update Cargo.lock

* tests WIP

* WIP refactor node-template-runtime and node-runtime

* implments sc_cli::RuntimeAdapter for node_template_runtime::Runtime

* final draft

* fix update_config for subcommands

* proper AccountId decoding

* test-runtime tests

* revert

* move RuntimeAdapter to cli-utils

* use &'static str for TryFrom::<&'a str>::Error for Ss58AddressFormat

* tests

* add frame-system to sc-cli dev-dependencies

* add frame-system to sc-cli dev-dependencies

* fix ui test

* wip

* fixed inspect test

* bump impl version

* bump impl version, fixx spaces remove todos

* pallet-balances-cli, rustc for some reason cannot resolve pallet_balances_cli in node-cli 😩

* wip

* Subcommand::run takes &self

* can't believe i missed that 🤦🏾‍♂️

* bump wasm-bindgen for some reason

* adds key subcommand, rename generate-node-key to generate-node-id

* cargo update and crossed fingers 🤞🏽

* update ui test

* update more ui tests

* should be all good now

* revert subkey change

* revert subkey change

* adds frame-utilities-cli

* Apply suggestions from code review

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* removes frame from sc-cli, fix license

* my editor and ci disagrees on line width

* bump spec version

* turn off default features for parity-scale-codec

* enable full_crypto feature for sp-core in cli-utils

* merge frame-utilities-cli with pallet-balances-cli

* remove full_crypto feature from sp_core in cli-utils

* bump Cargo.lock

* cli-utils -> frame-utils

* rename BlockNumber to GenericNumber, fix spaces

* fix spaces

* construct additional_signed manually

* sign test

* remove unused vars

* implement subkey with frame-utilities-cli and sc_cli

* fix moduleid test

* CI and clion disagree on line widths

* adds associated Params type to SignedExtensionProvider

* Apply suggestions from code review

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* move some code around

* removes unneccesary generic params

* moves module_id back to frame_utilities_cli

* Apply suggestions from code review

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* remove print_ext

* remove MaybeDisplay from pallet_balances::Trait::Balance

* a lot of stuff tbh

* adds ExtrasParamsBuilder

* remove tests for ModuleIdCmd

* address comments from PR

* bump Cargo.lock

* manually insert key into keystore

* remove unnecessary SharedParams

* add validation to vanity pattern, remove unused arg

* remove SharedParams from Sign, Vanity, Verify

* remove SharedParams from ModuleIdCmd, remove expect from Verify, new line to Cargo.toml

* remove SharedParams from InsertCmd

* 🤦🏾‍♂️

* deleted prometheus.yml

* move a few things around

* fix vanity test

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-08-20 09:55:03 +00:00
Bastian Köcher 7bf838f225 Combine default values used at initialization in a trait (#6857)
This moves default values used in the Substrate code base when
initializing a service into a common trait. Currently this trait only
contains listen ports, but this could be extended in the future.
Essentially this will make overriding these values much easier for
Cumulus, where we have 2 nodes running in one binary.
2020-08-19 17:30:56 +02:00
Ashley 5f751e4472 Remove ServiceBuilderCommand and implement the chain ops as standalone functions instead. (#6543)
* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* Make task_manager in run_until_exit and make subcommands async

* Change the async_run fn to return a future and task manager

* async_run should take a result fn

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix spaces in export_raw_state

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-07-02 12:57:56 +02:00
Pierre Krieger 505540a2a0 Increase the limit for the maximum size of the telemetry name (#6523)
* Increase the limit for the maximum size of the telemetry name

* Fix test
2020-06-26 14:48:11 +00:00
Pierre Krieger 39c3d1c026 Fix an extra semi-colon yielding a wrong error (#6520)
* Fix an extra semi-colon yielding a wrong error

* Update client/cli/src/commands/run_cmd.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-06-26 08:05:24 +00:00
Demi Obenour e2f5e4bd74 Add IPC support (#6348)
This is useful for both security and performance reasons. IPC is faster
than TCP, and it is subject to OS access controls.
2020-06-16 13:14:12 +03:00
André Silva 2b59d57c4c client: use appropriate ExecutionContext for initial sync / regular import (#6180)
* client: use appropriate ExecutionContext for sync/import

* client: remove dead code

* client: ExecutionContext: distinguish between own and foreign imports

* client: fix cli parameter doc

* Revert "client: ExecutionContext: distinguish between own and foreign imports"

This reverts commit 0fac11520704c364a82432c5b927e987ba043cdb.

* primitives: add docs for ExecutionContext

* cli: execution strategy docs

* cli: use different execution context for importing block on validator

* cli: remove defaults from execution context flags
2020-06-12 14:22:21 +02:00
Cecile Tonglet e3fc4f7fba Add a feature to create automatically a random temporary directory for base path & remove Clone (#6221)
* Initial commit

Forked at: 4adac40c07
Parent branch: origin/master

* Add a feature to create automatically a temporary directory for base path

* doc fix and todos

* use parking_lot instead

* use refcell instead since we stay in the main thread

* remove Clone derives

* add test

* solving dependency issue

* clarifying doc

* conflict argument with base-path

* WIP

Forked at: 4adac40c07
Parent branch: origin/master

* revert dep deletion

* fixing test and making base_path optional

* hold basepath while the service is running

* fixes

* Update client/cli/src/params/shared_params.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/mod.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/config.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* WIP

Forked at: 4adac40c07
Parent branch: origin/master

* improve doc

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-06-10 13:13:25 +02:00
Bastian Köcher 8a7c2c3142 CLI: Make --dev conflict with --chain (#6146)
If we are running `--dev` chain, we should forbid the `--chain`
argument. The `--dev` chain is always special by only having one
authority etc and some other chain spec is probably not setup for this
correctly. In the end `--dev` is just a shortcut for `--validator --alice`.
2020-05-26 21:25:15 +02:00
pscott 18d4fa10d2 Add JSON format to import blocks and set it as default (#5816)
* Add BlockStream Enum and utility fn

* WIP: Modify import closure to work with BlockStream

* Fix trait bounds

* Working prototype

* Revamp block importing

* Add export_import_flow tests

* Add comments and clean code

* Add more comments in the import fn

* Add link code to import function

* Add condition when returning Ready(Ok(()) to make sure we've imported every block

* Add check for imported blocks in JSON case

* Use rest pattern

* Fix compilation error for undeclared variable

* Add polling and waker before pending

* Print read_block_count instead of count

* Simplify binary cli option with structopt

* Update test to reflect changes in CLI api

* Change Stream to take SignedBlock<B> instead of B

* Add comments to BlockStream

* Move out logic to smaller functions for clearer code

* Remove result over import_blocks return type

* Check for error in command output rather than simply checking command exit status

* Revamp export/import/revert testing

* Fix minor typos and formatting errors

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove unnecessary if condition in terminating condition

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Explicit error instead of returning it as a string

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Pass BlockStream to log_importing_status_updates and simplify matching arms for block stream

* Use .contains() instead of regex match

* Line break in match block; return future::ready instead of poll_fn

* Update Cargo.lock

* Add check so that queue doesn't grow too big

* Use Iterator instead of Stream

* Remove allow dead_code

* Remove outdated comments

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Return Errors instead of logging them

* Simplify match arms

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove check before terminating block import

* Apply suggestions from code review

* Check that queue is not full BEFORE calling

* Revert "Remove check before terminating block import"

This reverts commit 377823c0a648a3eb2e61185a257a61023067893d.

* Improve unit tests to make sure we actually import blocks

* Remove Unpin implementation for BlockIter

* Add prototype of enum for ImportStates

* Add working prototype for StateMachine

* Add comments for clearer code

* Add sleep before calling Waker when waiting for import queue

* Add Speedometer

* add dbg!(&log) for test debugging

* Fix lines with more than 100 cols

* Fix regex capture for test

* Update regexes to take to capture the whole number

* Rename Cmd to Command

Co-authored-by: Gavin Wood <gavin@parity.io>

* Actually rename Cmd to Command

* Apply suggestions from code review

Co-authored-by: Gavin Wood <gavin@parity.io>

* Fix compilation errors for tests

* Fix compilation errors from code review suggestion

* Update bin/node/cli/tests/export_import_flow.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
2020-05-22 13:50:25 +02:00
Arkadiy Paronyan 6c47b130e8 Handle piping error for commands that output to stdout (#6098)
* Handle piping error for commands that output to stdout

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-05-21 13:50:37 +02:00
Pierre Krieger 717fa95bc7 Restore the empty line after the license (#6088) 2020-05-20 13:08:27 +00:00
Nikolay Volf d5411969ac fix whitespace (#6062) 2020-05-18 13:51:28 +02:00
Benjamin Kampmann be8c96adec Relax substrate licensing scheme (#5947) 2020-05-15 13:21:56 +02:00
Pierre Krieger 0690bb51a8 Allow passing multiple --log CLI options (#5982)
* Allow passing multiple --log CLI options

* Comment typo
2020-05-12 11:46:06 +02:00
Igor Matuszewski 9acf88f58b client: Replace unsafe_rpc_expose with an RpcMethods enum (#5729)
* client: Replace `unsafe_rpc_expose` with an `RpcMethods` enum

which can be either Default, Safe or Unsafe. The idea is to have the
following:
|                       | --rpc-external=false  | --rpc-external=true   |
|---------------------  |-------------------    |-----------------      |
| --rpc-methods=Default |                       | unsafe calls denied   |
| --rpc-methods=Safe    | unsafe calls denied   | unsafe calls denied   |
| --rpc-methods=Unsafe  |                       |                       |
Since the previous `unsafe-rpc-expose` option was confusing.

* client: Only warn against exposing externally unsafe RPC method set

* Apply suggestions from code review

Co-Authored-By: Cecile Tonglet <cecile.tonglet@cecton.com>

* cli: Rephrase doc comment for rpc_methods config

* Improve debuggability of build_spec_works

...by printing to stderr the stderr of the command. This is normally
suppressed for succesful tests but not for failing ones - if that's the
case then it's useful to see the test failure reason inline rather than
having to execute the command separately ourselves.

* Rename RpcMethods::{Default => Auto} variant

* Update bin/node/cli/tests/build_spec_works.rs

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-05-06 11:30:54 +02:00
Bastian Köcher 5b8d3607e9 Adds export-state subcommand (#5842)
* Export state cli

* More work

* Fix tests

* Make it work

* Fix compilation

* Apply suggestions from code review
2020-04-30 15:44:40 +02:00
Arkadiy Paronyan c3a6d8a881 Fix purge-chain and print DB info on startup (#5840)
* purge-chain accepts --db option

* print DB info on startup

* Small refactoring

* Added back &self

* Add DatabaseParams for PurgeChain, ImportParams and ExportBlocks

* Don't force default value

* Remove unused fields

* Update client/cli/src/commands/export_blocks_cmd.rs

* Fix stuff

Co-authored-by: Cecile Tonglet <cecile@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-30 14:53:04 +02:00
Saqib Rokadia aff24d6fbb Adding optional public addresses for proxying (#5807)
* Adding optional public addresses for use in network configurations to allow for proxies in front of a node.

* Apply suggestions from code review

Co-Authored-By: Cecile Tonglet <cecile.tonglet@cecton.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-04-29 19:08:30 +02:00
Seun Lanlege 4fa5941f44 Move sc-client into sc-service (#5502)
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api

* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa

* drop sc-service from sc-rpc

* drop sc-service from sc-consensus-aura

* drop sc-client from manual-seal and babe

* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli

* drop sc-client from bin/node and bin/node-template

* drop sc-client

* fix tests

* remove check -p sc-client from gitlab.yml

* fix warnings

* fixes ui test

* fix light client tests

* adds associated Client type to AbstractService

* adds UsageProvider to Client

* fixed ui test, again

* tried and failed to get node-cli to compile for wasm

* thanks to tomaka for helping me get node-cli to compile for wasmm

* ui test pls pas 🙏🏾

* all tests passing 🪄

* no_run documentation code

* rm -f documentation code

* ClientProvider

* fix mega trait

* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client

* adds license to sc-consensus

Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-04-28 11:59:31 +00:00
André Silva 5662501258 cli: fix unused unsafe-pruning flag (#5803) 2020-04-27 22:01:35 +02:00
Bernhard Schuster 72ee7d5797 feat/ocw/bookkeeping (#5200)
Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-04-24 16:46:19 +02:00
Igor Matuszewski 4b1f7d187f Allow to expose a subset of unsafe RPCs (#5233)
* sc-cli: Use type-safe constructors for RPC/Prometheus interfaces

* service: Simplify rpc handler creation

Could probably be further simplifies once [this][commit] lands.

[commit]: https://github.com/paritytech/jsonrpc/commit/20485387ed06a48f1a70bf4d609a7cde6cf0accf

* service: Streamline some HTTP & WS server start logic

* client: Introduce a simple RPC policy mechanism

* rpc/system: Check unsafe RPCs

* rpc/offchain: Check unsafe RPCs

* rpc/author: Check unsafe RPCs
2020-04-20 11:03:58 +02:00
Ashley 95dc400bb8 Make network_config_path an Option (#5661)
* Make network_config_path an Option

* Fix network tests

* Use None as the network config path

* Fix cli

* Don't make PathBuf an Option in a cli context
2020-04-17 14:48:45 +02:00
Gavin Wood 91af5b6fcc New database trait (#5549)
* Introduce trait

* The trait

* Generic

* Basic impls.

* Remove unneeded bounds

* Minor changes

* Switch over to the new DB trait

* Integrated parity-db and added CLI for db selection

* Default impl.

* Fix logs.

* Started integrating subdb

* Apply suggestions from code review

Co-Authored-By: Cecile Tonglet <cecile@parity.io>

* Apply suggestions from code review

Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Enable subdb

* Bump parity-db

* Fixed CLI macro

* Fixed browser build

* Fixed features

* Sort out features

* Use parity-db from crates.io

* Typo

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Cecile Tonglet <cecile@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-04-15 14:38:39 +02:00
Cecile Tonglet 3da069e359 CLI: refactoring: remove Options from sc_service::Configuration's fields (#5271)
* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Rename IntoConfiguration to CliConfiguration

* Renamed into_configuration to create_configuration

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Move keystore params to its own module

* Use in-memory keystore even for build-spec

* Enforce proper value for node name

* dev_key_seed

* Telemetry endpoints

* rustfmt

* Converted all RunCmd

* rustfmt

* Added export-blocks

* Missed something

* Removed config_path in NetworkConfiguration (not used)

* Fixed warnings

* public_addresses is used but never set, keeping it

* Merge Configuration.node and NetworkConfiguration.node_name

...because they are the same thing

* Added: import-blocks

* Adding a proc_macro to help impl SubstrateCli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Re-export spec_factory from sc_cli

* Re-added all the commands

* Refactored node_key_params

* Fixed previous refucktoring

* Clean-up and removed full_version()

* Renamed get_is_dev to not confuse with Configuration field

* Fixed sc-cli-derive example

* Fixing tests

* Fixing tests and removing some (will re-add later)

* Fixing more tests

* Removes the need of type parameter

* Converting bin/node and simplifying API

* Converting more

* Converting last command

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fixing tests and added default for WasmExecutionMethod

* Fixing stuff

* Fixed something I broke oops

* Update Cargo.lock

* Moving things around

* Convert everything to Result

* Added new macros to simplify the impl of CliConfiguration

* Added a macro to generate CliConfiguration automatically for subcommands

* Revert... too many macros (this one is not really useful)

This reverts commit 9c516dd38b40fbc420b02c1f8e61d5b2b1a4e434.

* Renamed is_dev to get_is_dev

Good enough for now

* Fixed name roles (this is plural, not singular)

* Clean-up

* Re-export NodeKeyConfig and TelemetryEndpoints from sc_service

* Improve styling/formatting

* Added copyrights

* Added doc and fixed warnings

* Added myself to code owners

* Yes it is needed according to the history

* Revert formatting

* Fixing conflict

* Updated build.rs

* Cargo.lock

* Clean-up

* Update client/cli-derive/Cargo.toml

Co-Authored-By: Seun Lanlege <seunlanlege@gmail.com>

* Fail if using proc_macro and build.rs is not set properly

* Dropped all get_ in front of methods

* Clean-up

* Fixing proc macro missing env var

* Get the configuration inside the Runtime (needed for polkadot)

* Clean-up

* Get is_dev from argument like the others

* Get chain ID instead of chain spec from shared params

* &self is passed to spec_factory/load_spec

* Wrong text

* Fix example

* Officialize macro and made a cool doc

* Renamed spec_factory to load_spec (substrate_cli_configuration)

* Removed not so useful ChainSpec

* Renamed SubstrateCLI to SubstrateCli

* Added changelog for impl_version being full now

* Renamed Runtime to Runner

* Update changelog to show example

* Removed option on database cache size

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Fix on removal of option

* typo

* Clean-up imports

* Added info in Cargo.toml

* typo

* remarks

* Moved function for build.rs to substrate-build-script-utils

* Fixed example & test of cli-derive

* Moved function for build.rs to substrate-build-script-utils

* Renamed substrate_cli_configuration to substrate_cli oops

It implements SubstrateCli not CliConfiguration!

* Added documentation and wrapper macro

* Removed option on database cache size

* Removed option on database cache size

* Clean-up

* Reduce risk of errors due to typos

* Removed option on database cache size

* Added NOTE as suggested

* Added doc as suggested

* Fixed test

* typo

* renamed runtime to runner

* Fixed weird argument

* More commas

* Moved client/cli-derive to client/cli/derive

* Added 7 tests for the macros

* Improve error message

* Upgrade assert_cmd

* Fixing missing stuff

* Fixed unused import

* Improve SubstrateCli doc

* Applied suggestions

* Fix and clean-up imports

* Started replacing macros WIP

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* Started removing substrate_cli

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* WIP

Forked at: d6aa8e954c
Parent branch: origin/master

* fixed bug introduced while refactoring

* Renamed NetworkConfigurationParams to NetworkParams for consistency sake

* Fixed test

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/runcmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/check_block_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 5906776953392c02beac6bc0bf50f8cbe1a12a01.

* Revert "Update client/cli/src/commands/check_block_cmd.rs"

This reverts commit f705f42b7f3d732be001141afee210fe46a1ef47.

* Revert "Update client/cli/src/commands/export_blocks_cmd.rs"

This reverts commit 8d57c0550164449e6eb2d3bacb04c750c714fcea.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 93e74cf5d2e1c0dc49cdff8608d59fc40fc59338.

* Revert "Update client/cli/src/commands/runcmd.rs"

This reverts commit 11d527ba345c0d79f0d3b5b071933d95474d0614.

* Update client/cli/src/commands/export_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/import_blocks_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Changed ::sc_cli to $crate in the macro

* fixed tests

* fixed conflicts

* Fixing test

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove comment as suggested

* Apply suggestion

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/purge_chain_cmd.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/runner.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/pruning_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/node_key_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/params/network_params.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/config.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added doc

* Fixed error introduced after applying suggestion

* Revert "Update client/cli/src/params/pruning_params.rs"

This reverts commit 0574d06a4f1efd86e94c1214420a12e7a4be0099.

* Print error

* Apply suggestions from code review

* Remove useless Results

* Fixed CI failing on polkadot approval

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-07 11:38:07 +02:00
pscott 308dbf1c7a Make verbosity level mandatory with telemetry opt (#5057)
* Make verbosity level mandatory instead of defaulting to 0 when using --telemetry-url

* Update README docs

* Change TelemetryError struct to enum

* Return TelemetryParsingError instead of a Boxed dyn error

* Replace spaces by tabs

* Add example of expected format for telemetry-url

* Remove UrlParsingError; Call to_string instead of parse for TelemetryEndpoints url
2020-04-05 14:36:45 +02:00