Commit Graph

64 Commits

Author SHA1 Message Date
Bastian Köcher 709a899f9d Cleanup transaction pool deps (#4782)
* Cleanup transaction pool deps

* Fix it properly

* Fix doc test
2020-01-31 12:40:32 +01:00
Cecile Tonglet 605f643eed CLI API refactoring and improvement (#4692)
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).

This touches a few important things:
 - the startup of the async task with tokei:
    This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
 - increased the version of structopt and tokei
 - no more use of structopt internal's API
 - less use of generics

Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want

Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
2020-01-30 11:40:08 +01:00
Sergei Pepyakin 2c90562818 Clean up indentation problems (#4762) 2020-01-29 13:27:23 +01:00
Benjamin Kampmann 3f9a05a0d3 clarify licensing (#4755)
* adding license fields to all crates

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-29 11:57:13 +01:00
Pierre Krieger b452867eb7 Add diagnostics to tasks (#4752) 2020-01-29 11:46:39 +01:00
Edwin 9a6d9f0db5 RPC api for offchain storage (#4694)
* Rpc api for offchain storage

* Replace Vec<u8> to Bytes, replace Mutex to RwLock

* Remove pub

* Modify copyright year
2020-01-28 23:31:28 +01:00
Pierre Krieger 5c8743510e Upgrade to libp2p 0.15 (#4732)
* Fixes for libp2p 0.15

* More work

* Update libp2p

* Update to libp2p 0.15
2020-01-27 18:27:36 +01:00
Nikolay Volf ed3da9f903 Fix quadratic iterations over transaction pool (#4736)
* transaction pool changes

* service & network changes

* address review

* reduce future pool
2020-01-27 18:26:42 +01:00
Ashley c2c429877e Switch GrandPa to std futures (replaces #3909) (#4612)
* Switch GrandPa to new futures

* Work on making tests work

* until_imported tests working again

* Work on switching tests to stable futures

* Modifications

* Re-add test as #[ignore]

* Don't ignore

* Add manual unpins

* Remove Header import

* Return concrete Sink type

* Switch to crates.io finality-grandpa version

* Remove use statement that slipped in

* Fix some nitpicks

* Remove unpin from i

* Fixed typo

* Move futures01 to dev-deps

* Fix nitpicks

* Update client/finality-grandpa/src/communication/mod.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* nitpicking

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-01-24 13:34:42 +01:00
Nikolay Volf 14e95f3398 Refactor tx-pool maintenance and other high-level api (#4629)
* Reduction.

* Reformation.

* add locked timer stuff

* fix issues and introduce full pool

* arrange together

* fix benches

* fix new_light

* Add revalidation test case

* review fixes

* review fixes

* use just ready future

* address review
2020-01-24 04:21:24 -08:00
Pierre Krieger 169a48c0c5 Pass an executor through the Configuration (#4688)
* Pass an executor through the Configuration

* Make tasks_executor mandatory

* Fix tests
2020-01-21 13:06:15 +01:00
Bastian Köcher afc3318f21 Implement runtime version checks in set_code (#4548)
* Implement runtime version checks in `set_code`

Check that the new runtime code given to `set_code` fullfills some
requirements:

- `spec_name` matches
- `spec_version` does not decreases
- `impl_version` does not decreases
- Either `spec_version` and `impl_version` increase

* Make tests almost work

* Some fixes after master merge

* Fix tests

* Add missed file

* Make depedency check happy?

* Remove leftover `sc-executor`

* AHHHHH

* Reset debug stuff

* Remove some 'static

* More 'static

* Some docs

* Update `Cargo.lock`
2020-01-16 13:58:37 +01:00
Robert Habermeier 69e0cf7fe2 reset a couple of versions down from v2.0 (#4572)
* reset a couple of versions down from v0.2.0

* Unstablise browser-utils (very new) and grafana-data-source (going to be replaced with prometheus sometime)

* unmark a bunch more client crates for stabilization

* fix versions in Cargo.tomls

* Downgrade network-test

* Set frame-metadata version to `10.0.0`

* Hide and documented storage generator as internal api

* Downgrade `sp-externalities`

* Downgrade `sc-cli`

* Downgrade sc-executor et al to 0.8.0

* Downgrade sp-state-machine to 0.8.0

* Downgrade pallet-contracts et al to 0.8.0

* Downgrade sp-sandbox to 0.8.0

* downgrade pallet-evm to 0.8.0

* Downgrade pallet-staking to 0.8.0

2.0 should be implementation of lazy payout alongside all the fixes of
current module

* Downgrade node-transaction-factory to 0.8.0

* frame pallets are 2.0.0

Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Wei Tang <accounts@that.world>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
2020-01-15 18:35:27 +01:00
Ashley 3219be2508 Update the service to std futures (#4447)
* Switch service to futures03

* Fix tests

* Fix service test and cli

* Re-add Executor trait to SpawnTaskHandle

* Fix node-service

* Update babe

* Fix browser node

* Update aura

* Revert back to tokio-executor to fix runtime panic

* Add todo item

* Fix service tests again

* Timeout test futures

* Fix tests

* nits

* Fix service test

* Remove zstd patch

* Re-add futures01 to aura and babe tests as a dev-dep

* Change failing test to tee

* Fix node

* Upgrade tokio

* fix society

* Start switching grandpa to stable futures

* Revert "Start switching grandpa to stable futures"

This reverts commit 9c1976346237637effc07c13f7d0403daf5e71cf.

* Fix utils

* Revert substrate service test

* Revert gitlab

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2020-01-14 15:43:45 +01:00
Benjamin Kampmann 5946d6f7a0 Allow node to add externalities extensions (#4583)
* allow extensions to be extensible from the outside
2020-01-13 12:07:54 +01:00
Bastian Köcher e8dd1205ee Add test for cli keystore path generation (#4571)
* Add test for cli keystore path generation

* Fix test
2020-01-11 18:47:26 +01:00
Bastian Köcher fd6b29dd2c Remove requirement on Hash = H256, make Proposer return StorageChanges and Proof (#3860)
* Extend `Proposer` to optionally generate a proof of the proposal

* Something

* Refactor sr-api to not depend on client anymore

* Fix benches

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Apply suggestions from code review

* Introduce new `into_storage_changes` function

* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code

* Put the `StorageChanges` into the `Proposal`

* Move the runtime api error to its own trait

* Adds `StorageTransactionCache` to the runtime api

This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(

* Remove `type NodeBlock` and switch to a "better" hack

* Start using the transaction cache from the runtime api

* Make it compile

* Move `InMemory` to its own file

* Make all tests work again

* Return block, storage_changes and proof from Blockbuilder::bake()

* Make sure that we use/set `storage_changes` when possible

* Add test

* Fix deadlock

* Remove accidentally added folders

* Introduce `RecordProof` as argument type to be more explicit

* Update client/src/client.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update primitives/state-machine/src/ext.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Integrates review feedback

* Remove `unsafe` usage

* Update client/block-builder/src/lib.rs

Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>

* Update client/src/call_executor.rs

* Bump versions

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
2020-01-10 10:48:32 +01:00
Tomasz Drwięga 74d6e660c6 Add documentation to SubmitSignedTransaction and actually make it work (#4200)
* Add documentation to signed transactions and actually make them work.

* Fix naming and bounds.

* Forgotten import.

* Remove warning.

* Make accounts optional, fix logic.

* Split the method to avoid confusing type error message.

* Move executor tests to integration.

* Add submit transactions tests.

* Make `submit_transaction` tests compile

* Remove a file that was accidently committed

* Add can_sign helper function.

* Fix compilation.

* Add a key to keystore.

* Fix the tests.

* Remove env_logger.

* Fix sending multiple transactions.

* Remove commented code.

* Bring back criterion.

* Remove stray debug log.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Make sure to initialize block correctly.

* Initialize block for offchain workers.

* Add test for transaction validity.

* Fix tests.

* Review suggestions.

* Remove redundant comment.

* Make sure to use correct block number of authoring.

* Change the runtime API.

* Support both versions.

* Bump spec version, fix RPC test.

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-10 01:46:55 +01:00
Pierre Krieger ca997cf1e4 Update networking code to libp2p 0.14 (#4383)
* Entirely update substrate-telemetry to futures 0.3

* Add a Closed error

* Update to libp2p 0.14

* More work

* More work

* More work

* More work

* Fix warnings

* Remove unwrap()

* Work on tests fixing

* Fix network tests

* Fix external network tests

* Update libp2p and restore Yamux in discovery test

* Ignore DNS if initializatio nfails

* Restore variables ordering

* Forgot browser-utils

* Fix downfall after merge

* Fix tests
2020-01-09 19:01:23 +01:00
Nikolay Volf df4058b556 i/o stats for backend databases (#4525) 2020-01-07 21:53:03 +03:00
Ashley bb44f8fc24 Add a browser-utils crate (#4394)
* Squash

* Fix keystore on wasm

* Update utils/browser/Cargo.toml

Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>

* export console functions

* Use an Option<PathBuf> in keystore instead of cfg flags

* Add a KeystoreConfig

* Update libp2p

* Bump kvdb-web version

* Fix cli

* Upgrade versions

* Update wasm-bindgen stuff

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
2020-01-07 16:30:04 +01:00
André Silva a89d87d708 client: add a block blacklist extension (#4544)
* client: add a block blacklist extension

* test-utils: fix client construction

* client: fix rustdoc test
2020-01-06 18:03:44 +00:00
André Silva 4fa4dfb77b client: helper to create standalone client without service (#4536) 2020-01-06 14:52:14 +00:00
Shawn Tabrizi 7efa62725e Update copyright year (#4532) 2020-01-05 14:05:44 +01:00
Max Inden c5fe4295f8 *: Register network event stream for authority discovery (#4344)
Previously one would create a sender and receiver channel pair, pass the
sender to the `build_network_future` through the service builder and
funnel network events returned from polling the network service into the
sender to be consumed by the authority discovery module owning the
receiver.

With recent changes it is now possible to register an `event_stream`
with the network service directly, thus one does not need to make the
detour through the `build_network_future`.
2020-01-03 20:47:12 +00:00
Nikolay Volf 6d06a19f41 Refactor to use only chain info (#4516) 2020-01-02 12:46:07 +01:00
Weiliang Li fe587c72ec Use sc_network::NetworkStateInfo instead of implementing redundant traits (#4436)
* Implement local_peer_id for gossip

* refactor local_peer_id

* fix

* reset gossip

* Update tests.rs

* fix ci

* fix review

* fix Cargo.lock

* fix Cargo.lock
2019-12-24 16:17:19 +01:00
Arkadiy Paronyan 9950ea98fc Shared params in CLI API (#4466)
* Common shared parames getter

* Expose more types from `service-builder`

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2019-12-20 16:05:01 +01:00
Robert Habermeier da87c3c1e3 RPCs for reserved peers (#4423)
* RPC forwarders for adding reserved peers

* implement service side of reserved-peer RPCs

* Clean up unnecessary format! invocation

Co-Authored-By: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* add some tests for the new RPC methods

* remove redundant `data` field

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2019-12-18 14:12:15 +01:00
Benjamin Kampmann 9200bfa997 Setting versions locally towards 2.0 release (#4404)
* clean up cargo.toml syntax

* bumping versions to 2.0

* bump networking to 0.8

* move consensus down to 0.8

* bump consensus pallets to 0.8.0, too

* Upping babe and aura pallets

* add remaining, missing version definitions

* missed some
2019-12-17 22:05:50 +08:00
Svyatoslav Nikolsky cd718cd77d recover light client integration tests (#4377) 2019-12-17 08:45:04 +01:00
Tomasz Drwięga 8778ca7dc8 [big refactor] Remove crate aliasing. (#4395)
* Rename: Phase 1.

* Unify codec.

* Fixing: Phase 2

* Fixing: Phase 3.

* Fixing: Phase 4.

* Fixing: Phase 5.

* Fixing: Phase 6.

* Fixing: Phase 7.

* Fixing: Phase 8. Tests

* Fixing: Phase 9. Tests!!!

* Fixing: Phase 10. Moar tests!

* Finally done!

* More fixes.

* Rename primitives:: to sp_core::

* Apply renames in finality-grandpa.

* Fix benches.

* Fix benches 2.

* Revert node-template.

* Fix frame-system in our modules.
2019-12-16 20:36:49 +08:00
Pierre Krieger c66c191b68 Extract consensus_gossip.rs and put it in its own crate (#4284)
* Extract gossiping system from network

* Finish porting GRANDPA tests

* Try put correct engine ID

* Fix messages encoding

* Fix communication tests

* Use a threads pool to spawn stuff

* Fix compilation everywhere

* Fix bad merge conflict

* Remove dependency on async-std

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* More suggestions

* Remove network startup GP future

* Update to futures_timer

* adjust wait_when_behind test

* Pass correct Roles after handshake

* Revert "adjust wait_when_behind test"

This reverts commit 23cb3a0a6d25ed732c2cd648607bc44ef2ab0919.

* Crate root documentation

* Remove MessageRecipient

* Address concerns

* Fix more concerns

* Forgot Cargo.lock
2019-12-13 19:16:10 +01:00
Benjamin Kampmann c5a709a882 Fix folder names in client and test (#4360)
* client/rpc/api -> client/rpc-api

* client/util/wasm-builder-runner -> utils/wasm-builder-runner

* client/grafana-data-source -> utils/grafana-data-source

* test/utils -> test-utils

* fix moved path

* Update Cargo.lock

* Update Cargo.lock
2019-12-11 16:41:38 +01:00
Benjamin Kampmann 7773daaf5b Clean up crate names and locations (#4361)
* rename crate: sp-transaction-pool-api -> sp-transaction-pool

* move primitives/core/derive-debug -> primitives/derive-debug; primitives/core/storage -> primitives/storage

* rename crate sp-core-storage -> sp-storage

* rename and move: test/utils/transaction-factory -> client/transaction-factory

* move transaction-factory -> node/transaction-factory

* fix missing rename

* Move chain-spec-builder into bin/utils

* move subkey into bin/utils

* Update new subkey location

* Update docs to reflect new location for utils

* fixing import name
2019-12-11 15:33:00 +01:00
Benjamin Kampmann f0cc89a4b0 Break appcrypto dep cycle (#4347)
* move application-crypto tests into seperate crate

* Fix crate name, update script and add docs

* pressing 'save' helps

* remove useless lines

* adding header

* split into actual modules
2019-12-10 22:12:57 +01:00
Benjamin Kampmann 8721d98dd6 break sp-api dependency cycle (#4352)
* move benches into tests, ignore non-passing doctests

* Rename sr-api folder

* Move test-primitives to primitives, use that for sp-api doctests
2019-12-10 20:18:01 +01:00
Weiliang Li 4f2cdb20c1 Rename: primitives/sr-io -> primitives/sp-io (#4328)
* primitives/sr-io -> primitives/io

* fix

* rename

* runtime-io -> sp-io

* git mv

* fix ci

* remove package name

* fix

* fix

* try minimizing diff

* try minimizing diff again

* try minimizing diff again
2019-12-10 16:08:35 +01:00
Weiliang Li 04fcc71809 Rename folder: primitives/sr-primitives -> primitives/runtime (#4280)
* primitives/sr-primitives -> primitives/runtime

* update
2019-12-09 08:49:32 +01:00
Tomasz Drwięga 3805393a13 Merge txpool-runtime-api with txpool-api (#4320)
* Remove transaction-pool-runtime-api

* Merge runtime-api with transaction-pool.
2019-12-06 17:24:17 +01:00
Ashley 925b23a3cd Use timestamp deltas in grafana-data-store (#4199)
* Optimize grafana storage slightly

* Test on grafana test binary

* Cast to f32

* Silence record_metrics warning

* Add source for error

* More changes

* Fix record_metrics_slice

* Test Database

* Docs

* Update client/grafana-data-source/src/lib.rs

Co-Authored-By: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Fix macro

* Update server.rs
2019-12-04 18:49:12 +01:00
Ashley a58e8a6e45 Update exit-future and make sc-cli compile on wasm (#4289)
* updated exit-future (github repo)

* Switch to broadcast crate

* Migrate client/cli

* Switch exit-future to modernize branch

* Small changes

* Switch to cargo version and fix fg tests

* Revert "Small changes"

This reverts commit a488106805d220cb4aee9e46a71481424c6d87d5.
2019-12-04 18:35:33 +01:00
Bryant Eisenbach c071276187 fix: Typo granpda -> grandpa (#4286) 2019-12-04 00:05:05 +01:00
Max Inden 2de6f82b8a client: Introduce --sentry-nodes flag (#4285)
* client/authority-discovery: Add smoke tests for intervall_at

* client/authority-discovery: Fix interval_at to fire on start

* .maintain/sentry-node: Update docker compose file

* client: Introduce --sentry-nodes flag

Enable operators to specify the public addresses of sentry nodes infront
of a validator node so that the validator node can announce the sentry
node addresses instead of its own public addresses on the DHT via the
authority discovery module.

* client/authority-discovery: Break lines at 100 characters

Limit line length to 100 instead of 120 characters.
2019-12-03 18:03:08 +01:00
Arkadiy Paronyan 5ec0923285 Reputation changes requires reason (#4277) 2019-12-03 11:33:33 +01:00
Andrew Jones 4c2ba5b71f Fix sp-session typo (#4273)
* Fix sp-session typo

* Remove erroneous whitespaces

* Don't mess with the file, editor
2019-12-02 20:24:25 +01:00
Benjamin Kampmann 810ac845b5 Fix sc-tracing-* -> sc-transaction- (#4263)
* fixing faulty tracing-rename

* missed one
2019-12-02 12:19:53 +01:00
Benjamin Kampmann 927e13c13a The crate rename (#4223)
* Adding script for rename, could be applicable for nodes on top of it, too

* add stderr and gitlab ci features

* apply script

* fix now minor details in expected stderr

* Update the Cargo.lock

* fix name: sc-transaction -> sc-tracing

* fix rename in script, too
2019-12-02 11:23:53 +01:00
Arkadiy Paronyan 504e2f8bd5 Check block CLI command (#4240)
* Check block operation

* Update client/cli/src/lib.rs

* Update client/cli/src/params.rs
2019-11-28 11:24:28 +00:00
Tomasz Drwięga f78b83e363 Handle Grafana server bind errors gracefuly. (#4241)
* Fix error handling for grafana sever start.

* Update client/grafana-data-source/src/server.rs

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
2019-11-28 11:16:54 +00:00