Commit Graph

2421 Commits

Author SHA1 Message Date
Nikolay Volf 41967186e4 Subsystems memory tracking: 1. Transaction pool (#4822)
* update sp-runtime

* total update

* usage informant

* update to crates.io version

* update Cargo.lock

* update dummy update

* fix todo

* cleanup

* avoid custom impl

* Update client/transaction-pool/graph/src/future.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* remove another custom impl

* remove another custom impl

* add kb in report

* update Cargo.lock

* review suggestions

* --amend

* --amend

* bump parity-util-mem to 0.5.0

* bumps

* update macro and versions

* add to grafana

* naming

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-02-07 11:53:11 +01:00
Marcio Diaz 41644c2ae8 Enable trace timings logs for transaction factory (#4845)
* Enable trace timings logs for transaction factory.
2020-02-07 08:12:19 +01:00
Nikolay Volf 8c373916a4 easy with threads (#4848) 2020-02-06 23:32:48 +01:00
Wei Tang 14b0c9a746 babe: pass epoch data via intermediates (#4807)
* babe: pass epoch data via intermediates

* Switch to use Box<dyn Any> for intermediates

* Set intermediate.epoch to be Option

* Fix proposer should put out an empty intermediate

* Remove unnecessary encode/decode

* Add EpochData to block_import_params in slot worker

* Fix aura compile

* Fix integration test
2020-02-06 17:52:30 +00:00
Wei Tang 4df27e760e Refactor epoch changes to a separate crate (#4785)
* Init epoch changes module

* Initial integration of new epoch changes module for BABE

* Fix all initial compile errors

* rename: digest -> digests

* Fix babe tests

* Bump impl_version

* Fix more test issues

* Remove test flag for tree

It unfortunately won't work for multiple crates.

* Update cargo lock

* Fix duplicate parking_lot version

* Add missing license header
2020-02-06 16:48:38 +01:00
Cecile Tonglet be075893b5 CLI improvements & fixes (#4812)
These are a few changes I missed during the refactoring.

1. Initialization issue and boilerplate

    Most importantly: part of the `Configuration` initialization was done in `sc_cli::init`. This means the user can not benefit from this initialization boilerplate if they have multiple `Configuration` since `sc_cli::init` can only be called once.

2. Boilerplate for `VersionInfo` and `Configuration`

    I'm also answering to the critic of @bkchr on the initialization using version: https://github.com/paritytech/substrate/pull/4692/files/bea809d4c14a2ede953227ac885e3b3f9771c548#r372047238 This will allow initializing a `Configuration` and provide the version by default.

3. Loading the `chain_spec` explicitly

    In the past it was done automatically but in some cases we want to delay this. I moved the code to `Configuration.load_spec()` so it can be called later on. `chain_spec` can also be written directly to the `Configuration` without using this `load_spec` helper.

4. [deleted]

5. Fixing issue that prevents the user to override the port

    In the refactoring I introduced a bug by mistake that could potentially prevent the CLI user to override the ports if defaults where provided for these ports (only on cumulus).

6. Change task_executor from Box to Arc

    This is useful for cumulus where we have 2 nodes with 2 separate Configuration that need to spawn tasks to the same runtime.

7. Renamed TasksExecutorRequired to TaskExecutor

    For consistency.

This is related to https://github.com/paritytech/cumulus/issues/24

This is the continuation (and hopefully the end of) #4692
2020-02-06 15:46:49 +01:00
Marcio Diaz f891342f20 Fix broken factory by adding keystore back. (#4840) 2020-02-06 14:48:19 +01:00
Arkadiy Paronyan 3c79e6d03c Fixed a few warnings (#4841) 2020-02-06 14:06:53 +01:00
Nikolay Volf 6148c7ce15 do join_all (#4832) 2020-02-06 13:07:52 +01:00
André Silva d53bb5ec50 grandpa: bump version to v0.11.1 (#4813) 2020-02-05 21:47:11 +01:00
Sergei Pepyakin 1af9e4f3ee executor: Simplify the SandboxCapabilities interface (#4825)
* Don't require `store` and `store_mut` in `SandboxCapabilities`.

* Simplify the sandbox a bit
2020-02-05 18:20:25 +01:00
Gavin Wood 9202cd87e0 Additional RPC for dumping all main storage key pairs under a prefix (#4803)
* Merge branch 'gav-split-balanecs-vesting' into gav-upsub

# Conflicts:
#	Cargo.lock
#	cli/Cargo.toml
#	collator/Cargo.toml
#	primitives/Cargo.toml
#	runtime/common/Cargo.toml
#	runtime/common/src/claims.rs
#	runtime/kusama/Cargo.toml
#	runtime/polkadot/Cargo.toml
#	service/Cargo.toml

* Update client/src/client.rs

* Update client/src/client.rs

* Fix merge conflict

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-02-05 17:27:50 +01:00
Pierre Krieger 54c10893ab Print an error if listener is closed (#4830)
* Print an error if listener is closed

* Oops, forgot to commit this
2020-02-05 17:25:14 +01:00
Pierre Krieger a3439999cf Remove support for secio (#4831) 2020-02-05 16:56:49 +01:00
Pierre Krieger 39c037ab22 Fix missing overrides of NetworkBehaviour (#4829) 2020-02-05 11:56:59 +01:00
Demi Obenour 0ab013bddb Bump parking_lot (#4817)
Hopefully this can be merged if the tests pass.
2020-02-04 16:56:50 +01:00
Xiliang Chen febb6e29b2 add state_getKeysPaged (#4718)
* add storage_getNextKey

* RPC storage_getKeysPages

* respect count

* use iterator

* improve

* add tests

* improve

* add doc comments

* Make prefix optional

* update error

* improve
2020-02-04 10:44:40 +01:00
Wei Tang bff1f4a18d block_import: switch to Box<dyn Any> for intermediates representation (#4809)
* block_import: switch to Box<dyn Any> for intermediates representation

* Use Cow and return Error instead of Option

* Remove unused error

* Distinguish NoIntermediate/InvalidIntermediate
2020-02-03 09:55:09 +01:00
Pierre Krieger b6e8fba179 Remove warning about connecting to non-Substrate nodes (#4797) 2020-02-01 14:02:10 +01:00
Nikolay Volf 8a3caaeb68 update prost-build (#4789) 2020-02-01 12:33:13 +01:00
Nikolay Volf 63ad759c69 Increase transaction pool default limits (#4792)
* increase limits, improve logging

* format in kB
2020-02-01 00:27:53 -08:00
Gavin Wood adde404e41 client/finality-grandpa: Add regression test for observer polling network (was #4778) (#4795)
* client/finality-grandpa: Add regression test observer polling network

Ensure `Future` implementation of `ObserverWork` is polling its
`NetworkBridge`. Regression test for bug introduced in d9837d7dd and
fixed in 504b4e89e.

When polled, `NetworkBridge` forwards reputation change requests from
the `GossipValidator` to the underlying `dyn Network`. This test
triggers a reputation change by calling `GossipValidator::validate` with
an invalid gossip message. After polling the `ObserverWork` which should
poll the `NetworkBridge`, the reputation change should be forwarded to
the test network.

* Nits

Co-authored-by: Max Inden <mail@max-inden.de>
2020-01-31 19:12:03 +00:00
thiolliere a36a170c15 remove ambiguous into_iter (#4794) 2020-01-31 18:49:02 +01:00
Bastian Köcher 709a899f9d Cleanup transaction pool deps (#4782)
* Cleanup transaction pool deps

* Fix it properly

* Fix doc test
2020-01-31 12:40:32 +01:00
Cecile Tonglet 605f643eed CLI API refactoring and improvement (#4692)
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).

This touches a few important things:
 - the startup of the async task with tokei:
    This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
 - increased the version of structopt and tokei
 - no more use of structopt internal's API
 - less use of generics

Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want

Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
2020-01-30 11:40:08 +01:00
Tomasz Drwięga fd03f4e950 Fix warnings. (#4768) 2020-01-29 21:37:26 +01:00
Max Inden 504b4e89e8 client/finality-grandpa/src/observer: Poll NetworkBridge (#4766)
Patch d9837d7dd reintegrated the periodic neighbor packet worker, by
moving its logic into a `Future` implementation on `NetworkBridge` which
needs to be polled by its upper layer.

Polling by the upper layer was implemented within the `Future`
implementation of the `VoterWork` struct but not within the `Future`
implementation of the `ObserverWork` struct. This patch adds polling of
the `NetworkBridge` to the latter.
2020-01-29 18:42:21 +01:00
Gavin Wood a9100449dc RPC calls should execute with configured strategy (#4772) 2020-01-29 17:57:50 +01:00
Bastian Köcher 4c36143375 Add get_global for Sandbox (#4756)
* Add `get_global` for `Sandbox`

This pr adds `get_global` to retrieve a `global` variable from an
instantiated sandbox wasm blob.

* Bump `spec_version`

* Update primitives/wasm-interface/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* `get_global` -> `get_global_val`

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
2020-01-29 16:24:40 +01:00
Pierre Krieger ae1e9002d7 Order network dependencies alphabetically (#4763) 2020-01-29 15:32:45 +01:00
Sergei Pepyakin 2c90562818 Clean up indentation problems (#4762) 2020-01-29 13:27:23 +01:00
Benjamin Kampmann 3f9a05a0d3 clarify licensing (#4755)
* adding license fields to all crates

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-29 11:57:13 +01:00
Pierre Krieger b452867eb7 Add diagnostics to tasks (#4752) 2020-01-29 11:46:39 +01:00
Bastian Köcher 34c1c4b954 Don't run dev light client as validator (#4757)
A light client can not be a validator, so we should not set `authority
== true` in dev mode.
2020-01-29 11:52:54 +03:00
Edwin 9a6d9f0db5 RPC api for offchain storage (#4694)
* Rpc api for offchain storage

* Replace Vec<u8> to Bytes, replace Mutex to RwLock

* Remove pub

* Modify copyright year
2020-01-28 23:31:28 +01:00
Tomasz Drwięga e7df428274 Display has_api_with results if invalid version of Offchain Workers is detected. (#4749)
* Display error message when checking has_api.

* Cargo.lock.
2020-01-28 22:34:41 +01:00
Tomasz Drwięga e58dc6a293 Make sure destroy_state is called in case of errors. (#4745)
* Make sure to destroy state in case of error.

* Update client/src/call_executor.rs

* Update client/src/call_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-28 16:18:09 +01:00
Bastian Köcher 793a1eb053 WASM runtime switch to import memory (#4737)
* WASM runtime switch to import memory

Up to now runtimes have exported their memory. To unify it with
sandboxing, this pr switches runtimes to import memory as well.

From a functional perspective, exporting/importing memory makes no
difference to the runtime.

To provide backwards compatibility, WASM exported memory is still supported.

* Revert debug stuff

* Revert some stuff
2020-01-28 09:36:57 +01:00
Pierre Krieger 5c8743510e Upgrade to libp2p 0.15 (#4732)
* Fixes for libp2p 0.15

* More work

* Update libp2p

* Update to libp2p 0.15
2020-01-27 18:27:36 +01:00
Nikolay Volf ed3da9f903 Fix quadratic iterations over transaction pool (#4736)
* transaction pool changes

* service & network changes

* address review

* reduce future pool
2020-01-27 18:26:42 +01:00
Nikolay Volf 94a187f291 Transaction queue tests update (#4734)
* shuffle tests

* update tests

* inc_nonce -> increment_nonce
2020-01-27 04:22:04 -08:00
Seun LanLege 9cea98e950 Manual Seal (#4143)
* instant/manual seal

unbounded queues are evil

Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

add fork tests, docs, remove todos

moar docs

Update client/consensus/manual-seal/src/rpc.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

remove unbound generic, parameter, docs, deps, code style changes

Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

code style chnges

remove unused deps, remove dep renames, check if block is empty before importing, use ? for error propagation

fix tests

log errors for instant seal

use debug

code style changes, updated copyright dates

use txpool::Pool instead of BasicPool, code style changes

fixed tests

* fix tests

* requested changes from review

* check inherents len

* rebase
2020-01-27 10:59:40 +01:00
Sergei Pepyakin 2421576f91 Cursed implementation of allowing missing imports on wasmtime (#4730)
* Quick and dirty impl.

* Clean up

* Check the signatures.

* Fix tests.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Ignore non function members.

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-25 23:53:08 +01:00
Bastian Köcher fc99887de0 Introduces author_hasKey and author_hasSessionKeys rpc endpoints (#4720)
* Introduces `author_hasKey` and `author_hasSessionKeys` rpc endpoints

Both endpoints can be used to check if a key is present in the keystore.

- `hasKey` works on with an individual public key and key type. It
checks if a private key for the given combination exists in the
keystore.
- `hasSessionKeys` works with the full encoded session key blob stored
on-chain in `nextKeys`. This requires that the given blob can be decoded
by the runtime. It will return `true`, iff all public keys of the
session key exist in the storage.

Fixes: https://github.com/paritytech/substrate/issues/4696

* Update client/rpc-api/src/author/error.rs

Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Indentation

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-01-24 15:20:45 +01:00
Nikolay Volf 1614ce0996 Async/await in transaction-graph (#4645)
* async/await in tx graph

* review notes

* remove unused typedef
2020-01-24 05:22:39 -08:00
Ashley c2c429877e Switch GrandPa to std futures (replaces #3909) (#4612)
* Switch GrandPa to new futures

* Work on making tests work

* until_imported tests working again

* Work on switching tests to stable futures

* Modifications

* Re-add test as #[ignore]

* Don't ignore

* Add manual unpins

* Remove Header import

* Return concrete Sink type

* Switch to crates.io finality-grandpa version

* Remove use statement that slipped in

* Fix some nitpicks

* Remove unpin from i

* Fixed typo

* Move futures01 to dev-deps

* Fix nitpicks

* Update client/finality-grandpa/src/communication/mod.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* nitpicking

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-01-24 13:34:42 +01:00
Nikolay Volf 14e95f3398 Refactor tx-pool maintenance and other high-level api (#4629)
* Reduction.

* Reformation.

* add locked timer stuff

* fix issues and introduce full pool

* arrange together

* fix benches

* fix new_light

* Add revalidation test case

* review fixes

* review fixes

* use just ready future

* address review
2020-01-24 04:21:24 -08:00
Nikolay Volf 81004eabfd Refactor and test spec block rules (#4670)
* Refactor and test spec block rules

* address review

* Update client/src/block_rules.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-01-23 05:41:22 -08:00
Max Inden 597c0a6c4f client/finality-grandpa: Make round_communication use bounded channel (#4691)
* clinet/finality-grandpa: Make round_communication use bounded channel

`round_communication` returns a `Sink` and a `Stream` for outgoing and
incoming messages. The messages send into the `Sink` are forwarded down
to the network as well as send back into the `Stream` to ensure the node
processes its own messages.

So far, to send messages into the `Sink` back into the `Stream`, an
unbounded channel was used. This patch updates `round_communication` and
`OutgoingMessages` to use a bounded channel.

This is part of a greater effort to reduce the number of owners of
components within `finality-grandpa` and `network` as well as to reduce
the amount of unbounded channels. For details see d9837d7dd and
5f80929dc.

* client/finality-grandpa: Import futures03::future::ready at the top

* client/finality-grandpa: Make tests use compat of future 03

* client/finality-grandpa: Do not import ready into scope

Instead of importing futures03::future::ready into the scope, only
import futures::future03 into scope and call ready as furure03::ready.
2020-01-23 12:58:36 +01:00
Wei Tang 4b2e6a5be2 Default fork choice value and intermediates for block import parameters (#4652)
* consensus, pow: intermediate separation and fail

* Fix compiles

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Document what None means for `fork_choice` in block import params

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2020-01-23 00:03:38 -08:00