Commit Graph

277 Commits

Author SHA1 Message Date
Shawn Tabrizi e5a7fcc8ea Create Benchmarking Setup for Identity Pallet #4695 (#4818)
* Starting

* closer

* Compiles!

* comments

* Create seperate mock

* Remove changes to test env

* Fix step calculation

* Add host function

* Add runtime api

* compiles

* Update to use offchain timestamp

* Gives a result

* added some CLI wip

* make generic

* Update instance

* Remove CLI stuff

* Remove last cli stuff

* undo more changes

* Update benchmarks

* Update Cargo.lock

* remove test

* Move loop out of runtime

* Benchmarking externalities

* Benchmarking state

* Implemented commit

* Make CLI work, move loop back into runtime

* Wipe resets to genesis

* Speedup benchmarks

* Use enum to select extrinsic within pallet

* CLI controls which module and extrinsic to call

* Select a pallet with cli

* Add steps and repeats to cli

* Output as CSV format

* Introduce benchmark pallet

* Append bench

* Use Results

* fix merge

* Clear Identity benchmark

* Bench request judgment and cancel request

* Add final benchmarks

* Fix CSV output

* Start cleaning up for PR

* Bump numbers in `wasmtime` integration tests.

* More docs

* Add rockdb feature to bench

* Fix formatting issues

* Add test feature to bench

* Add test feature to bench

* Add rocksdb feature flag

* Update bench.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
2020-02-10 10:23:08 +01:00
Nikolay Volf 147155359f Alter usage collection a bit (#4863)
* alter usage collection a bit

* replace with mutex
2020-02-09 13:11:22 +01:00
Bastian Köcher ec562fe937 Fix CLI setup again (#4851)
* Fix CLI setup again

We need to set `config_dir` and `database_path` for almost every
command.
This fixes `purge-chain` and also adds a test to make sure we don't
break it again.

* Adds missing test files

* Split methods
2020-02-07 15:00:51 +01:00
Cecile Tonglet 60bd49ff0d Update parity-multiaddr dependency (#4852) 2020-02-07 13:22:06 +01:00
Bastian Köcher 9277a53052 Move tracing cli args to ImportParams (#4850) 2020-02-07 13:21:46 +01:00
Wei Tang bf437cda8e pow: re-add support for algorithms where only linear verification is possible (#4843)
* pow: re-add support for algorithms where only linear verification is possible

* Remove unused generic parameters

* Clone impl for PowBlockImport
2020-02-07 11:58:31 +01:00
Nikolay Volf 41967186e4 Subsystems memory tracking: 1. Transaction pool (#4822)
* update sp-runtime

* total update

* usage informant

* update to crates.io version

* update Cargo.lock

* update dummy update

* fix todo

* cleanup

* avoid custom impl

* Update client/transaction-pool/graph/src/future.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* remove another custom impl

* remove another custom impl

* add kb in report

* update Cargo.lock

* review suggestions

* --amend

* --amend

* bump parity-util-mem to 0.5.0

* bumps

* update macro and versions

* add to grafana

* naming

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-02-07 11:53:11 +01:00
Marcio Diaz 41644c2ae8 Enable trace timings logs for transaction factory (#4845)
* Enable trace timings logs for transaction factory.
2020-02-07 08:12:19 +01:00
Nikolay Volf 8c373916a4 easy with threads (#4848) 2020-02-06 23:32:48 +01:00
Wei Tang 14b0c9a746 babe: pass epoch data via intermediates (#4807)
* babe: pass epoch data via intermediates

* Switch to use Box<dyn Any> for intermediates

* Set intermediate.epoch to be Option

* Fix proposer should put out an empty intermediate

* Remove unnecessary encode/decode

* Add EpochData to block_import_params in slot worker

* Fix aura compile

* Fix integration test
2020-02-06 17:52:30 +00:00
Wei Tang 4df27e760e Refactor epoch changes to a separate crate (#4785)
* Init epoch changes module

* Initial integration of new epoch changes module for BABE

* Fix all initial compile errors

* rename: digest -> digests

* Fix babe tests

* Bump impl_version

* Fix more test issues

* Remove test flag for tree

It unfortunately won't work for multiple crates.

* Update cargo lock

* Fix duplicate parking_lot version

* Add missing license header
2020-02-06 16:48:38 +01:00
Cecile Tonglet be075893b5 CLI improvements & fixes (#4812)
These are a few changes I missed during the refactoring.

1. Initialization issue and boilerplate

    Most importantly: part of the `Configuration` initialization was done in `sc_cli::init`. This means the user can not benefit from this initialization boilerplate if they have multiple `Configuration` since `sc_cli::init` can only be called once.

2. Boilerplate for `VersionInfo` and `Configuration`

    I'm also answering to the critic of @bkchr on the initialization using version: https://github.com/paritytech/substrate/pull/4692/files/bea809d4c14a2ede953227ac885e3b3f9771c548#r372047238 This will allow initializing a `Configuration` and provide the version by default.

3. Loading the `chain_spec` explicitly

    In the past it was done automatically but in some cases we want to delay this. I moved the code to `Configuration.load_spec()` so it can be called later on. `chain_spec` can also be written directly to the `Configuration` without using this `load_spec` helper.

4. [deleted]

5. Fixing issue that prevents the user to override the port

    In the refactoring I introduced a bug by mistake that could potentially prevent the CLI user to override the ports if defaults where provided for these ports (only on cumulus).

6. Change task_executor from Box to Arc

    This is useful for cumulus where we have 2 nodes with 2 separate Configuration that need to spawn tasks to the same runtime.

7. Renamed TasksExecutorRequired to TaskExecutor

    For consistency.

This is related to https://github.com/paritytech/cumulus/issues/24

This is the continuation (and hopefully the end of) #4692
2020-02-06 15:46:49 +01:00
Marcio Diaz f891342f20 Fix broken factory by adding keystore back. (#4840) 2020-02-06 14:48:19 +01:00
Arkadiy Paronyan 3c79e6d03c Fixed a few warnings (#4841) 2020-02-06 14:06:53 +01:00
Nikolay Volf 6148c7ce15 do join_all (#4832) 2020-02-06 13:07:52 +01:00
André Silva d53bb5ec50 grandpa: bump version to v0.11.1 (#4813) 2020-02-05 21:47:11 +01:00
Sergei Pepyakin 1af9e4f3ee executor: Simplify the SandboxCapabilities interface (#4825)
* Don't require `store` and `store_mut` in `SandboxCapabilities`.

* Simplify the sandbox a bit
2020-02-05 18:20:25 +01:00
Gavin Wood 9202cd87e0 Additional RPC for dumping all main storage key pairs under a prefix (#4803)
* Merge branch 'gav-split-balanecs-vesting' into gav-upsub

# Conflicts:
#	Cargo.lock
#	cli/Cargo.toml
#	collator/Cargo.toml
#	primitives/Cargo.toml
#	runtime/common/Cargo.toml
#	runtime/common/src/claims.rs
#	runtime/kusama/Cargo.toml
#	runtime/polkadot/Cargo.toml
#	service/Cargo.toml

* Update client/src/client.rs

* Update client/src/client.rs

* Fix merge conflict

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-02-05 17:27:50 +01:00
Pierre Krieger 54c10893ab Print an error if listener is closed (#4830)
* Print an error if listener is closed

* Oops, forgot to commit this
2020-02-05 17:25:14 +01:00
Pierre Krieger a3439999cf Remove support for secio (#4831) 2020-02-05 16:56:49 +01:00
Pierre Krieger 39c037ab22 Fix missing overrides of NetworkBehaviour (#4829) 2020-02-05 11:56:59 +01:00
Demi Obenour 0ab013bddb Bump parking_lot (#4817)
Hopefully this can be merged if the tests pass.
2020-02-04 16:56:50 +01:00
Xiliang Chen febb6e29b2 add state_getKeysPaged (#4718)
* add storage_getNextKey

* RPC storage_getKeysPages

* respect count

* use iterator

* improve

* add tests

* improve

* add doc comments

* Make prefix optional

* update error

* improve
2020-02-04 10:44:40 +01:00
Wei Tang bff1f4a18d block_import: switch to Box<dyn Any> for intermediates representation (#4809)
* block_import: switch to Box<dyn Any> for intermediates representation

* Use Cow and return Error instead of Option

* Remove unused error

* Distinguish NoIntermediate/InvalidIntermediate
2020-02-03 09:55:09 +01:00
Pierre Krieger b6e8fba179 Remove warning about connecting to non-Substrate nodes (#4797) 2020-02-01 14:02:10 +01:00
Nikolay Volf 8a3caaeb68 update prost-build (#4789) 2020-02-01 12:33:13 +01:00
Nikolay Volf 63ad759c69 Increase transaction pool default limits (#4792)
* increase limits, improve logging

* format in kB
2020-02-01 00:27:53 -08:00
Gavin Wood adde404e41 client/finality-grandpa: Add regression test for observer polling network (was #4778) (#4795)
* client/finality-grandpa: Add regression test observer polling network

Ensure `Future` implementation of `ObserverWork` is polling its
`NetworkBridge`. Regression test for bug introduced in d9837d7dd and
fixed in 504b4e89e.

When polled, `NetworkBridge` forwards reputation change requests from
the `GossipValidator` to the underlying `dyn Network`. This test
triggers a reputation change by calling `GossipValidator::validate` with
an invalid gossip message. After polling the `ObserverWork` which should
poll the `NetworkBridge`, the reputation change should be forwarded to
the test network.

* Nits

Co-authored-by: Max Inden <mail@max-inden.de>
2020-01-31 19:12:03 +00:00
thiolliere a36a170c15 remove ambiguous into_iter (#4794) 2020-01-31 18:49:02 +01:00
Bastian Köcher 709a899f9d Cleanup transaction pool deps (#4782)
* Cleanup transaction pool deps

* Fix it properly

* Fix doc test
2020-01-31 12:40:32 +01:00
Cecile Tonglet 605f643eed CLI API refactoring and improvement (#4692)
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).

This touches a few important things:
 - the startup of the async task with tokei:
    This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
 - increased the version of structopt and tokei
 - no more use of structopt internal's API
 - less use of generics

Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want

Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
2020-01-30 11:40:08 +01:00
Tomasz Drwięga fd03f4e950 Fix warnings. (#4768) 2020-01-29 21:37:26 +01:00
Max Inden 504b4e89e8 client/finality-grandpa/src/observer: Poll NetworkBridge (#4766)
Patch d9837d7dd reintegrated the periodic neighbor packet worker, by
moving its logic into a `Future` implementation on `NetworkBridge` which
needs to be polled by its upper layer.

Polling by the upper layer was implemented within the `Future`
implementation of the `VoterWork` struct but not within the `Future`
implementation of the `ObserverWork` struct. This patch adds polling of
the `NetworkBridge` to the latter.
2020-01-29 18:42:21 +01:00
Gavin Wood a9100449dc RPC calls should execute with configured strategy (#4772) 2020-01-29 17:57:50 +01:00
Bastian Köcher 4c36143375 Add get_global for Sandbox (#4756)
* Add `get_global` for `Sandbox`

This pr adds `get_global` to retrieve a `global` variable from an
instantiated sandbox wasm blob.

* Bump `spec_version`

* Update primitives/wasm-interface/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* `get_global` -> `get_global_val`

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
2020-01-29 16:24:40 +01:00
Pierre Krieger ae1e9002d7 Order network dependencies alphabetically (#4763) 2020-01-29 15:32:45 +01:00
Sergei Pepyakin 2c90562818 Clean up indentation problems (#4762) 2020-01-29 13:27:23 +01:00
Benjamin Kampmann 3f9a05a0d3 clarify licensing (#4755)
* adding license fields to all crates

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-29 11:57:13 +01:00
Pierre Krieger b452867eb7 Add diagnostics to tasks (#4752) 2020-01-29 11:46:39 +01:00
Bastian Köcher 34c1c4b954 Don't run dev light client as validator (#4757)
A light client can not be a validator, so we should not set `authority
== true` in dev mode.
2020-01-29 11:52:54 +03:00
Edwin 9a6d9f0db5 RPC api for offchain storage (#4694)
* Rpc api for offchain storage

* Replace Vec<u8> to Bytes, replace Mutex to RwLock

* Remove pub

* Modify copyright year
2020-01-28 23:31:28 +01:00
Tomasz Drwięga e7df428274 Display has_api_with results if invalid version of Offchain Workers is detected. (#4749)
* Display error message when checking has_api.

* Cargo.lock.
2020-01-28 22:34:41 +01:00
Tomasz Drwięga e58dc6a293 Make sure destroy_state is called in case of errors. (#4745)
* Make sure to destroy state in case of error.

* Update client/src/call_executor.rs

* Update client/src/call_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-28 16:18:09 +01:00
Bastian Köcher 793a1eb053 WASM runtime switch to import memory (#4737)
* WASM runtime switch to import memory

Up to now runtimes have exported their memory. To unify it with
sandboxing, this pr switches runtimes to import memory as well.

From a functional perspective, exporting/importing memory makes no
difference to the runtime.

To provide backwards compatibility, WASM exported memory is still supported.

* Revert debug stuff

* Revert some stuff
2020-01-28 09:36:57 +01:00
Pierre Krieger 5c8743510e Upgrade to libp2p 0.15 (#4732)
* Fixes for libp2p 0.15

* More work

* Update libp2p

* Update to libp2p 0.15
2020-01-27 18:27:36 +01:00
Nikolay Volf ed3da9f903 Fix quadratic iterations over transaction pool (#4736)
* transaction pool changes

* service & network changes

* address review

* reduce future pool
2020-01-27 18:26:42 +01:00
Nikolay Volf 94a187f291 Transaction queue tests update (#4734)
* shuffle tests

* update tests

* inc_nonce -> increment_nonce
2020-01-27 04:22:04 -08:00
Seun LanLege 9cea98e950 Manual Seal (#4143)
* instant/manual seal

unbounded queues are evil

Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

add fork tests, docs, remove todos

moar docs

Update client/consensus/manual-seal/src/rpc.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

remove unbound generic, parameter, docs, deps, code style changes

Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

code style chnges

remove unused deps, remove dep renames, check if block is empty before importing, use ? for error propagation

fix tests

log errors for instant seal

use debug

code style changes, updated copyright dates

use txpool::Pool instead of BasicPool, code style changes

fixed tests

* fix tests

* requested changes from review

* check inherents len

* rebase
2020-01-27 10:59:40 +01:00
Sergei Pepyakin 2421576f91 Cursed implementation of allowing missing imports on wasmtime (#4730)
* Quick and dirty impl.

* Clean up

* Check the signatures.

* Fix tests.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Ignore non function members.

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-01-25 23:53:08 +01:00
Bastian Köcher fc99887de0 Introduces author_hasKey and author_hasSessionKeys rpc endpoints (#4720)
* Introduces `author_hasKey` and `author_hasSessionKeys` rpc endpoints

Both endpoints can be used to check if a key is present in the keystore.

- `hasKey` works on with an individual public key and key type. It
checks if a private key for the given combination exists in the
keystore.
- `hasSessionKeys` works with the full encoded session key blob stored
on-chain in `nextKeys`. This requires that the given blob can be decoded
by the runtime. It will return `true`, iff all public keys of the
session key exist in the storage.

Fixes: https://github.com/paritytech/substrate/issues/4696

* Update client/rpc-api/src/author/error.rs

Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Indentation

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-01-24 15:20:45 +01:00