Commit Graph

106 Commits

Author SHA1 Message Date
Bastian Köcher 9ac30f7116 Rename task name to stick to the default naming scheme (#6768) 2020-07-30 14:19:34 +00:00
Bastian Köcher e7d8040af8 Update parity-scale-codec to prepare for breaking rustc release (#6746)
This updates parity-scale-codec{-derive} to prepare for a rustc release
that would otherwise break the derive implementation:
https://github.com/rust-lang/rust/pull/73084
2020-07-28 20:26:01 +00:00
Ashley 9220b646d2 Various small improvements to service construction. (#6738)
* Remove service components and add build_network, build_offchain_workers etc

* Improve transaction pool api

* Remove commented out line

* Add PartialComponents

* Add BuildNetworkParams, documentation

* Remove unused imports in tests

* Apply suggestions from code review

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Remove unused imports in node-bench

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-07-28 15:21:33 +02:00
Bastian Köcher 9310f15ac2 Name all the tasks! (#6726)
* Remove any implementation of `Spawn` or `Executor` from our task executors

* Fix compilation

* Rename `SpawnBlockingExecutor`

* Update primitives/core/src/traits.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix tests

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-26 12:56:17 +00:00
pscott 046fda914a Improve overall performance (#6699)
* Improve overall performance

* Clean up code

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-21 14:46:49 +02:00
Benjamin Kampmann 629a801b6a Fixup cargo unleash (#6689)
* Bumping to rc5

* Revert "Bumping to rc5"

This reverts commit 5a63a8230e5a12db1b4ce4ac62e4967c52291da7.

* Bumping for rc5, correctly this time
2020-07-21 10:26:47 +00:00
Ashley 234e7d0c3d Simplify a few chain components creation APIs related to the service (#6611)
* Simplify a few chain components creation APIs related to the service

* Fix basic-authorship doc tests

* Remove DefaultQueue

* Update client/service/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Move ExecutionExtensions comment around

* Remove unused BlakeTwo256

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-07-09 15:43:04 +02:00
Bastian Köcher 94cddee160 Improve transaction submission (#6599)
* Improve transaction submission

Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.

Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.

* Make sure we don't submit the same transaction twice from the network concurrently

* Remove added listener call

* Feedback

* Ignore banned on resubmit
2020-07-08 17:42:42 +02:00
Benjamin Kampmann f6ebd5f87d Releasing rc4 – Rhinoceros (#6515)
Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-06-25 23:18:43 +02:00
Arkadiy Paronyan 7b34438178 Validate encoding of extrinsics passed to runtime (#6442)
* Validate encoding of extrinsics passed to runtime

* Bump codec version explicitly
2020-06-19 19:27:16 +00:00
Sergei Shulepov 3ec4844616 Make NumberOrHex a common primitive. (#6321)
* Make NumberOrHex a common primitive.

* Update primitives/rpc/src/number.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-06-10 15:08:15 +00:00
Benjamin Kampmann 881072e590 Intent to release rc3 (#6290) 2020-06-08 23:29:52 +02:00
Hernando Castano 9bd23e7f01 Use Subscription Manager from jsonrpc-pubsub: The Sequel (#6254)
* Bump jsonrpc pubsub, core, http, and ws

Right now these are the packages which _need_ to be updated
so I can just the latest `jsonrpc-pubsub` code. Once a release
it cut upstream the rest of the dependencies should be updated
as well.

* Use jsonrpc-pubsub's SubscriptionManager

This places sc-rpc-api::Subscriptions

* Bump jsonrpc-core outside of sc-rpc-*

* Update client/rpc tests

Right now one of the `author` tests is failing, I
need to think a bit about how best to fix it.

* Remove Subscriptions manager

There's no need for this implementation since we're
using the one from `jsonrpc-pubsub` now

* Fix author RPC test

This test used to check for a numerial subscription ID,
whereas now it uses a string based ID which is the default
provided by `jsonrpc-pubsub`'s subscription manager.

* Remove unused NumericIdProvider

* Add missing bracket

Removed one too many with that last one, lol

* Bump `jsonrpc` to v14.2

There's an exception though. `jsonrpc-derive` cannot be bumped
past v14.0.5 just yet since it has a dependency on `quote` pinned
to v1.0.1. This means that at the moment it won't build on Substrate
since it's using v1.0.3.

* Track `jsonrpc-derive` master branch

* Bump `quote` version to v1.0.6

* Bump `jsonrpc-derive` to v14.2.1

This includes support for `quote` v1.0.6

* Use exact version for jsonrpc crates

Doing this to make sure any updates in jsonrpc don't
accidently trickle down to Polkadot.

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-06-08 17:47:09 +01:00
Ashley 84cdb02963 Fix the metered unbounded sender/recievers (#6246)
* Fix the metered unbounded sender/recievers

* Use a counter instead

* Update client/rpc/src/system/tests.rs

* Add an is_terminated check

* Add FusedStream impl

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-06-06 13:06:22 +02:00
Bastian Köcher d2846e2b9a Make transaction pool prune transactions only of canonical blocks (#6123)
* Make tx pool aware of retracted fork blocks

* Make it compile

* Update client/transaction-pool/src/lib.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Fix doc test

* Simplify the implementation

* Send tree route as arc to prevent heavy clones

* Switch to use `ExtrinsicHash` to make it more clear

* Fix benchmark

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-06-05 23:12:00 +02:00
André Silva 8f5a52fe1a Revert "Use Subscription Manager from jsonrpc-pubsub (#6208)" (#6252)
This reverts commit ea1eb4e57f.
2020-06-04 15:39:57 +01:00
Hernando Castano ea1eb4e57f Use Subscription Manager from jsonrpc-pubsub (#6208)
* Bump jsonrpc pubsub, core, http, and ws

Right now these are the packages which _need_ to be updated
so I can just the latest `jsonrpc-pubsub` code. Once a release
it cut upstream the rest of the dependencies should be updated
as well.

* Use jsonrpc-pubsub's SubscriptionManager

This places sc-rpc-api::Subscriptions

* Bump jsonrpc-core outside of sc-rpc-*

* Update client/rpc tests

Right now one of the `author` tests is failing, I
need to think a bit about how best to fix it.

* Remove Subscriptions manager

There's no need for this implementation since we're
using the one from `jsonrpc-pubsub` now

* Fix author RPC test

This test used to check for a numerial subscription ID,
whereas now it uses a string based ID which is the default
provided by `jsonrpc-pubsub`'s subscription manager.

* Remove unused NumericIdProvider

* Add missing bracket

Removed one too many with that last one, lol

* Bump `jsonrpc` to v14.2

There's an exception though. `jsonrpc-derive` cannot be bumped
past v14.0.5 just yet since it has a dependency on `quote` pinned
to v1.0.1. This means that at the moment it won't build on Substrate
since it's using v1.0.3.

* Track `jsonrpc-derive` master branch

* Bump `quote` version to v1.0.6

* Bump `jsonrpc-derive` to v14.2.1

This includes support for `quote` v1.0.6
2020-06-04 09:50:22 +01:00
Benjamin Kampmann b64f55089d Releasing rc2 (#6136) 2020-05-26 14:32:23 +02:00
Benjamin Kampmann fb32ac8c51 Tagging as rc1 2020-05-25 18:30:48 +02:00
Pierre Krieger 717fa95bc7 Restore the empty line after the license (#6088) 2020-05-20 13:08:27 +00:00
Nikolay Volf d5411969ac fix whitespace (#6062) 2020-05-18 13:51:28 +02:00
Benjamin Kampmann 7c565085e9 resetting to -dev (#6050) 2020-05-16 19:03:04 +02:00
Benjamin Kampmann d7463d348f Releasing Alpha.8 (#6048) 2020-05-15 21:56:56 +02:00
Benjamin Kampmann be8c96adec Relax substrate licensing scheme (#5947) 2020-05-15 13:21:56 +02:00
Bastian Köcher 73389df860 Fix RPC tests for machines with a high number of cores (#6021) 2020-05-14 11:08:39 +02:00
Benjamin Kampmann 3860999ea3 post release dev reset (#5911) 2020-05-05 22:08:31 +01:00
Benjamin Kampmann 45886bd197 Release of Alpha.7 (#5904) 2020-05-05 20:50:46 +02:00
Seun Lanlege 4fa5941f44 Move sc-client into sc-service (#5502)
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api

* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa

* drop sc-service from sc-rpc

* drop sc-service from sc-consensus-aura

* drop sc-client from manual-seal and babe

* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli

* drop sc-client from bin/node and bin/node-template

* drop sc-client

* fix tests

* remove check -p sc-client from gitlab.yml

* fix warnings

* fixes ui test

* fix light client tests

* adds associated Client type to AbstractService

* adds UsageProvider to Client

* fixed ui test, again

* tried and failed to get node-cli to compile for wasm

* thanks to tomaka for helping me get node-cli to compile for wasmm

* ui test pls pas 🙏🏾

* all tests passing 🪄

* no_run documentation code

* rm -f documentation code

* ClientProvider

* fix mega trait

* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client

* adds license to sc-consensus

Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-04-28 11:59:31 +00:00
Wei Tang 969720c2ad babe: secondary blocks with VRF (#5501)
* babe: secondary blocks with VRF

* Fix node runtime compile

* Fix test-utils runtime interface

* Fix babe tests

* typo: v == 2

* babe: support online configuration upgrades

* Fix rpc tests

* Fix runtime version tests

* Switch to use NextConfigDescriptor instead of changing runtime interface

* Fix tests

* epoch-changes: map function that allows converting with different epoch types

* Add migration script for the epoch config change

* Fix docs for PrimaryAndSecondaryVRFSlots

* Add docs of `SecondaryVRF` in babe crate

* babe-primitives: Secondary -> SecondaryPlain

* babe-client: Secondary -> SecondaryPlain

* Fix migration tests

* test-utils-runtime: Secondary -> SecondaryPlain

* Fix missing name change in test-utils-runtime

* Fix migration: Epoch should be EpochV0

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* Fix new epochChanges version

* Fix babe-primitives naming changes

* Fix merge issues in babe-client

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-04-24 16:03:03 +01:00
Yuanchao Sun 52d0173ec3 Add RPC function state_getProof, resolves #1110 (#5649)
* Add RPC function state_getProof, resolves #1110

* Apply suggestions from code review

* Update client/rpc/src/state/state_full.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock

* Make block hash optional

* Wrap StorageProof as Bytes

* Add struct ReadProof

* Fix typo

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-20 21:09:55 +02:00
cheme 4ffcf98d8d Child trie api changes BREAKING (#4857)
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
2020-04-20 15:21:22 +02:00
Igor Matuszewski 4b1f7d187f Allow to expose a subset of unsafe RPCs (#5233)
* sc-cli: Use type-safe constructors for RPC/Prometheus interfaces

* service: Simplify rpc handler creation

Could probably be further simplifies once [this][commit] lands.

[commit]: https://github.com/paritytech/jsonrpc/commit/20485387ed06a48f1a70bf4d609a7cde6cf0accf

* service: Streamline some HTTP & WS server start logic

* client: Introduce a simple RPC policy mechanism

* rpc/system: Check unsafe RPCs

* rpc/offchain: Check unsafe RPCs

* rpc/author: Check unsafe RPCs
2020-04-20 11:03:58 +02:00
Gavin Wood fd2cb9ca83 Transaction versioning in the RuntimeVersion (#5582)
* Add transaction_version

* Semantic versioning for runtimes

* Move new field to bottom

* Versioning

* Runtime versioning stuff.

* Fix test

* Adds tests and fixes bugs

* Bump runtime

Co-authored-by: Bastian Köcher <git@kchr.de>
2020-04-17 12:10:31 +02:00
Nikolay Volf fea626ca84 Add prometheus registry to transaction pool, with couple of initial metrics (#5657)
* make new contructor

* add metrics to txpool

* fix review

* fix doc comment

* change to counters

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <mail@max-inden.de>

* use dedicated wrapper

Co-authored-by: Max Inden <mail@max-inden.de>
2020-04-17 11:02:45 +02:00
Pierre Krieger 95ee37d242 Add alternative RPC methods to system_networkState (#5643)
* Add alternatives to system_networkState

* Fix tests

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-04-16 16:36:14 +01:00
Benjamin Kampmann 3426d662f7 Switch pre-release version to dev (#5637) 2020-04-15 13:18:08 +02:00
Benjamin Kampmann 51f9bb3c0e Prepping release of alpha.6 (#5629)
* bumping version

* cargo update

* adding changelog
2020-04-14 21:41:49 +02:00
Bastian Köcher 7cdfaff12b Add new RPC method to get the chain type (#5576)
* Add new RPC method to get the chain type

This adds a new RPC method to get the chain type of the running chain.
The chain type needs to be specified in the chain spec. This should make
it easier for tools/UI to display extra information without needing to
rely on parsing the chain name.

* Update client/rpc-api/src/system/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Primitive crate

* Feedback

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-04-08 20:41:51 +02:00
Benjamin Kampmann 247822bb33 Additional Metrics collected and exposed via prometheus (#5414)
This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:

- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)

refs #4679

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
2020-04-04 15:13:35 +02:00
Pierre Krieger 8c03a4fcef Split the Roles in three types (#5520)
* Split the Roles bitfield in three

* Forgot to include some changes

* Fix cli test

* More test fixes

* Oh God, merging master broke other tests

* Didn't run the doctests

* Address review

* I'm trying to fix the build blindly because it's taking a good hour to compile on my machine

* Address some review

* Also update the peerset's API to make sense

* Fix peerset tests

* Fix browser node

* client: distinguish between local and network authority

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-04-03 19:08:14 +02:00
Rakan Alhneiti e17a23e907 Introduce sign_with method in keystore (#4925)
* Add KEY_KIND_ID to the public trait

This change is being introduced for the purpose of identifying a public
key with it's identifier and algorithm "kind".

* Use `sign_with` as implemented in BareCryptoStore

* Implement `sign_with` in sc_keystore

* Fix inconsistencies, use *_KIND_ID in sp_core testing

* Rename KeyKindId to CryptoTypeId

* Remove pair-returning functions from BareCryptoStore trait

* Define CryptoTypeId in app-crypto macros

* Add functions to get keys supported by keystore

* Fix sign_with signature to include CryptoTypePublicPair

* Add `sign_with_any` and `sign_with_all`

* Use keystore.sign_with in auth_discovery

* Rename get_supported_keys -> supported_keys

* Added headers to function docstrings

* Use chain instead of extending a temp vector

* Fixed some code formatting

* Restrict size of CryptoTypeId

This is to be able to use Encode/Decode derives and the overcome having
the size being unknown at compile-time.

* Implement sign_with in the trait itself

* Remove whitespace

* Use key_type also as a CryptoTypeId in app_crypto macros

* Rename `get_keys` to `keys` in BareCryptoStore

* Remove usage of key_pair funcs in tests

* Adjust docstring for *_CYPTO_ID constants

* Fix failures

* Simplify mapping on keys

* Remove one let

* Fixed typo

* PR feedback

* remove whitespace

* Zip keys and signatures

* Use into_iter & remove cloned

* Pass index to MissingSignature

* Use typed errors instead of strings for BareCryptoStore

* Implement Debug for trait error

* Use hashsets for better performance for supported_keys

* Make sure keys are inserted into the keystore

* Make sign_with_all return type consistent with `sign_with`

* Rename Error to BareCryptoStoreError

* Rename CRYPT_TYPE_ID -> CRYPTO_ID

* Remove unnecessary CRYPTO_ID declaration in Public trait

* Convert pub key to CryptoTypePublicPair

* Fix use

* Fix code style

* Implement From on CryptoTypePublicPair in app_crypto macros

* Change CryptoTypePublicPair to a struct

* Implement Display on CryptoTypePublicPair

* Pass CryptoTypePublicPair to MissingSignature error

* Adjust docs according to function signature

* Unify keys implementation

* Fix RPC author tests

* Fix stackoverflow

* Tabify spaces

* Pass KeyTypeId to error for easier debugging

* Fix asserts

* Use ToHex to format public key

* Use constants from sp_core

* Rename testing KeyTypeId constants

* Please compiler

* Restore KeyTypeId names

apparently, they're not only used in tests

* Use BareCryptoStoreError instead of String

* Document return value

* Fix borrow check

* Convert to hashset internally

* WIP - iter_keys

* Return raw_public_keys

* Address PR feedback

* Address PR Feedback

* Fix hexdisplay import error

* Update primitives/core/src/traits.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-30 13:18:59 +02:00
pscott 408455f8bc Build for only one target for docs.rs (#5427)
* Add docs.rs metadata to all cargo.toml files

* Remove docs.rs metada in substrate's cargo.toml
2020-03-30 09:46:30 +02:00
Bastian Köcher 1242a7df3c Upgrade parity-scale-codec to 1.3.0 (#5443) 2020-03-28 22:16:05 +01:00
Seun Lanlege ab0f7d213a Adds state_queryStorageAt (#5362)
* adds state_queryStorageAt

* make at param for query_storage_at optional

* Update client/rpc/src/state/state_full.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* adds query_storage_at to StateBackend

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-03-26 16:03:34 +01:00
André Silva 3a136bdcf9 Internal version dependencies cleanup (#5401)
* always depend on latest alpha version

* integrate arithmetic fuzzer into root workspace
2020-03-25 15:02:39 +01:00
Tomasz Drwięga 04ccb179e9 Pass transaction source to validate_transaction (#5366)
* WiP

* Support source in the runtime API.

* Finish implementation in txpool.

* Fix warning.

* Fix tests.

* Apply suggestions from code review

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Extra changes.

* Fix test and benches.

* fix test

* Fix test & benches again.

* Fix tests.

* Update bumpalo

* Fix doc test.

* Fix doctest.

* Fix doctest.

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-03-25 14:09:23 +01:00
Benjamin Kampmann ba06d5ef64 Releasing 2.0.0-alpha.5 (#5340)
* bump pre-version

* Changelog alpha.4->alpha.5

* repo + homepage to sp-consensus-vrf

* Add default rocksdb feature to `frame-benchmarking-cli` (#5367)

* Add default rocksdb feature

* Update Cargo.toml

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-03-24 15:08:02 +01:00
Stanislav Tkach 4414d00082 Get rid of rustc_hex in favor of hex (#5370)
* Get rid of rustc_hex in favor of hex

* Use HexDisplay
2020-03-24 11:48:23 +01:00
Benjamin Kampmann 25e25a83b9 Releasing 2.0.0-alpha.4 (#5279)
* Version bump

* Split generate_changelog.sh into separate script

Can be run in the format `generate_changelog.sh $previous_version $version`.

* remove early exit from publish_draft_release.sh

* adding changelog

* ci: change last_github_release to also find pre-releases

Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-03-18 14:37:21 +01:00
Bastian Köcher 26fa6e3c76 Move runtime upgrade to frame-executive (#5197)
* Move runtime upgrade to `frame-executive`

Instead of storing the runtime upgraded in the space of `frame-system`,
this moves it to `frame-executive`. We also start storing the
`spec_version` and `impl_version` the last runtime upgrade was applied.

This scheme has multiple advantages:
- We don't need to make sure that runtime upgrade was set.
- This will work in the future when we want to test a runtime upgrade,
because the `on_runtime_upgrade` will be executed automatically when the
runtime changes.

* Move `LAST_RUNTIME_UPGRADE` key to `frame-executive`

* Add some more documentation

* Make sure `spec_version` always increases between runtime upgrades

* Fix test

* Upgrade `parity-multiaddr` to fix compilation

* Also check if the `spec_name` changed

* Remove `RuntimeUpgraded` storage entry

* Don't run on genesis
2020-03-15 20:28:57 +01:00