Commit Graph

446 Commits

Author SHA1 Message Date
Max Inden c3c8552aaa src/service/src/builder: Fix memory metric exposed in bytes not KiB (#5459)
The library `sysinfo` exposes process memory as kibibytes and not bytes,
thus the value needs to be multiplied by 1024 to comply with the metric
name and the Prometheus base units [1].

[1] https://prometheus.io/docs/practices/naming/#base-units
2020-03-30 15:48:59 +02:00
Pierre Krieger dbba4f8929 Fix tried to send handshake twice (#5413)
* Fix tried to send handshake twice

* Fix wrong boolean

* Change to debug
2020-03-30 13:33:46 +02:00
Rakan Alhneiti e17a23e907 Introduce sign_with method in keystore (#4925)
* Add KEY_KIND_ID to the public trait

This change is being introduced for the purpose of identifying a public
key with it's identifier and algorithm "kind".

* Use `sign_with` as implemented in BareCryptoStore

* Implement `sign_with` in sc_keystore

* Fix inconsistencies, use *_KIND_ID in sp_core testing

* Rename KeyKindId to CryptoTypeId

* Remove pair-returning functions from BareCryptoStore trait

* Define CryptoTypeId in app-crypto macros

* Add functions to get keys supported by keystore

* Fix sign_with signature to include CryptoTypePublicPair

* Add `sign_with_any` and `sign_with_all`

* Use keystore.sign_with in auth_discovery

* Rename get_supported_keys -> supported_keys

* Added headers to function docstrings

* Use chain instead of extending a temp vector

* Fixed some code formatting

* Restrict size of CryptoTypeId

This is to be able to use Encode/Decode derives and the overcome having
the size being unknown at compile-time.

* Implement sign_with in the trait itself

* Remove whitespace

* Use key_type also as a CryptoTypeId in app_crypto macros

* Rename `get_keys` to `keys` in BareCryptoStore

* Remove usage of key_pair funcs in tests

* Adjust docstring for *_CYPTO_ID constants

* Fix failures

* Simplify mapping on keys

* Remove one let

* Fixed typo

* PR feedback

* remove whitespace

* Zip keys and signatures

* Use into_iter & remove cloned

* Pass index to MissingSignature

* Use typed errors instead of strings for BareCryptoStore

* Implement Debug for trait error

* Use hashsets for better performance for supported_keys

* Make sure keys are inserted into the keystore

* Make sign_with_all return type consistent with `sign_with`

* Rename Error to BareCryptoStoreError

* Rename CRYPT_TYPE_ID -> CRYPTO_ID

* Remove unnecessary CRYPTO_ID declaration in Public trait

* Convert pub key to CryptoTypePublicPair

* Fix use

* Fix code style

* Implement From on CryptoTypePublicPair in app_crypto macros

* Change CryptoTypePublicPair to a struct

* Implement Display on CryptoTypePublicPair

* Pass CryptoTypePublicPair to MissingSignature error

* Adjust docs according to function signature

* Unify keys implementation

* Fix RPC author tests

* Fix stackoverflow

* Tabify spaces

* Pass KeyTypeId to error for easier debugging

* Fix asserts

* Use ToHex to format public key

* Use constants from sp_core

* Rename testing KeyTypeId constants

* Please compiler

* Restore KeyTypeId names

apparently, they're not only used in tests

* Use BareCryptoStoreError instead of String

* Document return value

* Fix borrow check

* Convert to hashset internally

* WIP - iter_keys

* Return raw_public_keys

* Address PR feedback

* Address PR Feedback

* Fix hexdisplay import error

* Update primitives/core/src/traits.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-30 13:18:59 +02:00
Pierre Krieger 462eaa3f41 Make transactions and block announces use notifications substre… (#5360)
* Make transactions and block announces use notifications

* Add documentation
2020-03-30 10:00:34 +02:00
pscott 408455f8bc Build for only one target for docs.rs (#5427)
* Add docs.rs metadata to all cargo.toml files

* Remove docs.rs metada in substrate's cargo.toml
2020-03-30 09:46:30 +02:00
Bastian Köcher ab0366a076 Add method to sc-cli to reset the SIGPIPE signal handler (#5447)
The signal handler will be reset to the default system one. This means
the program will be exited instead of panicking. This is required to
support piping in the console.
2020-03-29 14:12:18 +02:00
Nikolay Volf d7ffef43ce don't use delays in tests (#5404) 2020-03-29 10:40:00 +02:00
Bastian Köcher 1242a7df3c Upgrade parity-scale-codec to 1.3.0 (#5443) 2020-03-28 22:16:05 +01:00
Nikolay Volf eed0045a0c update futures dependency (#5426) 2020-03-27 12:35:48 +01:00
Gavin Wood acab5f0570 Add emojis (#5420) 2020-03-27 10:41:31 +01:00
Seun Lanlege ab0f7d213a Adds state_queryStorageAt (#5362)
* adds state_queryStorageAt

* make at param for query_storage_at optional

* Update client/rpc/src/state/state_full.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* adds query_storage_at to StateBackend

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-03-26 16:03:34 +01:00
Max Inden 2a67e6c437 client/finality-grandpa/src/until_imported: Refactor schedule_wait (#5386)
* client/finality-grandpa/src/until_imported: Refactor schedule_wait

Previously `BlockUntilImported::schedule_wait` took two closures, one to
report ready items and one to report items to await. None of the implementors of
`BlockUntilImported` call both closures. From a symantic perspective it
would as well not make sense to both await something and state that it
is ready.

Instead with this commit `BlockUntilImported::schedule_wait` simply
returns whether the given item needs waiting or is ready to be passed
on.

This reduces complexity by:

- Removing side effects through the two closures.

- Reducing borrowing given that `UntilImported` `ready` and `pending`
don't need to be borrowed from within the two closures.

- Removes the need for trait bounds for the two closures.

* client/finality-grandpa/src/until_imported: Fix comments

Co-Authored-By: André Silva <andre.beat@gmail.com>

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-03-26 13:49:45 +01:00
Pierre Krieger fdbbf14fa4 Improve sc-network's documentation for network protocols (#5364)
* Improve sc-network's documentation for network protocols

* Add note about protocol id

* Apply suggestions from code review

Co-Authored-By: Max Inden <mail@max-inden.de>

Co-authored-by: Max Inden <mail@max-inden.de>
2020-03-26 12:26:30 +01:00
Arkadiy Paronyan d474864d67 Allow syncing to peers with finalized common block (#5408)
* Allow syncing to peers with finalized common block

* Added test
2020-03-26 09:13:09 +01:00
pscott 73c99eec84 TelemetryEndpoints must be valid MutliAddr URL (#5069)
* Check for url validity when creating TelemetryEndpoints

* Update code that used TelemetryEndpoints::new()

* Update commennts that referred to TelemetryEndpoints::new()

* Add tests for telemetry

* Fix typo and fix code in docs

* Return error on failing to override telemetry

* Use expect instead of suppressing errors on must-be-valid telemetry endpoints

* Update telemetry unit tests to use expect instead of unwrap

* Implement custom deserializer for TelemetryEndpoints

* Fix typo

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-26 09:02:38 +01:00
Max Inden 77446d3b50 client/finality-grandpa/src/until_imported: Refactor BlockGlobalMessage (#5390)
* client/finality-grandpa/src/until_imported: Refactor BlockGlobalMessage

`BlockGlobalMessage` owns an `inner` which contains (1) a count for the
amount of outstanding blocks to be waited on and (2) the message itself.

Given that both is already wrapped in an `Arc` there is no need to keep
track of the outstanding blocks, given that it simply corresponds to the
amount of strong reference counts on the `Arc` itself.

This commit removes the atomic counter within `inner` and piggy backs on
the `Arc` reference counter instead.

* client/finality-grandpa/src/until_imported: Remove useless match

* client/finality-grandpa/src/until_imported.rs: Remove unused var

Co-Authored-By: André Silva <andre.beat@gmail.com>

* client/finality-grandpa/src/until_imported: Address comment suggestion

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-03-25 19:45:04 +01:00
Joshy Orndorff 0505e93733 Remove package aliases from node template (#5389)
* delias outer node template

* dealias pallet template

* Fix mock

* attempt to fix warning

* Revert "attempt to fix warning"

This reverts commit 3d39142b877fb7ae2bd9bd0b31442a00d4c8f0bf.

* Add call to test interface.
2020-03-25 19:44:13 +01:00
André Silva 3a136bdcf9 Internal version dependencies cleanup (#5401)
* always depend on latest alpha version

* integrate arithmetic fuzzer into root workspace
2020-03-25 15:02:39 +01:00
Tomasz Drwięga 04ccb179e9 Pass transaction source to validate_transaction (#5366)
* WiP

* Support source in the runtime API.

* Finish implementation in txpool.

* Fix warning.

* Fix tests.

* Apply suggestions from code review

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Extra changes.

* Fix test and benches.

* fix test

* Fix test & benches again.

* Fix tests.

* Update bumpalo

* Fix doc test.

* Fix doctest.

* Fix doctest.

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-03-25 14:09:23 +01:00
Nikolay Volf e85d7a4ba7 Fix propagation in network (#5395)
* fix propagation in network

* Update client/service/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* fix suggestion

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-25 13:03:58 +01:00
Wei Tang 2e558908e6 Split PrimaryPreDigest and SecondaryPreDigest (#5373)
* Split PrimaryPreDigest and SecondaryPreDigest

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update primitives/consensus/babe/src/digests.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update primitives/consensus/babe/src/digests.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update primitives/consensus/babe/src/digests.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-03-24 18:07:51 +01:00
Nikolay Volf 27fe4206ed Update schnorrkel with versioning (#5358)
* use versioning for deprecated api, remove deprecated api from regular verification

* Update primitives/core/src/sr25519.rs

* add test to transaction pool

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-24 18:05:57 +01:00
Bastian Köcher 6beb6acbd5 Print an error when a bootnode is on a different chain (#5377)
* Print an error when a bootnode is on a different chain

* Fix tests
2020-03-24 14:24:09 +00:00
Benjamin Kampmann ba06d5ef64 Releasing 2.0.0-alpha.5 (#5340)
* bump pre-version

* Changelog alpha.4->alpha.5

* repo + homepage to sp-consensus-vrf

* Add default rocksdb feature to `frame-benchmarking-cli` (#5367)

* Add default rocksdb feature

* Update Cargo.toml

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-03-24 15:08:02 +01:00
Stanislav Tkach 4414d00082 Get rid of rustc_hex in favor of hex (#5370)
* Get rid of rustc_hex in favor of hex

* Use HexDisplay
2020-03-24 11:48:23 +01:00
Bastian Köcher fde05dd236 Show an error when we connect to a bootnode with a different peer id (#5365)
Currently when we connect to a bootnode and it returns a different peer
id, than the one we provided, we disconnect silently. This pr changes
the behavior to printing an error when a bootnode returns a different
peer id than the provided one.
2020-03-23 18:49:49 +01:00
Wei Tang 9c7261fab3 Simplify BABE pre-digest definitions (#5289)
* Init vrf crate without type alias

* Generic PreDigest definition for BABE

* Fix BABE vrf interface change

* Missing default-features def in sp-consensus-babe

* Fix sp-consensus-babe compile

* frame-babe: fix type inference

* Unify type definitions of vrf output/proof and randomness

* frame-babe: fix tests

* Bump node impl version

* Update cargo lock

* Derive Copy for RawVRFOutput and RawVRFProof

* Fix duplicated derive

Co-authored-by: Gavin Wood <gavin@parity.io>
2020-03-23 11:53:41 +01:00
Pierre Krieger 1da48b3b3f Rewrite reconnect_after_disconnect test (#5351) 2020-03-23 11:50:11 +01:00
Max Inden 26a204da21 client/network-gossip/state_machine: Fix log message (#5350)
A gossip validator returns `ValidationResult::Discard` for messages that
should not be processed nor kept due to not being useful for the node.
Examples are out-of-scope messages or messages with bad signatures. The
corresponding log message did not reflect this, thus the update through
this patch.

One caveat is a finality grandpa neighbor packet. Given that the concept
of neighbor packets is handled in the finality grandpa gossip validator
only, valid neighbor packets are always discarded on the
`client/network-gossip` layer.
2020-03-23 11:04:48 +01:00
ddorgan 56264ad1fc Make pushing consensus message loglevel trace (#5342) 2020-03-21 15:49:51 +00:00
joe petrowski 40f57f8ffa Minor fixes to CLI arg docs (#5339)
* minor fixes to CLI arg docs

* Update client/cli/src/params/shared_params.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-21 08:52:57 +01:00
André Silva 459557aadc grandpa: allow noting same set in gossip with different authorities (#5323)
* grandpa: allow noting same set in gossip with different authorities

* grandpa: add test for note_set behavior
2020-03-20 19:07:10 +01:00
Bastian Köcher 5688f0b293 Update sysinfo to fix compilation on Windows (#5338) 2020-03-20 19:00:39 +01:00
Pierre Krieger 57af4facbd Remove the #[doc(hidden)] reexports of sc_network (#5264)
* Remove the #[doc(hidden)] reexports of sc_network

* Fix tests not building
2020-03-20 16:42:00 +01:00
Benjamin Kampmann 62b3e57470 Ensure schnorrkel is always _at least_ 0.9.1 (#5333) 2020-03-20 16:41:35 +01:00
joe petrowski 589c279adf fix lone txqueue log target (#5331) 2020-03-20 14:15:02 +01:00
Gavin Wood 7947cbf915 Remove migration code. (#5291)
* Remove migration code.

* Fix cargo

* Bump spec version
2020-03-19 23:01:13 +01:00
Bastian Köcher 1b1eea1590 Make ChainSpec implement Send (#5315) 2020-03-19 17:24:31 +01:00
Benjamin Kampmann 0cc54008c8 Update dependencies (#5316)
* switching to released wasmtime

* update depdencies in general

* Update fdlimit

* Update parity common deps

* Also update schnorrkel

* update kvdb-rocksdb

* update further dependents

* also update primitive types

* update cargo.lock

* update merlin

* Bumping evm version
2020-03-19 16:03:26 +01:00
Arkadiy Paronyan a66615446d Fixed a couple of syncing issues (#5277)
* Don't queue duplicate blocks

* Keep queue_blocks on restart
2020-03-19 14:02:07 +01:00
Roman Borschel cfa179f2ce Safe logger initialisation in test. (#5311)
env_logger::init() can lead to panics on concurrent test execution.

See: https://gitlab.parity.io/parity/substrate/-/jobs/422283
2020-03-19 13:53:16 +01:00
André Silva 04d3ab56b2 grandpa: bump dependency version to 0.11.2 (#5312) 2020-03-19 11:44:18 +00:00
André Silva c7a23d836d grandpa: support for hard forking any pending standard changes (#5306)
* grandpa: support for hard forking any pending standard changes

* grandpa: expose authority_set_hard_forks in block import constructor

* grandpa: don't break the public api
2020-03-19 10:37:29 +00:00
Max Inden 9201f8abf3 client/network-gossip/src/bridge: Finish when network event stream closes (#5282)
* client/network-gossip/src/bridge: Finish when network even stream closes

Previously within `<GossipEngine as Future>::poll` one would poll the
`network_event_stream` ignoring all messages other than
`Poll::Ready(Some())`. Ignoring `Poll::Ready(None)` leads to a panic on
the next poll of the stream, gien that it is not fused.

By design `network_event_stream` does not close unless an unbounded send
into it fails, or the `NetworkWorker` gets shut down.

> The stream never ends (unless the `NetworkWorker` gets shut down).
> (client/network/src/service.rs)

An `unbounded_send` to fail on an unbounded channel is unlikely. The
`NetworkWorker` shutting down is not unlikely. In such case the
`GossipEngine` should shut down as well.

With this patch a `<GossipEngine as Future>` finishes on
`Poll::Ready(None)` returned from `network_event_stream`.

* client/finality-grandpa/communication: Error on gossip engine finished

Have `<NetworkBridge as Future>::poll` return `Poll::Ready(Err)` instead
of `Poll::Ready(Ok)` to be consistent with the handling of the neighbor
packet worker stream and the gossip validator report stream. Both `Err`
as well as `Ok` shut down the `NetworkBridge` as well as the
`VoterWorker`.

* client/network-gossip/src/bridge: Add regression test

* client/network-gossip: Move substrate test client to dev dependencies

* client/network-gossip: Remove TODO

Addressed in a follow up pull request.

* client/network-gossip/bridge: Put match on newline after loop

* client/finality-grandpa/src/observer: Fix regression test

Make sure the event stream sender side is not dropped till the end.
2020-03-18 20:15:57 +01:00
Benjamin Kampmann becbe0a339 alpha.4 release fixes (#5303)
* Adding missing metadata for benchmark-pallet

* Missed to bump a few

* release on roling tag

* Wasm executors were missed
2020-03-18 15:47:34 +01:00
Benjamin Kampmann 25e25a83b9 Releasing 2.0.0-alpha.4 (#5279)
* Version bump

* Split generate_changelog.sh into separate script

Can be run in the format `generate_changelog.sh $previous_version $version`.

* remove early exit from publish_draft_release.sh

* adding changelog

* ci: change last_github_release to also find pre-releases

Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-03-18 14:37:21 +01:00
Toralf Wittner 89b93d9d4b sc-network: Return on Poll::Pending. (#5295)
If polling encounters a `Poll::Pending` we need to return it instead of
continuing the loop which may turn it into a blocking operation, causing
problems with executors.
2020-03-18 11:21:10 +00:00
Nikolay Volf db86094b03 Produce block always on updated transaction pool state (#5227)
* make sure return ready iterator once state is updated

* update sc_basic_authorship tests

* update node tests

* fix manual seal

* actually fix service test

* add tests

* Update client/basic-authorship/src/basic_authorship.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* helper function

* review suggestions

* warning and continue

* add debug log

* use futures::chennel::oneshot

* use declaration bound

* no option for updated_at

* no allocation

* ready_at / ready

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-17 16:24:04 +01:00
Arkadiy Paronyan d146cc3ae3 Don't format when telemetry is disabled (#5273) 2020-03-17 12:22:05 +01:00
Max Inden e91d4be998 client/network/src/protocol: Refactor Prometheus metric logic (#5266)
* client/network/src/protocol: Start Prometheus metric help with capital

* client/network/src/protocol: Differentiate metric status as label

Prometheus query language is powerful through its multi-dimensional data
model. Metric names are hirarchical whereas labels enable data to become
multi-dimensional.

Exposing the justification of finality-proof status as a label allows
for more powerful queries.

* client/network/src/protocol: Remove 'Total' from non counter metric help

The word 'total' is reserved for accumulating counters. Counters have to
be monotonically increasing. `obsolete_requests` can decrease, thereby
it is defined as a `Gauge` and not a `Counter`.

For more details on metric naming see
https://prometheus.io/docs/practices/naming/
2020-03-17 11:47:55 +01:00