Commit Graph

1184 Commits

Author SHA1 Message Date
Bastian Köcher db8a0cafa9 Fixes logging of target names with dashes (#7281)
* Fixes logging of target names with dashes

There was a bug in tracing-core which resulted in not supporting dashes
in target names. This was fixed upstream. Besides that a test was added
to ensure that we don't break this again.

* Extend test
2020-10-08 18:40:27 +00:00
Alexander Theißen 443725f0f6 seal: Add automated weights for contract API calls (#7017)
* seal: Add capability to put uninstrumented code (for benchmarks)

Benchmarks should only measure the overhead of the API calls itself.
For that reason we want to run them without instrumentation.

* seal: Cap the the data length for deposited events

Data used in events has storage implications for archive nodes.
Those need to keep the events in storage forever. For that reason
we want to limit the amount of storage that can be used inside events.

* seal: Fix error reporting in the case out of bound sandbox access

* seal: Refactor existing benchmarks

* seal: Convert benchmark file to tabs

* seal: Add benchmarks for functions called by contracts

* seal: Create a default schedule from benchmark generated WeightInfo

* seal: Make use of WeightInfo in extrinsic weight annotations

* seal: Replace the old schedule by the benchmark generated one

* Review: Fix copy paste typo in schedule construction

* Review: Fix stale docs

* Fix whitespace errors

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Review: Use checked_div in order to be more defensive

* Review: Rename no_charge to already_charged

* Review: Whitelist caller of extrinsics

* Review: Remove trailing whitespace

* Review: Remove confusing "self::" syntax

* Review: Add docs for the benchmark prepration submodule

* Review: Move code generation functions to own module

* Review: Refactor and document benchmark helper functions

* Remove additional empty line

* Added missing comment on caller_funding

* Update frame/contracts/src/benchmarking/code.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Fix missing sp_std::prelude import in code.rs

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_contracts --heap-pages 4096

* Use weights from the benchmark machine for the substrate node

* Remove prefixes from Schedule members

* Data lengths in the WeightInfo Trait are specified in kilobytes

* Rename ApiWeights to HostFunctionWeights

Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 15:36:37 +00:00
Sergei Shulepov aba0128f6f bump-allocator: document & poison (#7277)
* bump-allocator: refine comments

* Rename EMPTY_MARKER to Nil

It's a misnomer since it doesn't actually denote an empty list as it might suggest but rather the end of a list. So I borrowed Nil from Cons/Nil.I could've gone with Null, especially considering that the variant name is already named `Null`, but I preferred `Nil` because the NIL_MARKER is not 0 as `null` may suggest, but rather 0xFFFFFFFF.

* Rename N to N_ORDERS; docs;

Supply a, hopefully, more readable comment as well.

* allocator poisoning: document and enforce.

* Review: Fix line wrapping for a call

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* add a test on oom

* Update primitives/allocator/src/freeing_bump.rs

* Make ui tests happy

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
2020-10-08 11:40:52 +00:00
Max Inden a772eb27a2 client/cli: Update to fdlimit 0.2.1 (#7249)
fdlimit 0.2.1 contains a small patch allowing compilation on platforms
where libc represents rlimits as u32.
2020-10-02 08:51:21 +00:00
Denis Pisarev cdff10e1b2 WIP: remove deprecated only/except clauses (#7229)
* change (CI): remove deprecated only/except clauses

* change (CI): remove

* change (CI): debug regex

* change (CI): regex shouldn't be surrounded with quotes in `rules`

* change (CI): no quotes in `rules`

* change (CI): debug workflow

* change (CI): almost equal

* change (CI): splinters

* change (CI): some details [chaos:basic]

* change (CI): add manual substrate build on PRs

* change (CI): mistake with publish-to-crates-io

* change (CI): reapproach chaos-only; rename rules anchors [skip ci]

* change (CI): add linux and mac to manual jobs for PRs [skip ci]

* change (CI): remove `pre-v` tags rules [skip ci]

* change (CI): add subkey for macos build as manual on PRs as well

* change (CI): add glob to match every changed file in monitoring

* test (CI): [chaos:basic]
2020-09-30 22:16:48 +02:00
Bastian Köcher adc1892fcb Fix compilation with environmental on latest nightly (#7238) 2020-09-30 09:44:29 +00:00
Alexander Popiak 1a65205278 Move proxies migration (#7205)
* move the time delayed proxies migration into a separate function

* add use statement

* Update frame/proxy/src/lib.rs

* bump proxy cargo version

* update Cargo.lock

* Update lib.rs

* better format

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Gav Wood <gavin@parity.io>
2020-09-25 09:11:48 +00:00
Shawn Tabrizi 4e5f9fb72b More Improvements to Benchmarking CLI + Generic WeightInfo (#7168)
* Generic WeightInfo, Output File Path

* Make `WeightInfo` generic

* Add support for header file

* fix line width

* add `--spaces` flag

* Configurable trait and struct strings

* make elections phragmen generic

* Add date to generated file

* Apply suggestions from code review

Co-authored-by: Gavin Wood <gavin@parity.io>

* fixes

* add settings metadata to output

Co-authored-by: nikvolf <nikvolf@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
2020-09-23 13:56:30 +00:00
Kian Paimani 313f86ec23 PhragMMS election. (#6685)
* Revamp npos-elections and implement phragmms

* Update primitives/npos-elections/src/phragmms.rs

* Fix build

* Some review grumbles

* Add some stuff for remote testing

* fix some of the grumbles.

* Add remote testing stuff.

* Cleanup

* fix docs

* Update primitives/arithmetic/src/rational.rs

Co-authored-by: Dan Forbes <dan@danforbes.dev>

* Small config change

* Better handling of approval_stake == 0

* Final touhces.

* Clean fuzzer a bit

* Clean fuzzer a bit

* Update primitives/npos-elections/src/balancing.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Fix fuzzer.

* Better api for normalize

* Add noramlize_up

* A large number of small fixes.

* make it merge ready

* Fix warns

* bump

* Fix fuzzers a bit.

* Fix warns as well.

* Fix more tests.

Co-authored-by: Dan Forbes <dan@danforbes.dev>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-09-23 08:16:10 +00:00
Benjamin Kampmann ecdc94420e Releasing 2.0 – two dot 😮 (#7182)
Tagging the release of substrate 2.0


[ci: skip-checks]
2020-09-22 19:47:38 +02:00
Guillaume Thiolliere 98951b33a9 Allow pallet in construct_runtime to have fixed index (#6969)
* implement index for pallet + some tests

* add test and doc

* remove deprecated and document behavior

* update internal doc

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* address review

* use index for all module, break construct_runtime

* fix line length

* implement migration helper funciton in scheduler

* fix start at index 0

* Update frame/scheduler/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update frame/support/procedural/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* bump frame-metadata crate

* factorize

* avoid some unwrap and remove nightly join

* Update frame/support/src/event.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fix test

* add test and improve error message

* factorize test

* keep iterator, and use slice instead of vec

* refactor to avoid to have expects

* small refactor

* Test something

* Make sure we update the `Cargo.lock`

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* return 2 error

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update frame/scheduler/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* fix typo

* Revert "fix typo"

This reverts commit f2de8f2db34e8ac72bc9c34437c60dca3fa4ac22.

* Revert "Update frame/scheduler/src/lib.rs"

This reverts commit 6feb4605c6f784b64591e229de7a6fec6dbffb4b.

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-09-22 14:54:03 +00:00
Shawn Tabrizi ea21afa42a Docs for Frame Benchmarking (#7121)
* remove test benchmark pallet

* docs

* finish docs

* Update README.md

* simplify intro

* introduce weight later

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update frame/benchmarking/README.md

Co-authored-by: David <dvdplm@gmail.com>

* Update frame/benchmarking/README.md

* Update frame/benchmarking/README.md

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update README.md

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-09-21 21:10:03 +00:00
Shawn Tabrizi 6d654a336b Remove generic asset palet (#7156) 2020-09-21 11:16:50 +02:00
Maciej Hirsz ae52587f41 Bump jsonrpc-* dependencies to v15 (#7157)
* Bump jsonrpc-* dependencies to v15

* Remove unused imports
2020-09-21 08:46:10 +00:00
Jon Häggblad 8a0ac4c40c grandpa-rpc: use FinalityProofProvider to check finality for rpc (#6215)
* grandpa-rpc: use FinalityProofProvider to check finality for rpc

* grandpa-rpc: minor tidy

* grandpa-rpc: remove dyn FinalityProofProvider

* grandpa-rpc: remove unused dependencies

* node: move finality_proof_provider setup

* grandpa-rpc: print error reported by finality_proof_provider

* grandpa-rpc: add note about unnecessary encode/decode

* grandpa-rpc: dont encode/decode and use correct hash

* grandpa-rpc: set_id is optional

* grandpa-rpc: create test for prove_finality

* grandpa-rpc: set visibility back to how it was

* grandpa-rpc: remove unused dependency

* grandpa-rpc: minor tidy

* grandpa: doc strings

* grandpa-rpc: rename to prove_finality

* grandpa-rpc: use current set id if none is provided

* grandpa-rpc: remove unnecessary check in test

* node: group finality_proof_provider in rpc_setup

* grandpa: make prove_finality concrete in FinalityProofProvider

* grandpa-rpc: wrap finality output in struct and store as Bytes

* grandpa-rpc: exhaustive error codes and wrap

* grandpa-rpc: let prove_finality take a range instead of a starting point

* grandpa-rpc: fix test for changed API

* grandpa-rpc: fix line length

* grandpa: fix reviewer nits

* node/rpc: fix reviewer comments
2020-09-18 16:39:27 +00:00
Anton Gavrilov 2c5a82f8f1 Prometheus metrics for RPC calls (#7088)
* WS and HTTP middlewares added

* Prometheus endpoint added

* Counters renamed

* Proper style for inc

* Metrics initialization re-written

* Rework to handler middleware

* Introduce transport prefix for metrics

* String shortened

* Commented code removed, new line inserted

* One more string shortened

* Wasm build fixed

* Wasm build fixed once again

* Rework to shared metrics

* Added collectors label

* Tilde removed from cargo

* Switch to owned metrics in parameters
2020-09-18 15:15:19 +02:00
Wei Tang 5380bbce04 pow: replace the thread-base mining loop with a future-based mining worker (#7060)
* New worker design

* Remove unused thread import

* Add back missing inherent data provider registration

* Add function to get a Cloned metadata

* Add some docs

* Derive Eq and PartialEq for MiningMetadata

* Fix cargo lock

* Fix line width

* Add docs and fix issues in UntilImportedOrTimeout

* Update client/consensus/pow/src/lib.rs

Co-authored-by: David <dvdplm@gmail.com>

* Add back comments

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
2020-09-18 09:37:31 +00:00
Benjamin Kampmann a9c73113a8 Tracing for wasm with bridging to native (#6916)
* implement events handling, implement parent_id for spans & events

* add events to sp_io::storage

* update test

* add tests

* adjust limit

* let tracing crate handle parent_ids

* re-enable current-id tracking

* add test for threads with CurrentSpan

* fix log level

* remove redundant check for non wasm traces

* remove duplicate definition in test

* Adding conditional events API

* prefer explicit parent_id over current,

enhance test

* limit changes to client::tracing event implementation

* remove From impl due to fallback required on parent_id

* make tracing codecable

* replace with global tracing

* new tracing interface

* impl TracingSubscriber in client

* implement access to global TracingSubscriber from primitives

* span for wasm

* increment towards Wasm Tracing Subscriber implementation

* increment, remove sp-tracing from runtime-interface

* increment, it compiles

* attained original functionality with new mechanism

* implement remaining TracingSubscriber functions

* remove spans from decl_module

* add handling for encoded values

* Revert "replace with global tracing"

This reverts commit 8824a60deea54d9b437407a21c8ceaf6a1902ee5.

* Wasm Side Tracing

* tracing on wasm

* enable tracing wasm on node-runtime

* export all the macros in std

* tracing subscriber on wasm-side only

* pass spans and events over and record them

* reactivate previous code and  cleanup

* further cleaning up

* extend the span macros, activate through executive

* tracking the actual extrinsic, too

* style

* fixing tests

* spaces -> tabs

* attempting to reactivate params

* activate our tests in CI

* some passing

* tests passing

* with core lazy

* global tracer for wasm side with pass over

* fixing metadata referencing

* remove const_fn feature requirement

* reenable dispatch traces

* reset client tracing

* further cleaning up

* fixing runtime-test

* move tracing-build setup into runtime-test

* Merge DebugWriter from tracing and frame-support, move to sp-std

* remove dangling fixme

* Docs for tracing primitives

* cleaning up a bit more

* Wasm interface docs

* optimise docs.rs setup

* adding tracing flags to uncomment

* remove brace

* fixing imports

* fixing broken syntax

* add required modules

* nicer formatting

* better target management

* adding low level storage tracing events into frame

* add custom Debug impl for WasmMetadata

* cloning profiler

* adding info about cloning profiler

* using in-scope for within calls

* proper time tracing, cleaning up println

* allow to disable tracing on runtime_interface-macro

* disable tracing for wasm-tracing-interface

* simplify wasm-tracing-api

* update client to new interface

* fixing docs and tests for sp-tracing

* update integration tests

* re-activating enter_span

* dropping FIXME, it's documented

* fix formatting

* fix formatting

* fix imports

* more debug info

* inform wasm about it being disabled by returning 1

* only one tracer, but enabled multi-all support

* make trait pub again for tests

* Apply suggestions from code review

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* fixing wasm doc tests for proper usage

* remove unnecessary import

* fixing formatting

* minor style fixes

* downgrading wabt

* update error message for UI

* Fix interface test

* next attempt to fix macros

* geee

* revert tracing on hashed for future PR

* remove local macros, use originals

* we are able to convert to static items

* implement more WasmValue types

* adding support to convert str, debug and encoded values

* more minor fixes

* revert unsafe 'static making

* fix indentation

* remove commented lines

* bump all them tracing versions

* cleaning up docs and info

* document new flag

* the new layered system handles span cloning better

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: David <dvdplm@gmail.com>
2020-09-18 06:10:59 +00:00
Wei Tang 9fdd4672b0 Use tracing-based subscriber logging (#6825)
* init_logger: switch from log-based to tracing-based and add compatibility layer

* Move tracing profiling subscriber related config realization

* sp-tracing: change profiling to be a layer instead of a subscriber

* Enable profiling layer in cli

* Change all test env_logger init to sp_tracing::try_init_simple

* Remove all local env_logger dependency

* Add missing tracing-subscriber dependency

* frame-sudo: fix tests

* frame-support: fix tests

* Fix frame/pallet and executor tests

* Fix the remaining tests

* Use subscriber's try_init as recommended by @davidbarsky

* Be explict that the tracing-log feature is needed

* Set subscriber writer to stderr

* Shorter line width

* Update cargo lock tracing version

* Fix sc_tracing crate compile

* Fix sc_authority_discovery crate test

* unremove default-features

* Leave enabled to default true

* Warn if global default cannot be set

* Fix unused import

* Remove unused PROXY_TARGET

* Change all reference from rc5 to rc6

* Change all reference of rc2 to rc6

* Fix styling

* Fix typo

* make logger init error'ing

* re-fixing the test issue

Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-09-17 11:04:43 +02:00
Shawn Tabrizi ab97e94972 Add benchmarking pipeline to node-template (#7122) 2020-09-17 10:09:59 +02:00
Bastian Köcher 9dd5812eaf Improve error output of wasm-builder when wasm ins't installed (#7105)
This improves the error message of wasm-builder when the wasm toolchain
isn't installed. Currently we print that the wasm toolchain is not
installed, but the actual problem is that there is a bug in the
packaging in rust. This will now be much easier to debug, by printing
the full error message of the compiler.
2020-09-15 11:42:23 +02:00
Roman Borschel 51706a7620 Upgrade to libp2p-0.28. (#7077)
* Upgrade to libp2p-0.28

* Clean up test imports.

* CI

* CI

* CI?

* CI once more.

* One more.

* CI

* CI

* CI
2020-09-14 14:27:58 +00:00
Jon Häggblad 2da6adfdf1 grandpa-rpc don't share subscription manager, only executor (#7039)
* service builder: fix todo about jsonrpc Option workaround

* grandpa-rpc: only share executor instead of sub manager

* grandpa-rpc: fix compilation

* grandpa-rpc: rename to subscription_executor

* node/cli: remove another unused jsonrpc dependency

* grandpa: apply style fixes from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-09-14 11:02:37 +00:00
Vincent Ulitzsch 2348f03f0f Add fuzzer for the compact custom codec implementation from PR #6720 (#7091)
* Add fuzzer for the compact custom codec implementation introduced in PR #6720.

This commit adds a fuzzing harness for the custom compact encoding/decoding
introduced in PR #6720.

* Update primitives/npos-elections/fuzzer/src/compact.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock: Add changes in elections-fuzzer

* Change indentation from spaces to tabs

Co-authored-by: Vincent Ulitzsch <vincent@srlabs.de>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-09-13 16:28:10 +00:00
cheme 232a30fdb4 state_machine no_std witness externalities (#6934)
* checkpoint before removing CT from change trie

* before trie backend without tx

* undo

* Started no transaction, but would need using a different root
calculation method, out of the scope of this pr, will roll back.

* Remove NoTransaction.

* partially address review.
dummy stats implementation for no_std.

* Remove ChangeTrieOverlay.

* modified function

* Remove witness_ext

* need noops changes root

* update from cumulus branch

* line break

* remove warning

* line break

* From review: renamings and stats active in no std (except time).

* include cache, exclude change trie cache with individual temporary bad looking
no_std check

* little test

* fuse imports and filter_map prepare_extrinsics_input_inner fold.

* put back ExtInner into Ext, awkward double proto for new function.

* Apply suggestions from code review

* Update primitives/state-machine/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-09-11 10:11:25 +00:00
kaichao 91c0213413 Set reserved nodes with offchain worker. (#6996)
* add offchain worker api to set reserved nodes.

* new offchain api to get node public key.

* node public key from converter

* refactor set reserved nodes ocw api.

* new ndoe authorization pallet

* remove unnecessary clone and more.

* more

* tests for node authorization pallet

* remove dependency

* fix build

* more tests.

* refactor

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update frame/node-authorization/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update frame/node-authorization/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update frame/node-authorization/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* format code

* expose NetworkService

* remove NetworkStateInfo in offchain

* replace NodePublicKey with PeerId.

* set max length of peer id.

* clear more

* use BTreeSet for set of peers.

* decode opaque peer id.

* extract NetworkProvider for client offchain.

* use OpaquePeerId in node authorization pallet.

* fix test

* better documentation

* fix test

* doc

* more fix

* Update primitives/core/src/offchain/mod.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update client/offchain/src/api.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* derive serialize and deserialize

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-09-10 15:26:09 +00:00
Sergei Shulepov e9982eb797 What happens if we remove wat? (#7056)
* What happens if we remove wat?

* Update Cargo.lock
2020-09-09 13:23:40 +00:00
Sergei Shulepov a137160924 Change wabt to wat (#7050) 2020-09-08 22:49:54 +02:00
Benjamin Kampmann 4ef025b761 Downgrade wabt = 0.9.1 (#7042) 2020-09-08 08:52:04 +00:00
Seun Lanlege b0ff817ba0 manual seal is now consensus agnostic (#7010)
* manual seal is now consensus agnostic

* pr grumbles
2020-09-03 12:55:12 +00:00
Max Inden aa5a10802c *: Update to Prometheus v0.10.0 (#6964)
* *: Update to Prometheus v0.10.0-rc.1

* *: Update to Prometheus v0.10.0
2020-09-01 11:23:10 +00:00
cheme 734d30bf5d Update substrate bip39 version. (#6955)
* update bip39 version

* and lock
2020-08-31 14:16:30 +00:00
Max Inden 875daa31bf client/cli/src/config: Warn on low file descriptor limit (#6956)
* client/cli/src/config: Warn on low file descriptor limit

Substrate sets the soft file descriptor limit to the hard limit at
startup. In the case of the latter being low already (< 10_000) a
Substrate node under high demand might run into issues e.g. when opening
up new TCP connections or persisting data to the database.

With this commit a warn message is printed to stderr.

* client/cli/Cargo.toml: Update to fdlimit 0.2.0
2020-08-31 12:37:06 +00:00
Andronik Ordian d0a58a7360 update kvdb-rocksdb to 0.9.1 and rocksdb to 6.11.4 (#6963) 2020-08-29 03:30:13 +00:00
Pierre Krieger 37d0e00d83 Implement request-responses protocols (#6634)
* Implement request-responses protocols

* Add tests

* Fix sc-cli

* Apply suggestions from code review

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix naming

* Fix other issues

* Other naming fix

* Fix error logging

* Max sizes to u64

* Don't kill connections on refusal to process

* Adjust comment

Co-authored-by: Max Inden <mail@max-inden.de>
2020-08-27 12:53:20 +00:00
Guillaume Thiolliere d628c62d6d update tracing attribute (#6950) 2020-08-24 15:33:15 +00:00
Ashley f009f6c227 Add a LightSyncState field to the chain spec (#6894)
* Reset code, almost ready for PR

* Improved build_hardcoded_spec

* Fix line widths

* Fix tests

* Fix sc-service-test

* Suggestions from code review

* Rename to LightSyncState

* It's not syncing :^(

* It syncs!

* Remove rpc call

* Convert spaces to tabs

* Moved sc-service things to export_sync_state.rs

* Fix tests

* Wait for syncing with network_status_sinks

* Remove sc-network from node-template

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Various changes, split the flag up into 2 pieces to make testing easier.

* Update client/cli/src/commands/build_spec_cmd.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert a lot of changes

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-08-24 08:34:16 +00:00
Max Inden 5c500aa783 client/authority-discovery: Append PeerId to Multiaddr at most once (#6933)
* client/authority-discovery/worker: Extract address getter

* client/authority-discovery: Test for no duplicate p2p components

* client/authority-discovery: Append PeerId to Multiaddr at most once

When collecting the addresses to be published for the local node,
`addresses_to_publish` adds the local nodes `PeerId` to each
`Multiaddr`. Before doing so, ensure the `Multiaddr` does not already
contain one.

* client/authority-discovery: Remove explicit return
2020-08-24 09:37:32 +02:00
Qinxuan Chen 8ae1695526 Remove duplicated dependencies (#6930)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2020-08-21 16:25:58 +02:00
Max Inden 69e349bd51 *: Update to next libp2p version 0.24.0 (#6891)
* *: Update to next libp2p version (likely v0.24.0)

* Revert "*: Update to next libp2p version (likely v0.24.0)"

This reverts commit ffe1545aba6c2557a2843579de331f3fc1c60743.

* */Cargo.toml: Update to libp2p v0.24.0

* client/network/src/service: Handle ConnectionClosed returning Option

* Cargo.*: Test kad usize conversion

* Revert "Cargo.*: Test kad usize conversion"

This reverts commit ad317879782f982cb4a4c76029a72b5b97e82bec.

* Cargo.lock: Update to libp2p-kad v0.22.1

* client/cli/Cargo.toml: Update to libp2p 0.24.0
2020-08-21 06:25:23 +00:00
Benjamin Kampmann 2416aa7999 prepping for releasing rc6 (#6922)
* Bump version

* update test-utils crates to be ready for publishing

* adding changelog

* Adding automaticly generated READMEs

* fixing versions

* another version mishap
2020-08-20 17:04:42 +02:00
Seun Lanlege 6963272451 Merge Subkey into sc-cli (#4954)
* draft

* revert

* WIP

* all that remains is tests

* update Cargo.lock

* tests WIP

* WIP refactor node-template-runtime and node-runtime

* implments sc_cli::RuntimeAdapter for node_template_runtime::Runtime

* final draft

* fix update_config for subcommands

* proper AccountId decoding

* test-runtime tests

* revert

* move RuntimeAdapter to cli-utils

* use &'static str for TryFrom::<&'a str>::Error for Ss58AddressFormat

* tests

* add frame-system to sc-cli dev-dependencies

* add frame-system to sc-cli dev-dependencies

* fix ui test

* wip

* fixed inspect test

* bump impl version

* bump impl version, fixx spaces remove todos

* pallet-balances-cli, rustc for some reason cannot resolve pallet_balances_cli in node-cli 😩

* wip

* Subcommand::run takes &self

* can't believe i missed that 🤦🏾‍♂️

* bump wasm-bindgen for some reason

* adds key subcommand, rename generate-node-key to generate-node-id

* cargo update and crossed fingers 🤞🏽

* update ui test

* update more ui tests

* should be all good now

* revert subkey change

* revert subkey change

* adds frame-utilities-cli

* Apply suggestions from code review

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* removes frame from sc-cli, fix license

* my editor and ci disagrees on line width

* bump spec version

* turn off default features for parity-scale-codec

* enable full_crypto feature for sp-core in cli-utils

* merge frame-utilities-cli with pallet-balances-cli

* remove full_crypto feature from sp_core in cli-utils

* bump Cargo.lock

* cli-utils -> frame-utils

* rename BlockNumber to GenericNumber, fix spaces

* fix spaces

* construct additional_signed manually

* sign test

* remove unused vars

* implement subkey with frame-utilities-cli and sc_cli

* fix moduleid test

* CI and clion disagree on line widths

* adds associated Params type to SignedExtensionProvider

* Apply suggestions from code review

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* move some code around

* removes unneccesary generic params

* moves module_id back to frame_utilities_cli

* Apply suggestions from code review

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* remove print_ext

* remove MaybeDisplay from pallet_balances::Trait::Balance

* a lot of stuff tbh

* adds ExtrasParamsBuilder

* remove tests for ModuleIdCmd

* address comments from PR

* bump Cargo.lock

* manually insert key into keystore

* remove unnecessary SharedParams

* add validation to vanity pattern, remove unused arg

* remove SharedParams from Sign, Vanity, Verify

* remove SharedParams from ModuleIdCmd, remove expect from Verify, new line to Cargo.toml

* remove SharedParams from InsertCmd

* 🤦🏾‍♂️

* deleted prometheus.yml

* move a few things around

* fix vanity test

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-08-20 09:55:03 +00:00
Shawn Tabrizi 481ad884d6 Dynamic Benchmarking DB Whitelist (#6815)
* Add `get_whitelist` api

* add whitelisted caller

* Whitelist caller

* remove caller 0

* initial piping of origin (not actual value yet)

* remove attempt to pass origin around

* Add whitelist for `DidUpdate` storage on `pallet_timestamp`

* fix traits

* only add to whitelist if !contains

* PassBy not implemented error

* Whitelist read/writes explicitly per key

* update docs

* reduce trait constraint

* copy pasta

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* rename functions @apopiak

* missed some renaming

* enable doc tests

* Update docs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
2020-08-19 16:15:50 +00:00
Seun Lanlege 6c1737fe77 RpcHandlers Refactorings (#6846)
* allow access to the underlying Pubsub instance from RpcHandlers

* bump Cargo.lock

* no more Arc<RpcHandlers>

* bump Cargo.lock

* Debug,.

* Arc<RpcHandlers>

* RpcHandler

* RpcHandlers::io_handler

* remove chain spec from cli

* address pr comments

* remove stray newline

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
2020-08-15 09:08:31 +00:00
Roman Borschel 327e02942c Update to libp2p-0.23. (#6870)
* Update to libp2p-0.23.

Thereby incorporate bandwidth measurement along the
lines previously done by libp2p itself.

* Tweak dependencies for wasm32 compilation.

For wasm32 we need to enable unstable features to
make `task::Builder::local` available.

* Simplify dependencies.

* Simplify.

Leave the calculation of bytes sent/received per second
to the outer layers of the code, subject to their own
individual update intervals.

* Cleanup

* Re-add lost dev dependency.

* Avoid division by zero.

* Remove redundant metric.

* Enable sending of noise legacy handshakes.

* Add comment about monotonic gauge.

* CI
2020-08-14 08:41:47 +00:00
Kian Paimani 775e84cc04 More renaming to move away from phragmen. (#6886) 2020-08-13 23:30:22 +02:00
Cecile Tonglet 93612c11f4 Add async test helper to timeout and provide a task_executor automatically (#6651)
* Initial commit

Forked at: 74655edebc
Parent branch: origin/master

* Add async test helper to timeout and provide a task_executor automatically

* simplify error message to avoid difference between CI and locally

* forgot env var

* Use runtime env var instead of build env var

* Rename variable to SUBSTRATE_TEST_TIMEOUT

* CLEANUP

Forked at: 74655edebc
Parent branch: origin/master

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Re-export from test-utils

* Default value to 120

* fix wrong crate in ci

* Revert "Default value to 120"

This reverts commit 8e458717078b242ffce7d3c4f66241d76f075125.

* Fix version

* WIP

Forked at: 74655edebc
Parent branch: origin/master

* WIP

Forked at: 74655edebc
Parent branch: origin/master

* WIP

Forked at: 74655edebc
Parent branch: origin/master

* remove feature flag

* fix missing dependency

* CLEANUP

Forked at: 74655edebc
Parent branch: origin/master

* fix test

* Removed autotests=false

* Some doc...

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* WIP

Forked at: 74655edebc
Parent branch: origin/master

* WIP

Forked at: 74655edebc
Parent branch: origin/master

* Update test-utils/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-08-12 14:07:11 +00:00
mattrutherford 9442c19b48 Implement tracing::Event handling & parent_id for spans and events (#6672)
* implement events handling, implement parent_id for spans & events

* add events to sp_io::storage

* update test

* add tests

* adjust limit

* let tracing crate handle parent_ids

* re-enable current-id tracking

* add test for threads with CurrentSpan

* fix log level

* remove redundant check for non wasm traces

* remove duplicate definition in test

* Adding conditional events API

* prefer explicit parent_id over current,

enhance test

* limit changes to client::tracing event implementation

* remove From impl due to fallback required on parent_id

* implement SPAN_LIMIT

change event log output

* change version of tracing-core

* update dependancies

* revert limit

* remove duplicate dependency

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-08-12 11:53:21 +00:00
Nikolay Volf c347300e3e Move to upstream wasmtime, refactor globals snapshot (#6759)
* refactor globals snapshot

* ignore test

* update pwasm-utils ref

* line width

* add doc comment for internal struct

* add explanation for iteration

* Demote rustdoc to a comment

* use 0.14

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2020-08-11 15:05:31 +00:00
Alexander Theißen 04b185e3d4 seal: Change prefix and module name from "ext_" to "seal_" for contract callable functions (#6798)
* seal: Change prefix "ext_" to "seal_" for contract callable functions

The word Ext is a overloaded term in the context of substrate. It usually
is a trait which abstracts away access to external resources usually in order
to mock them away for the purpose of tests. The contract module has its own
`Ext` trait in addition the the substrate `Ext` which makes things even more
confusing.

In order to differentiate the contract callable functions more clearly from
this `Ext` concept we rename them to use the "seal_" prefix instead.

This should change no behaviour at all. This is a pure renaming commit.

* seal: Rename import module from "env" to "seal0"

* seal: Fixup integration test

* seal: Add more tests for new import module names
2020-08-10 13:14:34 +00:00