Commit Graph

3223 Commits

Author SHA1 Message Date
Nikolay Volf 6e31cd7af7 reference sc-service with rocksdb feature (#4918) 2020-02-13 15:26:56 +01:00
Sergei Pepyakin 49af986ad4 executor: Migrate wasmtime backend to a high-level API (#4686)
* Migrate wasmtime backend to wasmtime-api

* Port to a newer version of wasmtime

* Update to the latest changes.

* Rejig the sandbox module a bit

* Materialze

* Fixes.

* executor wasm_runtime fix

* Refactor everything

* More refactoring

* Even more refactorings

* More cleaning.

* Update to the latest wasmtime

* Reformat

* Renames

* Refactoring and comments.

* Docs

* Rename FunctionExecutor to host.

* Imrpove docs.

* fmt

* Remove panic

* Assert the number of arguments are equal between wasmtime and hostfunc.

* Comment a possible panic if there is no corresponding value variant.

* Check signature of the entrypoint.

* Use git version of wasmtime

* Refine and doc the sandbox code.

* Comment RefCells.

* Update wasmtime to the latest-ish master.

This may solve a problem with segfaults.

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Use full SHA1 hash of wasmtime commit.

* Add a panic message.

* Add some documentation

* Update wasmtime version to include SIGSEGV fix

* Update to crates.io version of wasmtime

* Make it work.

* Move the creation of memory into `InstanceWrapper::new`

* Make `InstanceWrapper` !Send & !Sync

* Avoid using `take_mut`

* Update client/executor/wasmtime/Cargo.toml

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Limit maximum size of memory.

* Rename `init_state` to `with_initialized_state`

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-02-13 14:54:19 +01:00
Kian Paimani c871eaacbc Per-things trait. (#4904)
* Give perthigns the trait it always deserved.

* Make staking and phragmen work with the new generic per_thing

* Make everything work together 🔨

* a bit of cleanup

* Clean usage

* Bump.

* Fix name

* fix grumbles

* hopefully fix the ui test

* Some grumbles

* revamp traits again

* Better naming again.
2020-02-13 13:09:33 +01:00
Nikolay Volf e6454eb091 Full block import benchmark (#4865)
* full block import benchmark

* try rocksdb cache

* add profiling helper

* use random keyring instead of zero caching

* update docs

* add more io stats

* remove last sentence

* add ci job to see

* Update primitives/keyring/src/sr25519.rs

Co-Authored-By: Marcio Diaz <marcio.diaz@gmail.com>

* switch to 100tx-block

* remove ci script

Co-authored-by: Marcio Diaz <marcio@parity.io>
2020-02-13 14:09:00 +03:00
Bastian Köcher 96862f835b Fix chain-spec and make sure it does not breaks again (#4906)
* Fix chain-spec and make sure it does not breaks again

* REview feedback
2020-02-13 10:06:10 +01:00
Nikolay Volf a84c05da22 add sr25519 bench (#4905) 2020-02-12 17:32:38 +03:00
Xiliang Chen 35b5cdd29b serialize partial_fee into string (#4898)
* serialize partial_fee into string

* implement deserialize

* bump version
2020-02-12 13:16:00 +01:00
Robert Habermeier b3cf97940c add some more docs on PreRuntime digests (#4896) 2020-02-12 13:15:49 +01:00
Max Inden 3f3910ccaf client/network-gossip: Integrate GossipEngine tasks into Future impl (#4767)
`GossipEngine` spawns two tasks, one for a periodic tick, one to forward
messages from the network to subscribers. These tasks hold an `Arc` to a
`GossipEngineInner`.

To reduce the amount of shared ownership (locking) this patch integrates
the two tasks into a `Future` implementation on the `GossipEngine`
struct. This `Future` implementation can now be called from a single
owner, e.g. the `finality-grandpa` `NetworkBridge`.

As a side effect this removes the requirement on the `network-gossip`
crate to spawn tasks and thereby removes the requirement on the
`finality-grandpa` crate to spawn any tasks.

This is part of a greater effort to reduce the number of owners of
components within `finality-grandpa`, `network` and `network-gossip` as
well as to reduce the amount of unbounded channels. For details see
d9837d7dd, 5f80929dc and 597c0a6c4.
2020-02-12 13:15:26 +01:00
Shawn Tabrizi 13971fe2a7 Benchmark the Balances Pallet (#4879)
* Initial transfer bench

* Add best case

* Transfer keep alive

* Set balance benchmarks

* Bump impl

* Fix text

Co-authored-by: Gavin Wood <github@gavwood.com>
2020-02-12 12:11:20 +01:00
Toralf Wittner 51a45c5d9f network: Use "one shot" protocol handler. (#3520)
* network: Use "one shot" protocol handler.

Add two new `NetworkBehaviour`s, one handling remote block requests
and another one to handle light client requests (both local and from
remote). The change is motivated by the desire to use multiple
substreams of a single connection for different protocols. To achieve
this, libp2p's `OneShotHandler` is used as a protocol handler in each
behaviour. It will open a fresh substream for the duration of the
request and close it afterwards. For block requests, we currently only
handle incoming requests from remote and tests are missing. For light
client handling we support incoming requests from remote and also
ported a substantial amount of functionality over from
`light_dispatch.rs` (including several tests). However the result lacks
in at least two aspects:

(1) We require external updates w.r.t. the best block per peer and
currently nothing updates this information.
(2) We carry a lot of peer-related state around.

Both aspects could be simplified by externalising peer selection and
just requiring a specific peer ID where the request should be sent to.
We still have to maintain some peer related state due to the way
libp2p's swarm and network behaviour work (e.g. we must make sure to
always issue `NetworkBehaviourAction::SendEvent`s to peers we are
connected to, otherwise the actions die a silent death.

Another change implemented here is the use of protocol buffers as the
encoding for network messages. Certain individual fields of messages
are still SCALE encoded. There has been some discussion about this
in another PR (https://github.com/paritytech/substrate/pull/3452), so
far without resolution.

* Uncomment `Behaviour::light_client_request`.

* Add license headers.
2020-02-12 11:50:52 +01:00
Shawn Tabrizi 173644c8b9 Do not allow zero Existential Deposit when using Balances (#4894)
* Add non-zero ed check on Balances genesis

* Update ED from 0 to 1

* bump impl

* bump spec

* Found remove more ed = 0

* Fix some contract tests

* Use ctx.overlay.set_balance for contracts

* Fix staking test

* Remove obsolete logic

* Allow death of payout account in society

* Update frame/balances/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Dont create genesis balances if balance is zero in transaction payment pallet

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
2020-02-12 11:29:30 +01:00
Toralf Wittner 818c05bbc2 Add command-line flag to enable yamux flow control. (#4892)
* Add command-line flag to enable yamux flow control.

We never enabled proper flow-control for yamux streams which may cause
stream buffers to exceed their configured limit when the stream
producer outpaces the stream consumer. By switching the window update
mode to on-read, producers will only receive more sending credit when
all data has been consumed from the stream buffer. Using this option
creates backpressure on producers. However depending on the protocol
there is a risk of deadlock, if both endpoints concurrently attempt to
send more data than they have credit for and neither side reads before
finishing their writes. To facilitate proper testing, this PR adds a
command-line flag `use-yamux-flow-control`.

* Replace comment with generic message.
2020-02-12 11:24:25 +01:00
Marcio Diaz ae70b10326 Benchmark Timestamp Pallet (#4891)
* Add selected_benchmark! macro.

* Use selected_benchmark! in Identity pallet.

* Implement timestamp pallet benchmark.

* Fix some nits.

* Bump impl_version.
2020-02-12 11:00:08 +01:00
Hayden Bakkum b34df420f8 Add a sub command to generate a node key file (#4884)
* Add a sub command to generate a node key file in the format required by a substrate node

* Update lock file

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Updates as per code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-02-11 21:17:08 +01:00
Pierre Krieger 9ecbb420f4 Increase the penality for being offline (#4889) 2020-02-11 20:38:56 +01:00
Wei Tang b7582a4645 pallet-evm: optional nonce parameter (#4893)
* pallet-evm: optional nonce parameter

* Consume all gases when nonce mismatches

* Bump node runtime version
2020-02-11 18:52:59 +01:00
Marcio Diaz a90b4fdb8c Refactor tx factory 1 (#4870)
* Remove modes.

* Refactor.
2020-02-11 13:59:52 +01:00
Pierre Krieger b0a749548d Pause Kademlia if too many connections (#4828)
* Pause Kademlia if too many connections

* Fix test

* Update client/network/src/discovery.rs

Co-Authored-By: Toralf Wittner <tw@dtex.org>

* Change the limit

Co-authored-by: Toralf Wittner <tw@dtex.org>
2020-02-11 12:22:25 +01:00
Robert Habermeier ea69238d03 impl Randomness trait for Babe and remove unused RandomBeacon trait (#4886)
* impl Randomness trait for Babe and remove unused RandomBeacon trait

* bump runtime version
2020-02-11 11:07:14 +01:00
Hayden Bakkum 0d45d12e3f Add support for json output in subkey (#4882)
* Add support for json output in subkey

* Updates as per code review

* Apply suggestions from code review

Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Clean up error handler as per code review

* Apply suggestions from code review

Co-Authored-By: Marcio Diaz <marcio@parity.io>

* Fix compilation error

* Remove accidental file commit

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
2020-02-11 10:35:58 +01:00
cheme da42d7408c Use prefixed iterator from trie. (#4858) 2020-02-11 10:00:56 +01:00
Tomasz Drwięga 05b3e1f654 Run offchain workers at hash, not number. (#4878)
* Run offchain workers at particular hash, not number.

* Don't run if not new best.

* Don't run if not new best.

* Update client/service/src/builder.rs

Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Update client/service/src/builder.rs

Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Update client/service/src/builder.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-02-11 00:41:59 +01:00
Nikolay Volf ef7d19ed93 update primitive types to 0.6.2 (#4866) 2020-02-10 22:21:38 +03:00
Bastian Köcher 973d2c7253 Fix runtime-interface tests on windows (#4805) 2020-02-10 14:49:42 +01:00
Marcio Diaz fa27c8145d Add trait to get module and call names. (#4854)
* Add trait to get module and call names.


Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-02-10 14:27:40 +01:00
Gavin Wood aff0445b24 Avoid challenging those that can't be suspended anyway (#4804)
* Merge branch 'gav-split-balanecs-vesting' into gav-upsub

# Conflicts:
#	Cargo.lock
#	cli/Cargo.toml
#	collator/Cargo.toml
#	primitives/Cargo.toml
#	runtime/common/Cargo.toml
#	runtime/common/src/claims.rs
#	runtime/kusama/Cargo.toml
#	runtime/polkadot/Cargo.toml
#	service/Cargo.toml

* Fix tests
2020-02-10 14:04:50 +01:00
Sergei Pepyakin 5aa5c71107 Refactor and document allocator (#4855)
* Clarify code a bit.

* Move code around.

* Introduce `Order`.

* Introduce `Link` structure.

* Get rid of ptr_offset

This is beneficial since ptr_offset is essentially makes us handle two different address spaces, global (i.e. `mem`) and heap local and without it things are becoming simpler.

* Rename PREFIX_SIZE to HEADER_SIZE.

This will come in the next commits.

* Introduce a separate `Memory` trait.

This is not necessary, but will come in handy for the upcoming changes.

* Rename `ptr` to `header_ptr` where makes sense.

* Introduce a `Header` type.

* Make `bump` dumber.

This allows us to pull `HEADER_SIZE` to see that we actually allocate `order.size() + HEADER_SIZE`.

* Clean up.

* Introduce a freelists struct.

* Update documentation.

* Make Sized requirement optional to make the PR truly back-compatible.

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-02-10 13:37:10 +01:00
Marcio Diaz 6a62ec9c75 Add trace on import block. (#4871) 2020-02-10 13:25:20 +01:00
Bastian Köcher 4254fbf56d Don't expose Benchmarking host functions by default (#4875)
* Don't expose `Benchmarking` host functions by default

* Fix tests

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-02-10 13:13:04 +01:00
Ashley ead6815ae4 Fix timer panics in the wasm light client (#4561)
* Make WASM browser thing compile

* Fix

* updated exit-future (github repo)

* Switch to broadcast crate

* Migrate client/cli

* Switch exit-future to modernize branch

* Small changes

* Switch to cargo version and fix fg tests

* fix basic-authorship

* Fix crash on grafana macro

* Fix grafana macro

* Switch node python version

* Disable record_metrics_slice in grafana macro on wasm

* Update client/grafana-data-source/src/lib.rs

* Revert "Update client/grafana-data-source/src/lib.rs"

This reverts commit 888009a8e0b7051bd4bfbbfdb0448bcf2e2aae93.

* Add wasm support for state machine

* Switch to my own libp2p version

* Revert "Switch to my own libp2p version"

This reverts commit ce613871b59264b3165b45c37943e6560240daa7.

* Revert "Add wasm support for state machine"

This reverts commit de7eaa0694d9534fc3b164621737968e9a6a7c5f.

* Add sc-browser

* Squash

* remove sc-browser

* Fix keystore on wasm

* stubs for removed functions to make env compatible with old runtimes

* Add test (that doesn't work)

* Fix build scripts

* Revert basic-authorship due to no panics

* Revert cli/informant

* Revert consensus

* revert offchain

* Update utils/browser/Cargo.toml

Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>

* export console functions

* Add new chainspec

* Fix ws in chain spec

* revert chainspec

* Fix chainspec

* Use an Option<PathBuf> in keystore instead of cfg flags

* Remove crud

* Only use wasm-timer for instant and systemtime

* Remove telemetry changes

* Assuming this is ok

* Add a KeystoreConfig

* Add stubs back in

* Update libp2p

* Revert "Add stubs back in"

This reverts commit 4690cf1882aa0f99f7f00a58c4080c8aa9b77c36.

* Remove commented js again

* Bump kvdb-web version

* Fix cli

* Switch branch on futures-timer

* Fix tests

* Remove sc-client test build in check-web-wasm because there isn't a good way to build futures-timer with wasm-bindgen support in the build

* Remove more things ^^

* Switch branch on futures-timer back

* Put DB io stats behind a cfg flag

* Fix things

* Don't timeout transports on wasm

* Update branch of futures-timer and fix bad merge

* Spawn informant

* Fix network test

* Fix delay resets

* Changes

* Fix tests

* use wasm_timer for transaction pool

* Fixes

* Switch futures-timer to crates

* Only diagnose futures on native

* Fix sc-network-test tests

* Select log level in js

* Fix syncing ;^)

* Allow disabling colours in the informant

* Use OutputFormat enum for informant

* MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master

* Update futures-diagnose

* Revert "MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master"

This reverts commit baa4ffc94fd968b6660a2c17ba8113e06af15548.

* Pass whole chain spec in start_client

* Get Instant::now to work in transaction pool again

* Informant dep reordering

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
2020-02-10 12:23:55 +01:00
Cecile Tonglet 34bf0caa05 Update trie-db to the latest (#4874) 2020-02-10 11:51:56 +01:00
Gavin Wood 78a8d9f3b6 Fix vesting logic (#4864)
* Fix vesting logic

* Bump runtime version

* Docs.
2020-02-10 10:30:42 +01:00
Shawn Tabrizi e5a7fcc8ea Create Benchmarking Setup for Identity Pallet #4695 (#4818)
* Starting

* closer

* Compiles!

* comments

* Create seperate mock

* Remove changes to test env

* Fix step calculation

* Add host function

* Add runtime api

* compiles

* Update to use offchain timestamp

* Gives a result

* added some CLI wip

* make generic

* Update instance

* Remove CLI stuff

* Remove last cli stuff

* undo more changes

* Update benchmarks

* Update Cargo.lock

* remove test

* Move loop out of runtime

* Benchmarking externalities

* Benchmarking state

* Implemented commit

* Make CLI work, move loop back into runtime

* Wipe resets to genesis

* Speedup benchmarks

* Use enum to select extrinsic within pallet

* CLI controls which module and extrinsic to call

* Select a pallet with cli

* Add steps and repeats to cli

* Output as CSV format

* Introduce benchmark pallet

* Append bench

* Use Results

* fix merge

* Clear Identity benchmark

* Bench request judgment and cancel request

* Add final benchmarks

* Fix CSV output

* Start cleaning up for PR

* Bump numbers in `wasmtime` integration tests.

* More docs

* Add rockdb feature to bench

* Fix formatting issues

* Add test feature to bench

* Add test feature to bench

* Add rocksdb feature flag

* Update bench.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
2020-02-10 10:23:08 +01:00
h4x3rotab 376deef36f Export GRANDPA AuthorityPair when full_crypto is enabled (#4872)
* Export crypto_full feature in primitives/finality-grandpa

* Export GRANDPA AuthorityPair when full_crypto is enabled
2020-02-10 09:26:09 +01:00
h4x3rotab 916a3392b2 Remove the unnecessary std dependency from 'blake2-rfc' (#4868) 2020-02-09 20:29:45 +01:00
h4x3rotab 62b60c32de Wrap runtime_print! macro in its own scope (#4867)
* Wrap runtime_print! macro in its own scope

Before this fix runtime_print! can be used only once in a scope
because of the name pollution.

* Bump runtime impl versioin (no logic change)
2020-02-09 20:11:53 +01:00
Nikolay Volf 147155359f Alter usage collection a bit (#4863)
* alter usage collection a bit

* replace with mutex
2020-02-09 13:11:22 +01:00
Alexander Popiak cb567d6b8b Rename StorageMap::exists to ::contains_key (Resolves #4839) (#4847)
* rename StorageMap::exists(key) to ::contains_key(key)

* bump impl_version
2020-02-08 20:31:35 +01:00
Cecile Tonglet 503bfc9da9 WIP: Test: does cargo update work? (#4844)
* Initial commit

Forked at: be075893b5
Parent branch: origin/master

* Update Cargo.lock

* Cargo.lock

* Add missing features of libsecp256k1

* Update dependencies
2020-02-07 17:38:13 +01:00
Shawn Tabrizi 164f620918 Implement From<DispatchError> for '&static str (#4856) 2020-02-07 16:25:20 +01:00
Bastian Köcher ec562fe937 Fix CLI setup again (#4851)
* Fix CLI setup again

We need to set `config_dir` and `database_path` for almost every
command.
This fixes `purge-chain` and also adds a test to make sure we don't
break it again.

* Adds missing test files

* Split methods
2020-02-07 15:00:51 +01:00
Cecile Tonglet 60bd49ff0d Update parity-multiaddr dependency (#4852) 2020-02-07 13:22:06 +01:00
Bastian Köcher 9277a53052 Move tracing cli args to ImportParams (#4850) 2020-02-07 13:21:46 +01:00
Wei Tang bf437cda8e pow: re-add support for algorithms where only linear verification is possible (#4843)
* pow: re-add support for algorithms where only linear verification is possible

* Remove unused generic parameters

* Clone impl for PowBlockImport
2020-02-07 11:58:31 +01:00
Nikolay Volf 41967186e4 Subsystems memory tracking: 1. Transaction pool (#4822)
* update sp-runtime

* total update

* usage informant

* update to crates.io version

* update Cargo.lock

* update dummy update

* fix todo

* cleanup

* avoid custom impl

* Update client/transaction-pool/graph/src/future.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* remove another custom impl

* remove another custom impl

* add kb in report

* update Cargo.lock

* review suggestions

* --amend

* --amend

* bump parity-util-mem to 0.5.0

* bumps

* update macro and versions

* add to grafana

* naming

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-02-07 11:53:11 +01:00
Marcio Diaz 41644c2ae8 Enable trace timings logs for transaction factory (#4845)
* Enable trace timings logs for transaction factory.
2020-02-07 08:12:19 +01:00
Nikolay Volf 8c373916a4 easy with threads (#4848) 2020-02-06 23:32:48 +01:00
Wei Tang 14b0c9a746 babe: pass epoch data via intermediates (#4807)
* babe: pass epoch data via intermediates

* Switch to use Box<dyn Any> for intermediates

* Set intermediate.epoch to be Option

* Fix proposer should put out an empty intermediate

* Remove unnecessary encode/decode

* Add EpochData to block_import_params in slot worker

* Fix aura compile

* Fix integration test
2020-02-06 17:52:30 +00:00
Wei Tang 4df27e760e Refactor epoch changes to a separate crate (#4785)
* Init epoch changes module

* Initial integration of new epoch changes module for BABE

* Fix all initial compile errors

* rename: digest -> digests

* Fix babe tests

* Bump impl_version

* Fix more test issues

* Remove test flag for tree

It unfortunately won't work for multiple crates.

* Update cargo lock

* Fix duplicate parking_lot version

* Add missing license header
2020-02-06 16:48:38 +01:00