Commit Graph

16174 Commits

Author SHA1 Message Date
Cecile Tonglet 7e37e79317 Improved send_transaction helper to return an error in case of error (#6592) 2020-07-09 12:10:34 +02:00
Alan Sapede 20d403aebe Make evm errors public (#6598) 2020-07-09 09:51:21 +00:00
David Craven 4d209543b9 Fix build when with-kvdb-rocksdb is disabled. (#6546) 2020-07-08 22:46:53 -04:00
Robert Habermeier 151d73af5b Implement PoV Distribution Subsystem (#1344)
* introduce candidatedescriptor type

* add PoVDistribution message type

* loosen bound on PoV Distribution to account for equivocations

* re-export some types from the messages module

* begin PoV Distribution subsystem

* remove redundant index from PoV distribution

* define state machine for pov distribution

* handle overseer signals

* set up control flow

* remove `ValidatorStatement` section

* implement PoV fetching

* implement distribution logic

* add missing `

* implement some network bridge event handlers

* stub for message processing, handle our view change

* control flow for handling messages

* handle `awaiting` message

* handle any incoming PoVs and redistribute

* actually provide a subsystem implementation

* remove set-builder notation

* begin testing PoV distribution

* test that we send awaiting messages only to peers with same view

* ensure we distribute awaited PoVs to peers on view changes

* test that peers can complete fetch and are rewarded

* test some reporting logic

* ensure peer is reported for flooding

* test punishing peers diverging from awaited protocol

* test that we eagerly complete peers' awaited PoVs based on what we receive

* test that we prune the awaited set after receiving

* expand pov-distribution in guide to match a change I made

* remove unneeded import
2020-07-08 18:15:39 -04:00
Gavin Wood f83ba174f1 Bump Substrate (#1382) 2020-07-08 18:48:48 +02:00
Pierre Krieger 647e303df5 Update the Kusama bootnode IDs (#1377) 2020-07-08 18:24:38 +02:00
Shawn Tabrizi 099e41dc9e Companion for #6575 (Add WeightInfo) (#1352)
* Add `WeightInfo`

* fix test

* fix tests

* fix more tests
2020-07-08 18:22:22 +02:00
Shawn Tabrizi 2302898b8a Add WeightInfo to all pallets with benchmarks. (#6575)
* Start adding weight info

* More weightinfo

* finish weight info

* more fixes

* inital update of node runtime

* fix the rest of the compilation

* update balances

* add docs

* fix balances tests

* Fix more tests

* Fix compile

* Fix pallet-evm tests
2020-07-08 18:22:01 +02:00
Bastian Köcher 283ffd4ec4 Companion for #6599 (#1371)
* Companion for #6599

https://github.com/paritytech/substrate/pull/6599

* Update Substrate ref
2020-07-08 16:15:59 +00:00
Bastian Köcher 94cddee160 Improve transaction submission (#6599)
* Improve transaction submission

Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.

Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.

* Make sure we don't submit the same transaction twice from the network concurrently

* Remove added listener call

* Feedback

* Ignore banned on resubmit
2020-07-08 17:42:42 +02:00
Bastian Köcher 51a698803c Use SpawnNamed to give tasks names (#1379) 2020-07-08 17:42:29 +02:00
Pierre Krieger fef3c181d9 Update to substrate#59ee76a0 (#1380) 2020-07-08 17:21:37 +02:00
Max Inden faa72caf91 *: Update to libp2p v0.21.1 (#6559)
* *Cargo.toml: Update versions

* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Adjust to one_shot.rs changes

* client/network/discovery: Log address list on trace level

* client/network/discovery: Ignore RoutablePeer and PendingRoutablePeer

* Commit Cargo.lock

* Finish update

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-07-08 14:00:30 +00:00
pscott acb5875e85 Companion PR for #6564 (#1350)
* Add log_rotation_opt

* Update Cargo lock

* Use SubstrateCli's log-directory instead of re-defining it
2020-07-08 14:05:41 +02:00
Pierre Krieger 8ca05879e8 Send Status message on all newly-opened legacy substreams (#6593)
* Send Status message on all newly-opened legacy substreams

* Fix tests
2020-07-08 11:44:51 +00:00
pscott 6eb2eb81c5 Add log rotation (#6564)
* Use flexi_logger; Add log rotation

* Add default rotation; Add FlexiLogger error

* Fix compilation error

* Remove logging to stdout if it's not a tty

* Fix formatting

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove needless debug statement

* Default to unlimited size for log rotation

* Add more comments about log-age option

* Remove unused variable

* Fix typo in comment

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-08 10:11:09 +00:00
Bastian Köcher 61ba45aaad Companion for #6584 (#1373)
* Companion for #6584

https://github.com/paritytech/substrate/pull/6584

* updating sp-io

* Update dev-dependency, too

* update lockfile

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
2020-07-08 11:42:46 +02:00
Max Inden 62f306d972 client/network: Rename DebugInfoBehaviour to PeerInfoBehaviour (#6556)
Information retrieved via `DebugInfoBehaviour` is not only used for
debugging purposes, e.g. disconnecting from nodes not responding to
pings, using external addresses retrieved via indentify, ...

In order for the name to reflect the usage of the module this commit
renames it.
2020-07-08 09:08:47 +00:00
Bastian Köcher ce0b55ff09 Upgrade kvdb-*, trie-db and memory-db (#6584)
* Upgrade `kvdb-*`, `trie-db` and `memory-db`

The updates of `trie-db` and `memory-db` are important, as they fix the
non-deterministic build of Polkadot/Substrate.

* Change `trie-db` version

* Update test-utils/runtime/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update primitives/trie/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update `Cargo.lock` and `trie-bench`

* Fix UI tests

* Switch to fixed version of memory-db

Co-authored-by: Andronik Ordian <write@reusable.software>
2020-07-08 10:42:56 +02:00
Shaopeng Wang c9046d9686 Companion for #6500 (decl_module: frame_system as default ident) (#1314)
* Update decl_storage usage with frame_system as default ident.

* More decl_module usage update.

* 'Update substrate'

Co-authored-by: parity-processbot <>
2020-07-08 09:07:19 +02:00
Shaopeng Wang a6702b7121 decl_module! macro: use 'frame_system' instead of system as default ident (#6500)
* Use frame_system as default ident.

* Remove unused 'frame_system' to 'system' renaming.

* Fix construct_runtime_ui tests.

* Rename system to frame_system in sudo/utility pallet test.

* Bump runtime impl_version.

* Update formatting.
2020-07-07 23:06:21 +00:00
Robert Habermeier 42bd096413 I am dumb and can't spell (#1366)
* rename implementor's guide to implementer's guide

* fix typos in more places
2020-07-07 10:10:36 -04:00
Pierre Krieger 36cf6e7dc4 Use async/await in build_network_future (#6533)
* Use async/await in build_network_future

* Address concerns

* Fix test
2020-07-07 10:32:30 +00:00
Bastian Köcher 37da08a764 Companion for #6589 (#1367)
https://github.com/paritytech/substrate/pull/6589
2020-07-07 12:06:57 +02:00
Bastian Köcher cccf57a79f Fix UI tests (#6589) 2020-07-07 11:41:55 +02:00
Peter Goodspeed-Niklaus f3c2db1a28 convert SignedAvailabilityBitfields from newtype to typedef (#1342)
Closes #1339.
2020-07-07 10:01:15 +02:00
Cecile Tonglet 27c8654c57 Script to register a parachain (#137) 2020-07-06 17:29:01 +02:00
Robert Habermeier ac8e1ca206 Implement the Statement Distribution Subsystem (#1326)
* set up data types and control flow for statement distribution

* add some set-like methods to View

* implement sending to peers

* start fixing equivocation handling

* Add a section to the statement distribution subsystem on equivocations and flood protection

* fix typo and amend wording

* implement flood protection

* have peer knowledge tracker follow when peer first learns about a candidate

* send dependents after circulating

* add another TODO

* trigger send in one more place

* refactors from review

* send new statements to candidate backing

* instantiate active head data with runtime API values

* track our view changes and peer view changes

* apply a benefit to peers who send us statements we want

* remove unneeded TODO

* add some comments and improve Hash implementation

* start tests and fix `note_statement`

* test active_head seconding logic

* test that the per-peer tracking logic works

* test per-peer knowledge tracker

* test that peer view updates lead to messages being sent

* test statement circulation

* address review comments

* have view set methods return references
2020-07-06 11:16:17 -04:00
Bastian Köcher 72d0f09659 Companion for #6576 (#1351)
* Companion for #6576

* 'Update substrate'

Co-authored-by: parity-processbot <>
2020-07-06 13:36:00 +02:00
Shawn Tabrizi fe79b24692 Derive RuntimeDebug for Runtime (#6581) 2020-07-06 10:51:01 +00:00
Bastian Köcher ad2e832289 Don't require module name in inherents (#6576)
* Start

* Cleanup `construct_runtime!`

* Add tests

* Fix after merge

* Update the docs
2020-07-06 10:29:17 +00:00
Shawn Tabrizi 4af14c5191 Companion for #6567 (Benchmark Writer CLI) (#1349)
* Update usage of macro

* 'Update substrate'

* cargo update -p sp-io

Co-authored-by: parity-processbot <>
2020-07-06 10:06:02 +00:00
Shawn Tabrizi 2019f70768 Benchmarks Writer CLI (#6567)
* initial mockup

* add and wipe

* track writes

* start to add to pipeline

* return all reads/writes

* Log reads and writes from bench db

* causes panic

* Allow multiple commits

* commit before ending benchmark

* doesn't work???

* fix

* Update lib.rs

* switch to struct for `BenchmarkResults`

* add to output

* fix test

* line width

* @kianenigma review

* Add Whitelist to DB Tracking in Benchmarks Pipeline (#6405)

* hardcoded whitelist

* Add whitelist to pipeline

* Remove whitelist pipeline from CLI, add to runtime

* clean-up unused db initialized whitelist

* Add regression analysis to DB Tracking (#6475)

* Add selector

* add tests

* debug formatter for easy formula

* initial idea

* use all benchmarks

* broken

* working without trait

* Make work for multiple pallets

* Fix merge issues

* writer appends to file

* implement () for balances weight trait

* update name of trait

* Weights to WeightInfo

* auto trait writer

* Heap pages are configurable

* clean out runtime changes

* more clean up

* Fix string generation

* Update comments

* Update bin/node/runtime/src/lib.rs

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
2020-07-06 09:34:24 +00:00
Peter Goodspeed-Niklaus f2104562d8 implement custom proposer (#1320)
* network bridge skeleton

* move some primitives around and add debug impls

* protocol registration glue & abstract network interface

* add send_msgs to subsystemctx

* select logic

* transform different events into actions and handle

* implement remaining network bridge state machine

* start test skeleton

* make network methods asynchronous

* extract subsystem out to subsystem crate

* port over overseer to subsystem context trait

* fix minimal example

* fix overseer doc test

* update network-bridge crate

* write a subsystem test-helpers crate

* write a network test helper for network-bridge

* set up (broken) view test

* Revamp network to be more async-friendly and not require Sync

* fix spacing

* fix test compilation

* insert side-channel for actions

* Add some more message types to AllMessages

* introduce a test harness

* impl ProvideInherent for InclusionInherent

* reduce import churn; correct expect message

* move inclusion inherent identifier into primitives

It's not clear precisely why this is desired, but it's a pattern
I've seen in several places, so I'm going this to be on the
safe side. Worst case, we can revert this commit pretty easily.

* bump kusama spec_version to placate CI

* copy sc_basic_authorship::{ProposerFactory, Proposer}

We have from the problem description:

> This Proposer will require an OverseerHandle to make requests via.

That's next on the plate.

* use polkadot custom proposer instead of basic-authorship one

* add some tests

* ensure service compiles and passes tests

* fix typo

* fix service-new compilation

* Subsystem test helpers send messages synchronously

* remove smelly action inspector

* remove superfluous let binding

* fix warnings

* add license header

* empty commit; maybe github will notice the one with changes

* Update node/network/bridge/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* add sanity check to only include valid inherents

* stub: encapsulate block production mechanics instead of copying them

The goal is to end up with something like what's in
validation::block_production::*, which encapsulates
basic block production mechanics. This is a better idea than
just straight-up copying those mechanics.

* partial implementation of propose fn

Doesn't actually compile yet; need to bring in some other
commits to ensure ProvisionerMessage is a thing, and also
figure out how to get the block hash given the current
context.

* fix compilation

* clear a few more compile errors

* finish fn propose

* broken: add timeout to proposal

* add timeout to proposal

* guide: provisioner is responsible for selecting parachain candidates

* implement ProvisionerMessage::RequestInherentData & update fn propose

* impl CreateProposer::init; clean up

* impl std::error::Error for Error

* document error-handling rationale

* cause polkadot-service-new to compile correctly

* Move potentially-blocking call from fn init -> fn propose

This means that we can wrap the delayed call into the same
timeout check used elsewhere.

* document struct Proposer

* extract provisioner data fetch

This satisfies two requirements:

- only applies the timeout to actually fetching the provisioner data,
  not to constructing the block after
- simplifies the problem of injecting default data if we could not
  get the real provisioner data in time.

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
2020-07-05 19:22:52 +02:00
André Silva 19ef40a7c9 babe: enable equivocation reporting on all runtimes (#1330)
* enable BABE equivocation reporting on all runtimes

* runtime: fix parachains mock test runtime

* cargo update -p sp-io

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-07-04 15:19:58 +02:00
André Silva a9c21b8b84 babe: report equivocations (#6362)
* slots: create primitives crate for consensus slots

* offences: add method to check if an offence is unknown

* babe: initial equivocation reporting implementation

* babe: organize imports

* babe: working equivocation reporting

* babe: add slot number to equivocation proof

* session: move duplicate traits to session primitives

* babe: move equivocation stuff to its own file

* offences: fix test

* session: don't have primitives depend on frame_support

* babe: use opaque type for key owner proof

* babe: cleanup client equivocation reporting

* babe: cleanup equivocation code in pallet

* babe: allow sending signed equivocation reports

* node: fix compilation

* fix test compilation

* babe: return bool on check_equivocation_proof

* babe: add test for equivocation reporting

* babe: add more tests

* babe: add test for validate unsigned

* babe: take slot number in generate_key_ownership_proof API

* babe: add benchmark for equivocation proof checking

* session: add benchmark for membership proof checking

* offences: fix babe benchmark

* babe: add weights based on benchmark results

* babe: adjust weights after benchmarking on reference hardware

* babe: reorder checks in check_and_report_equivocation
2020-07-04 12:18:13 +02:00
Wei Tang 61635e75c1 pallet-evm: return Ok(()) when EVM execution fails (#6493)
* pallet-evm: return Ok(()) when EVM execution fails

* Bump spec version

* Init test module

* Add fail_call_return_ok test

* Fix tests and use full match pattern

Co-authored-by: Gav Wood <gavin@parity.io>
2020-07-04 11:57:50 +02:00
Bastian Köcher f77b8139c0 Make a collator send a collation as backup as well (#1353)
Currently a collator will only send a collation to validators it is a
primary for. While testing this could lead to the situation that the
same collator was registered as prime for all Parachain validators but
failed for other reasons to generate a PoVBlock. However no other
collator was sending a collation, which stopped the Parachain until the
faulty collator was stopped.

This pr solves this problem by making sure that every collator sends a
collation to one of his validators he is connected to, but registered as backup.
2020-07-03 20:10:47 -04:00
André Silva e3ba163714 ci: fix merge on polkadot companion job (#6574) 2020-07-03 21:07:11 +02:00
Gavin Wood f744d6911e Make the encoded-Call Vec<u8> explicitly so in metadata (#6566) 2020-07-03 19:29:35 +02:00
Cecile Tonglet 9c566f54b8 Remove polkadot companion detection from branch name (#6568)
* Initial commit

Forked at: 2ddbbd7c08
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.
2020-07-03 19:16:32 +02:00
André Silva fb2fa25331 primitives: use generic Header in testing runtime (#6561)
* primitives: use generic Header in testing runtime

* frame: remove unused imports

* Remove warning

Co-authored-by: Bastian Köcher <git@kchr.de>
2020-07-03 17:07:46 +00:00
Cecile Tonglet 2ddbbd7c08 Remove polkadot companion detection from branch name (#6565)
* Initial commit

Forked at: f54614e256
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.
2020-07-03 17:29:08 +02:00
Gavin Wood 79954ae589 Reduce Westend deposit requirements (#1341)
* Switch branch

* Return chain ops parts in new_chain_ops

* Remove where param from new_chain_ops

* Add task manager to new_chain_ops return

* Revert branch switch

* Revert "Revert branch switch"

This reverts commit 7c7900c047abd794ddc759aa092811db4961a7a6.

* network/test/src/lib: Adjust network worker polling

Companion for https://github.com/paritytech/substrate/pull/6552.

* Fix adder parachain

* Fix collator tests

* Revert branch switch

* Bump everything

- Remove old migration code
- Reduce deposit requried for westend

* Reapply fixes

* Bump locl

* Fix for #6550

* Fix message

Co-authored-by: Ashley Ruglys <ashley.ruglys@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
2020-07-03 15:35:36 +02:00
Cecile Tonglet f54614e256 New testing helpers (#6555)
* Initial commit

Forked at: 424d5c722d
Parent branch: origin/master

* Add send_transaction to RpcHandlers

* Extension trait for RpcHandlers

* Revert "Add send_transaction to RpcHandlers"

This reverts commit 03c89e13d404bae3f3123387dd50f026061bca82.

* Add an extension trait for BlockchainEvents

* Update test-utils/client/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update test-utils/client/src/lib.rs

* fix

* deps fix

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-07-03 09:49:42 +00:00
Bastian Köcher e7bb3aef39 Check candidate signatures before including them in set_heads (#1335)
* Check candidate signatures before including them in `set_heads`

This work around the bug described in: https://github.com/paritytech/polkadot/issues/1327

We check the candidate signatures in `create_inherent` to ensure that
all are valid. If one isn't valid, we don't include any candidate for
the current inherent.

* Bump runtimes
2020-07-03 10:02:55 +02:00
Max Inden 635310c8d4 client/network: Remove unused Result returned by NetworkWorker (#6552) 2020-07-02 15:18:08 +02:00
Bastian Köcher 4f7f312be5 Move create_inherents into the block-builder (#6553)
* Move `create_inherents` into the block-builder

This moves the `create_inherents` call into the block-builder. This has
the advantage that `create_inherents` will be able to reuse the same
context that will be used when applying the extrinsics and we also save
one call to `on_initialize`. To make sure that `create_inherents` does
not modify any state, we execute it in a transaction that is
rolled-back after doing the runtime call.

* Feedback and build fix

* Update primitives/runtime/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Update client/block-builder/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2020-07-02 15:17:14 +02:00
Gavin Wood aeae0e1422 Revert "Companion PR for Allow specify schedule dispatch origin #6387" (#1340) 2020-07-02 15:09:03 +02:00
Gav Wood b05d7413bf Merge branch 'xlc-update-scheduler' 2020-07-02 15:06:18 +02:00