Commit Graph

198 Commits

Author SHA1 Message Date
dependabot[bot] bae12f41b8 Bump structopt from 0.3.23 to 0.3.25 (#10253)
* Bump structopt from 0.3.23 to 0.3.25

Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.23 to 0.3.25.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.23...v0.3.25)

---
updated-dependencies:
- dependency-name: structopt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* cargo fmt

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2021-11-13 02:35:07 +00:00
cheme ca5b07243f Fast sync child trie support. (#9239)
* state machine proofs.

* initial implementation

* Remove todo.

* Extend test and fix import.

* fix no proof, with proof ko.

* fix start at logic.

* Restore response size.

* Rework comments.

* Add explicit ref

* Use compact proof.

* ref change

* elaborato on empty change set condition.

* KeyValueState renaming.

* Do not add two time child trie with same root to sync reply.

* rust format

* Fix merge.

* fix warnings and fmt

* fmt

* update protocol id to V2
2021-11-07 13:13:02 +00:00
Bastian Köcher b08d3017a8 Switch to Rust 2021 (#10170)
* Switch to Rust 2021

* Update trybuild to fix errors
2021-11-04 19:54:48 +01:00
Arkadiy Paronyan e6ff531d0b Warp sync part II (#9284)
* Gap sync

* Gap epoch test

* Simplified network requests

* Update client/db/src/utils.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Fixed v1 migration and added some comments

* Next epoch is always regular

* Removed fork tree change

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Added a comment and converted assert to error

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-10-07 09:31:39 +00:00
Liu-Cheng Xu 00973f5b62 Clean up sc-peerset (#9806)
* Clean up sc-peerset

* cargo +nightly fmt --all

* Nit

* Nit

* .

* Nit

* .

* Apply suggestions from code review

* .

* Update client/peerset/src/peersstate.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-10-01 09:43:55 +00:00
Bastian Köcher 62187b5916 Prepare UI tests for rust 1.55 (#9637)
* Prepare UI tests for rust 1.54

* Delete wrong_page.stderr

* CI: run with a staging CI image

* Revert "CI: run with a staging CI image"

This reverts commit 66f5b00d14b50fd9d8fbf773f7e884f380697591.

* CI: debug, again

* LOG_TARGET is only used in std

* Remove unnecessary unsafe

* Fixes

* Use correct rustc locally

* FMT

* Compile with benchmarking

* Review feedback

* Some ui tests

* I know...

* Fix wasm tests

Co-authored-by: Denis P <denis.pisarev@parity.io>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2021-09-24 10:42:08 +00:00
Liu-Cheng Xu 2562f8c65e Clean up sc-network (#9761)
* Clean up sc-network

- Avoid using clone() for the Copy type `PeerId`.
- Use `find_map` for `filter_map` and `next`.
- Use `Self`.

* More on Copy types

* Cargo +nightly fmt --all

* More ..

* fmt

* Revert vec![default_notif_handshake_message]
2021-09-13 20:11:29 +02:00
Arkadiy Paronyan cd19c7b79e Fixed block response limit check (#9692)
* Fixed block response limit check

* Fixed start block detection and added a test

* Missing test
2021-09-06 07:25:30 +00:00
David 2de7e51c2a Remove substrate-in-the-browser (#9541)
* Comment out browser stuff

* Remove browser stuff

* Remove more wasm transport code

* Remove ExtTransport and rework how telemetry initialises.

* Change (most) wasm-timer using code to use std::time

* Rename CI-job

* Aura does not compile for wasm

* Remove testing in the browser on CI

* Update README

* Leave `StreamSink` be

* fmt
2021-08-17 18:06:23 +00:00
Trevor Arjeski d9f02296ab RUSTSEC-2021-0076 bump libsecp256k1 (#9391)
* RUSTSEC-2021-0076 bump libsecp256k1

libsecp256k1 allows overflowing signatures
https://rustsec.org/advisories/RUSTSEC-2021-0076

Changes were made to conform to libsecp256k1 version differences.

Closes #9356

* parse_standard_slice() -> parse_overflowing_slice()

* Added v2 host function for ecdsa_verify

* Add feature tag over helpers

* Added ecdsa_verify v2 to test runner

* PR feedback

- Spaces -> tabs
- renamed two helper functions

* Fixed imports after rebasing

* Bump rest of libsecp256k1 (and libp2p)

libp2p also uses libsecp256k1 so it is required to be bumped too, along
with all the version difference changes.

* Add version2 for ecdsa pubkey recovery

* libp2p rebase master fixes

* Fix test panic when non Behaviour event is returned

* Update bin/node/browser-testing/Cargo.toml

* Update primitives/core/src/ecdsa.rs

* Update primitives/core/src/ecdsa.rs

* Update Cargo.lock

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-08-16 14:17:10 +00:00
Squirrel 7e9b8d278e depend-o-pocalipse (#9450)
Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.  
Updated code related to deprecated warning on tracing-subscriber `scope()` 
( See https://github.com/tokio-rs/tracing/issues/1429 )
2021-08-13 15:18:37 +01:00
Kian Paimani abd08e29ce Rewrap all comments to 100 line width (#9490)
* reformat everything again

* manual formatting

* last manual fix

* Fix build
2021-08-11 14:56:55 +00:00
Arkadiy Paronyan f56232602f Warp sync part I (#9227)
* Started warp sync

* BABE & GRANDPA recovery

* Warp sync protocol

* Sync warp proofs first

* Added basic documentation

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Style changes

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* fmt

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fixed chage trie pruning wrt missing blocks

* Restore parent finalization

* fmt

* fmt

* Revert pwasm-utils bump

* Change error type & check API version

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Build fix

* Fixed target block check

* Formatting

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-08-02 08:27:18 +00:00
Squirrel df59596ec0 Move client consensus parts out of primitives and into client/consensus/api (#9319)
* moved client code out of primitives

* bump ci

* Fixup from merge.

* Removed unused deps thanks to review feedback

* Removing unneeded deps

* updating lock file

* note about rustfmt

* fixed typo to bump ci

* Move lonely CacheKeyId to parent

* cargo fmt

* updating import style

* Update docs/STYLE_GUIDE.md

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-07-30 13:27:17 +00:00
Squirrel aafe64315a Remove extra commas made redundent after rustfmt (#9404)
* Remove extra commas made redundent after rustfmt
2021-07-22 11:06:17 +01:00
Bastian Köcher 7b56ab15b4 Run cargo fmt on the whole code base (#9394)
* Run cargo fmt on the whole code base

* Second run

* Add CI check

* Fix compilation

* More unnecessary braces

* Handle weights

* Use --all

* Use correct attributes...

* Fix UI tests

* AHHHHHHHHH

* 🤦

* Docs

* Fix compilation

* 🤷

* Please stop

* 🤦 x 2

* More

* make rustfmt.toml consistent with polkadot

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-07-21 14:32:32 +00:00
Arkadiy Paronyan 5a65bf5515 Storage chains sync (#9171)
* Sync storage chains

* Test

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Separate block body and indexed body

* Update client/db/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-07-17 10:58:37 +02:00
Squirrel 20347c2edb Use MIN associated const (#9199) 2021-06-24 15:48:39 +01:00
Squirrel ea1f21a904 Use MAX associated const (#9196)
* Use MAX associated const
2021-06-24 11:53:49 +01:00
Arkadiy Paronyan 77a4b980ae Fast sync (#8884)
* State sync

* Importing state fixes

* Bugfixes

* Sync with proof

* Status reporting

* Unsafe sync mode

* Sync test

* Cleanup

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* set_genesis_storage

* Extract keys from range proof

* Detect iter completion

* Download and import bodies with fast sync

* Replaced meta updates tuple with a struct

* Fixed reverting finalized state

* Reverted timeout

* Typo

* Doc

* Doc

* Fixed light client test

* Fixed error handling

* Tweaks

* More UpdateMeta changes

* Rename convert_transaction

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Code review suggestions

* Fixed count handling

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-06-22 09:32:43 +00:00
André Silva c44b552d8f consensus: handle justification sync for blocks authored locally (#8698)
* consensus: add trait to control justification sync process

* network: implement JustificationSyncLink for NetworkService

* slots: handle justification sync in slot worker

* babe: fix slot worker instantiation

* aura: fix slot worker instantiation

* pow: handle justification sync in miner

* babe: fix tests

* aura: fix tests

* node: fix compilation

* node-template: fix compilation

* consensus: rename justification sync link parameter

* aura: fix test compilation

* consensus: slots: move JustificationSyncLink out of on_slot
2021-06-04 21:31:06 +00:00
Pierre Krieger a7a326ee07 Make sure nodes don't hammer each other even when reserved (#8901)
* Make sure nodes don't hammer each other even when reserved

* Make the ban random
2021-05-25 11:03:35 +00:00
Pierre Krieger d849110c6b Small improvement to network log line (#8811)
* Small improvement to network log line

* Use {:?} for the set_id
2021-05-14 14:31:03 +00:00
Arkadiy Paronyan 3c0270fe57 Maintain fork_targets (#8791)
* Maintain fork_targets

* Added a test
2021-05-13 13:53:30 +00:00
Pierre Krieger 62650989a3 Allow fallback names for protocols (#8682)
* Allow fallback names for protocols

* Apply suggestions from code review

Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>

* Fix some issues

* Fix compilation after merging master

Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
2021-05-06 14:01:01 +00:00
Pierre Krieger 747beffe70 Fix too generous error detection in behaviour.rs (#8717) 2021-05-03 17:18:35 +00:00
Roman Proskuryakov ef07c3be0d Add a JSON-RPC layer for reserved nodes (#8704)
* Add boilerplate for JSON-RPC layer for reserved nodes

* Add more boilerplate for JSON-RPC layer for reserved nodes

* Make JSON-RPC layer for reserved nodes async

* Use more realistic data in reserver_peers tests

* Make JSON-RPC layer for reserved nodes blocking

* Apply tomaka's suggestion to reduce .into_iter() for an iter

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2021-05-03 12:17:57 +00:00
Jon Häggblad d3d02d8851 Support sending and receiving multiple Justifications (#8266)
* client/network: support sending multiple justifications

* network: flag support for multiple justifications in protobuf request

* Update client/network/src/protocol.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* network: update comment on protobuf field

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2021-04-28 10:59:50 +00:00
Pierre Krieger 54ae55f439 Revert "Properly close notification substreams (#8534)" (#8646)
This reverts commit 6c9c687a31.

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-04-20 18:39:07 +00:00
Pierre Krieger d64f79924a Update to libp2p 0.37 (#8625)
* Update to libp2p 0.37

* Line widths

* Fix tests
2021-04-18 08:04:45 +00:00
Pierre Krieger 6c9c687a31 Properly close notification substreams (#8534)
* Properly close notification substreams

* Some debug asserts

* Fix state inconsistency

* Remove erroneous debug_assert!

* Some comments
2021-04-15 10:42:47 +00:00
Arkadiy Paronyan e3c1fe299a Drain blocks on peer disconnect (#8553)
* Drain blocks on peer disconnect

* Finish comment

* Fixed test

* Update client/network/src/protocol/sync.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-04-07 15:43:24 +03:00
Pierre Krieger b16bc0552e Clean up log levels in sc_network (#8529)
* Clean up log levels in sc_network

* Fix imports
2021-04-05 18:18:47 +00:00
Arkadiy Paronyan 484a630db9 Fixed sync skipping some block announcements (#8459)
* Fixed sync missing some block announcements

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-04-01 00:02:31 +03:00
Arkadiy Paronyan 2ed268286e Fix sync restart (#8497) 2021-03-31 10:28:52 +03:00
Bastian Köcher 217c4be226 Make BlockImport and Verifier async (#8472)
* Make grandpa work

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make it async

* Fix fix

* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* Make grandpa tests compile

* More Babe tests work

* Fix network test

* Start fixing service test

* Finish service-test

* Fix sc-consensus-aura

* Fix fix fix

* More fixes

* Make everything compile *yeah*

* Fix build when we have Rust 1.51

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix warning

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-03-30 09:19:49 +00:00
Pierre Krieger fb922e2794 Refactor NotifsHandler::poll (#8422)
* Refactor a bit NotifsHandler::poll

* Avoid some spurious wake-ups
2021-03-23 10:02:07 +00:00
Pierre Krieger 5087634ccd Rename GenericProto to Notifications (#8415)
* Rename GenericProto to Notifications

* Small comment fix
2021-03-22 10:32:47 +00:00
Jon Häggblad 0d6884b919 Storing multiple Justifications per block (#7640)
* primitives/runtime: initial changes on supporting multiple Justifications

* primitives/runtime: make Justifications strongly typed

* Encode/decode Justifications

* primitives/runtime: add Justification type

* backend: apply_finality and finalize_block takes a single Justification

* manual-seal: create engine id and let rpc take encoded justification

* backend: skeleton functions for appending justifications

* backend: initial implementation append_justification

Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k

* backend: guard against duplicate consensus engine id

* client/db: add check for block finality

* client/api: add append_justification to in_mem db

* client/light: add no-op append_justification

* network: fix decode call for Justification

* network: only send a single Justification in BlockData

* network: minor comment update

* protocol: update field names to distinguish single justification

* client: further field renames to plural

* client: update function names to plural justifications

* client/db: upgrade existing database for new format

* network: remove dependency on grandpa crate

* db: fix check for finalized block

* grandpa: check for multiple grandpa justifications hwne importing

* backend: update Finalizer trait to take multiple Justifications

* db: remove debugging statements in migration code

* manual-seal: update note about engine id

* db: fix check for finalized block

* client: update variable name to reflect it is now plural

* grandpa: fix incorrect empty Justications in test

* primitives: make Justifications opaque to avoid being empty

* network: fix detecting empty Justification

* runtime: doc strings for Justifications functions

* runtime: add into_justifications

* primitives: check for duplicates in when adding to Justifications

* network/test: use real grandpa engine id in test

* client: fix reviewer comments

* primitives: rename Justifications::push to append

* backend: revert changes to Finalizer trait

* backend: revert mark_finalized

* backend: revert changes to finalize_block

* backend: revert finalized_blocks

* db: add a quick early return for performance

* client: minor reviewer comments

* service/test: use local ConsensusEngineId

* network: add link to issue for sending multiple Justifications

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* network: tweaks to review suggestions

* network: revert change to BlockData for backwards compatibility

* Apply suggestion from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* primitives: update doc comment for Justifications

* client/db/upgrade: avoid grandpa crate dependency

* consensus: revert to single Justification for import_justification

* primitives: improve justifications docs

* style cleanups

* use and_then

* client: rename JUSTIFICATIONS db column

* network: revert to using FRNK in network-test

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-03-17 21:18:16 +00:00
Pierre Krieger 39f3b77f4b Fix #8306 (#8330) 2021-03-11 19:07:52 +00:00
Pierre Krieger 5f004b4428 Remove legacy network protocol (#8296) 2021-03-11 13:57:06 +00:00
Pierre Krieger d5b308f13c Fix networking debug_asserts (#8200)
* Fix networking debug_asserts

* Fix comment
2021-02-25 13:51:29 +01:00
Pierre Krieger f6273f7914 Remove all code related to sentry nodes (#8079)
* Remove all code related to sentry nodes

* More fixing
2021-02-15 08:45:03 +00:00
Benjamin Kampmann 8e36d87ca8 Update dependencies ahead of next release (#8015)
Updates dependencies:
parity-db 0.2.2
paste
prometheus 0.11
cfg-if 1.0
strum 0.20
env_logger 0.8
pin-project
prost
nix
platforms
quickcheck 1.0
2021-02-04 19:17:42 +01:00
Bastian Köcher d40f9c166f Sync: Fix issue of not freeing a block announcement slot (#8006)
* Sync: Fix issue of not freeing a block announcement slot

There was a bug that when the block announcement validation returned an
error, the slot reserved for this validation wasn't freed. This could
lead to a situation where we rejected any block announcement from such a
peer for that the block announcement returned an error multiple times.

* Better logging

* Fuck I'm dumb

* 🤦
2021-01-29 16:29:25 +00:00
Guillaume Thiolliere f48296e9ad Update parity-scale-codec to 2.0 (#7994)
* update cargo.toml

* use 2.0 in mmmr
2021-01-29 12:22:45 +00:00
Pierre Krieger dc70c889ce Introduce sc_peerset::DropReason (#7996)
* Introduce sc_peerset::DropReason

* Fix peerset tests
2021-01-29 09:33:27 +00:00
Bastian Köcher 450b96c50d Sync: Propagate block announcement data (#7903)
* Sync: Propagate block announcement data

This pr adds a feature to the sync protocol to propagate the data that
we received alongside a block announcement. This is done by adding a
cache that caches the last X block announcement data where X is set to
the number of `in_peers` (giving every peer the chance to send us a
different block). This will be required by parachains to ensure that
even peers who are not connected to a collator receive the data
alongside the block announcement to properly validate it and request the
block.

* Review comment

* Bring back the code and add new variant to ensure we don't insert block
announce data when something wasn't checked

* Also use out_peers
2021-01-19 17:01:11 +01:00
Pierre Krieger 8e04515912 Add explicit limits to notifications sizes and adjust yamux buffer size (#7925)
* Add explicit limits to notifications sizes and adjust yamux buffer size

* Docfix

* Tests

* Document these 10 bytes
2021-01-19 11:00:37 +00:00
Pierre Krieger 7db15cfb75 Fix not restoring non-poisoned state (#7906) 2021-01-15 12:47:32 +00:00