Commit Graph

431 Commits

Author SHA1 Message Date
Benjamin Kampmann c78c2892a4 Attempting to fix publishing (#9140)
* mark template and utils as non-publish

* switch to development version for testing

* activate unleash check

* maybe if I disable all rules...

* Fix isolated compilation of `max-encoded-len-derive` with `syn`

error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
  --> src/lib.rs:88:29
   |
88 |             .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
   |                            --------- ^^ ----------------------------------- _
   |                            |
   |                            syn::Path

error: aborting due to previous error

For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`

* WIP: bump changes crates since v3 tag to next breaking

cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0

FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`

* Update lockfile

* WIP: Bump sp-transaction-pool as well

* WIP: Bump sp-offchain as well

* WIP: Bump frame-system-rpc-runtime-api as well

* WIP: Bump sp-authority-discovery as well

* Manually deactivate dev-deps before `cargo unleash check`

Otherwise we run into `Cycle detected` error.

* Bump sp-consensus-slots

* Add missing Cargo.lock change

* Bump sp-consensus-vrf as well

* Bump sp-keyring as well

* Bump sp-consensus-pow as well

* Try to speed up the `unleash-check` job

Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help

* fixup: Ensure the temp target dir exists for unleash check

* Bump pallet-transaction-payment-rpc-runtime-api as well

Needed for Polkadot

* Bump pallet-transaction-payment-rpc as well

Needed for Polkadot

* Try updating crates after patching in the Polkadot CI job

* Use another approach to update patched Substrate crates

* Try to update all sp-core versions in Polkadot CI job

* Simplify sp-core version checking

* Apply another shellcheck lint

* Just do the simplest thing I guess

* Welp don't do --offline then

* Clean up `unleash-check` job triggers

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Fix a note in unleash-check cache step

* Add a note about temporary optimization in cargo-unleash

* Pin a newer version of cargo-unleash

Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2021-07-11 12:17:53 +00:00
Squirrel 20347c2edb Use MIN associated const (#9199) 2021-06-24 15:48:39 +01:00
Squirrel ea1f21a904 Use MAX associated const (#9196)
* Use MAX associated const
2021-06-24 11:53:49 +01:00
Arkadiy Paronyan 77a4b980ae Fast sync (#8884)
* State sync

* Importing state fixes

* Bugfixes

* Sync with proof

* Status reporting

* Unsafe sync mode

* Sync test

* Cleanup

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* set_genesis_storage

* Extract keys from range proof

* Detect iter completion

* Download and import bodies with fast sync

* Replaced meta updates tuple with a struct

* Fixed reverting finalized state

* Reverted timeout

* Typo

* Doc

* Doc

* Fixed light client test

* Fixed error handling

* Tweaks

* More UpdateMeta changes

* Rename convert_transaction

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Code review suggestions

* Fixed count handling

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-06-22 09:32:43 +00:00
André Silva a26ec52450 make SelectChain async (#9128)
* make SelectChain async

* make JustificationImport async
2021-06-20 11:01:09 +00:00
André Silva c44b552d8f consensus: handle justification sync for blocks authored locally (#8698)
* consensus: add trait to control justification sync process

* network: implement JustificationSyncLink for NetworkService

* slots: handle justification sync in slot worker

* babe: fix slot worker instantiation

* aura: fix slot worker instantiation

* pow: handle justification sync in miner

* babe: fix tests

* aura: fix tests

* node: fix compilation

* node-template: fix compilation

* consensus: rename justification sync link parameter

* aura: fix test compilation

* consensus: slots: move JustificationSyncLink out of on_slot
2021-06-04 21:31:06 +00:00
Pierre Krieger 152587bcb9 Discard notifications if we have failed to parse handshake (#8806) 2021-06-02 20:41:21 +02:00
Squirrel dc88e6dd7e Reduce cargo doc warnings (#8947)
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-06-01 10:26:10 +01:00
Roman Proskuryakov 587ea2549b Refactor code a little bit (#8932) 2021-05-31 16:05:50 +00:00
Roman Proskuryakov 8dfb8cd978 Remove: (#8748)
* `NetworkStatusSinks`
* `sc_service::SpawnTasksParams::network_status_sinks`

Also:
* `sc_service::build_network()` does not return `network_status_sinks`
2021-05-27 12:54:37 +00:00
André Silva 61507e1ba3 grandpa: enable light clients to participate in gossip (#8796)
* network: allow gossiping to light clients

* grandpa: gossip global messages to light clients

* grandpa: don't send neighbor packets to light clients

* grandpa: fix tests

* grandpa: export run_grandpa_observer

* node: run grandpa observer on light client

* node: start network at end

* Use wasm_timer in finality-grandpa

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2021-05-26 11:33:11 +00:00
Pierre Krieger a7a326ee07 Make sure nodes don't hammer each other even when reserved (#8901)
* Make sure nodes don't hammer each other even when reserved

* Make the ban random
2021-05-25 11:03:35 +00:00
Pierre Krieger d849110c6b Small improvement to network log line (#8811)
* Small improvement to network log line

* Use {:?} for the set_id
2021-05-14 14:31:03 +00:00
Pierre Krieger 3b1b339b80 Add more debug_assert to networking (#8807)
* Add more debug_assertsudo dnf update to networking

* Swap the lines
2021-05-14 14:17:10 +00:00
Arkadiy Paronyan 3c0270fe57 Maintain fork_targets (#8791)
* Maintain fork_targets

* Added a test
2021-05-13 13:53:30 +00:00
Pierre Krieger d443a0b929 Don't remove addresses from known_external_addresses (#8778)
* Don't remove addresses from known_external_addresses

* Remove `remove`
2021-05-11 10:34:35 +00:00
Pierre Krieger f16f8def08 Increase the number of external IPs in cache (#8771) 2021-05-10 10:18:21 +00:00
Pierre Krieger 62650989a3 Allow fallback names for protocols (#8682)
* Allow fallback names for protocols

* Apply suggestions from code review

Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>

* Fix some issues

* Fix compilation after merging master

Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
2021-05-06 14:01:01 +00:00
Pierre Krieger 19d34eb87a Add some builder functions for NonDefaultSetConfig (#8712) 2021-05-04 09:04:49 +00:00
Pierre Krieger 747beffe70 Fix too generous error detection in behaviour.rs (#8717) 2021-05-03 17:18:35 +00:00
Roman Proskuryakov ef07c3be0d Add a JSON-RPC layer for reserved nodes (#8704)
* Add boilerplate for JSON-RPC layer for reserved nodes

* Add more boilerplate for JSON-RPC layer for reserved nodes

* Make JSON-RPC layer for reserved nodes async

* Use more realistic data in reserver_peers tests

* Make JSON-RPC layer for reserved nodes blocking

* Apply tomaka's suggestion to reduce .into_iter() for an iter

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2021-05-03 12:17:57 +00:00
Roman Proskuryakov be1dce5618 Fix reliance on non-empty NodeInfo::endpoints (#8684)
* Use as_deref instead of .map in Node::client_version

* Fix reliance on non-empty NodeInfo::endpoints
2021-04-29 13:37:05 +00:00
Jon Häggblad d3d02d8851 Support sending and receiving multiple Justifications (#8266)
* client/network: support sending multiple justifications

* network: flag support for multiple justifications in protobuf request

* Update client/network/src/protocol.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* network: update comment on protobuf field

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2021-04-28 10:59:50 +00:00
Pierre Krieger 54ae55f439 Revert "Properly close notification substreams (#8534)" (#8646)
This reverts commit 6c9c687a31.

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-04-20 18:39:07 +00:00
Pierre Krieger d64f79924a Update to libp2p 0.37 (#8625)
* Update to libp2p 0.37

* Line widths

* Fix tests
2021-04-18 08:04:45 +00:00
Pierre Krieger 6c9c687a31 Properly close notification substreams (#8534)
* Properly close notification substreams

* Some debug asserts

* Fix state inconsistency

* Remove erroneous debug_assert!

* Some comments
2021-04-15 10:42:47 +00:00
Arkadiy Paronyan e3c1fe299a Drain blocks on peer disconnect (#8553)
* Drain blocks on peer disconnect

* Finish comment

* Fixed test

* Update client/network/src/protocol/sync.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-04-07 15:43:24 +03:00
Pierre Krieger 29a9ba6346 Don't report confusing unsupported protocol errors (#8482) 2021-04-06 12:35:12 +00:00
Pierre Krieger b16bc0552e Clean up log levels in sc_network (#8529)
* Clean up log levels in sc_network

* Fix imports
2021-04-05 18:18:47 +00:00
Pierre Krieger 6092b56dda Adjust number of slots in sync requests channel (#8492) 2021-04-05 18:31:14 +02:00
Pierre Krieger 09aa03e3c5 Add a feedback when response is successfully sent (#8510)
* Add a feedback when response is successfully sent

* Fix gp warp sync
2021-04-01 16:25:37 +00:00
Arkadiy Paronyan 484a630db9 Fixed sync skipping some block announcements (#8459)
* Fixed sync missing some block announcements

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-04-01 00:02:31 +03:00
Arkadiy Paronyan 2ed268286e Fix sync restart (#8497) 2021-03-31 10:28:52 +03:00
Bastian Köcher 217c4be226 Make BlockImport and Verifier async (#8472)
* Make grandpa work

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make it async

* Fix fix

* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* Make grandpa tests compile

* More Babe tests work

* Fix network test

* Start fixing service test

* Finish service-test

* Fix sc-consensus-aura

* Fix fix fix

* More fixes

* Make everything compile *yeah*

* Fix build when we have Rust 1.51

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix warning

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-03-30 09:19:49 +00:00
Pierre Krieger 5d2640240c Fix &mut self -> &self in add_known_address (#8468) 2021-03-26 22:28:55 +01:00
Pierre Krieger e8aeadf202 Add NetworkService::add_known_address (#8467) 2021-03-26 17:02:41 +00:00
Pierre Krieger 1063fc7e35 Use Debug for genesis mismatch message (#8449) 2021-03-24 20:51:58 +01:00
Roman Borschel e1570b9ab4 Update to libp2p-0.36 (#8420)
* Update to libp2p-0.36

* Some more Cargo.lock updates.
2021-03-24 18:29:35 +00:00
André Silva ee8e7f1bcd grandpa: speed up tests (#8439)
* grandpa: tests: add peers with authority role

* grandpa: tests: manually wake-up poll_fn future
2021-03-24 09:56:25 +01:00
Pierre Krieger fb922e2794 Refactor NotifsHandler::poll (#8422)
* Refactor a bit NotifsHandler::poll

* Avoid some spurious wake-ups
2021-03-23 10:02:07 +00:00
Pierre Krieger 5087634ccd Rename GenericProto to Notifications (#8415)
* Rename GenericProto to Notifications

* Small comment fix
2021-03-22 10:32:47 +00:00
Arkadiy Paronyan 4a0d6d9490 Storage chains: indexing, renewals and reference counting (#8265)
* Transaction indexing

* Tests and fixes

* Fixed a comment

* Style

* Build

* Style

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Code review suggestions

* Add missing impl

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* impl JoinInput

* Don't store empty slices

* JoinInput operates on slices

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-03-18 11:46:27 +00:00
Jon Häggblad 0d6884b919 Storing multiple Justifications per block (#7640)
* primitives/runtime: initial changes on supporting multiple Justifications

* primitives/runtime: make Justifications strongly typed

* Encode/decode Justifications

* primitives/runtime: add Justification type

* backend: apply_finality and finalize_block takes a single Justification

* manual-seal: create engine id and let rpc take encoded justification

* backend: skeleton functions for appending justifications

* backend: initial implementation append_justification

Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k

* backend: guard against duplicate consensus engine id

* client/db: add check for block finality

* client/api: add append_justification to in_mem db

* client/light: add no-op append_justification

* network: fix decode call for Justification

* network: only send a single Justification in BlockData

* network: minor comment update

* protocol: update field names to distinguish single justification

* client: further field renames to plural

* client: update function names to plural justifications

* client/db: upgrade existing database for new format

* network: remove dependency on grandpa crate

* db: fix check for finalized block

* grandpa: check for multiple grandpa justifications hwne importing

* backend: update Finalizer trait to take multiple Justifications

* db: remove debugging statements in migration code

* manual-seal: update note about engine id

* db: fix check for finalized block

* client: update variable name to reflect it is now plural

* grandpa: fix incorrect empty Justications in test

* primitives: make Justifications opaque to avoid being empty

* network: fix detecting empty Justification

* runtime: doc strings for Justifications functions

* runtime: add into_justifications

* primitives: check for duplicates in when adding to Justifications

* network/test: use real grandpa engine id in test

* client: fix reviewer comments

* primitives: rename Justifications::push to append

* backend: revert changes to Finalizer trait

* backend: revert mark_finalized

* backend: revert changes to finalize_block

* backend: revert finalized_blocks

* db: add a quick early return for performance

* client: minor reviewer comments

* service/test: use local ConsensusEngineId

* network: add link to issue for sending multiple Justifications

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* network: tweaks to review suggestions

* network: revert change to BlockData for backwards compatibility

* Apply suggestion from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* primitives: update doc comment for Justifications

* client/db/upgrade: avoid grandpa crate dependency

* consensus: revert to single Justification for import_justification

* primitives: improve justifications docs

* style cleanups

* use and_then

* client: rename JUSTIFICATIONS db column

* network: revert to using FRNK in network-test

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-03-17 21:18:16 +00:00
Robert Klotzner def7b4dca2 Derive common classes for IfDisconnected. (#8346) 2021-03-12 20:54:44 +01:00
Bastian Köcher 3743cec9bd Do not ban peers for sending multiple valid requests (#8325)
We introduced banning of peers who spam us with the same request (more
than 2 times). However, we missed that it is completely legal to send
the same request multiple times as long as we did not provide any
answer. An example for that is the justification request. This request
is send multiple times until we could fetch the justification from one
of our peers. So, the solution to this problem is to tag requests as
fulfilled and to start counting these fulfilled requests. If the number
is higher than what we allow, the peer should be banned.
2021-03-12 11:13:44 +00:00
Pierre Krieger 39f3b77f4b Fix #8306 (#8330) 2021-03-11 19:07:52 +00:00
Pierre Krieger 5f004b4428 Remove legacy network protocol (#8296) 2021-03-11 13:57:06 +00:00
Bastian Köcher 85f32e8813 Decrease the peer reputation on invalid block requests (#8260)
* Decrease the peer reputation on invalid block requests

This pr changes the block request handler to decrease the reputation of
peers when they send the same request multiple times or they send us an
invalid block request.

* Review feedback

* Change log target

* Remove unused code
2021-03-09 14:43:19 +01:00
Guillaume Thiolliere 9c4e4d453c Fix doc build with --all-features (#8277)
* implement

* make default pre/post_upgrade

* simplify Cargo.toml

* revert removal of outdated/private links

* link in pallet-mmr
2021-03-05 16:08:44 +01:00
Pierre Krieger add5cbebb6 Fix state mismatch in case of bad handshake (#8230) 2021-03-01 18:57:00 +01:00