Commit Graph

332 Commits

Author SHA1 Message Date
Arkadiy Paronyan c58a9b7f05 Don't update common block on ancient block import (#4073) 2019-11-10 13:35:15 +01:00
Pierre Krieger 401e213aa2 Add a --no-private-ipv4 CLI option (#4042)
* Add a --no-private-ipv4 CLI option

* Fix tests

* Fix tests
2019-11-10 11:14:36 +01:00
Arkadiy Paronyan 9defa6a281 Further fix for common block update (#4071) 2019-11-09 15:46:43 +01:00
Arkadiy Paronyan 4b8578e7e4 Fix sync downloading ancient chains (#4060)
* Update best block on announcement

* Added a test
2019-11-09 09:11:13 +01:00
Arkadiy Paronyan 2cd3b2bce1 Batch gossip messages (#4055) 2019-11-08 21:34:45 +01:00
André Silva a8ce80b72d grandpa: progressively increase target gossip peers (#4050)
* grandpa: stricter gossip message filtering

* gossip: remove filtered message on send_message

* gossip: add test for tracking of broadcast attempts

* grandpa: only restrict gossip if we're connected to more than 5 authorities

* grandpa: add test for progressive gossip

* grandpa: add test for gossip filtering on local non-authority node

* grandpa: fix doc

* gossip, grandpa: tabify

* grandpa: relax filtering logic for global messages
2019-11-08 20:08:14 +01:00
Arkadiy Paronyan dfded93411 Increase parallel downloads to 5 (#4045)
* Increase parallel downloads to 5

* CLI param
2019-11-08 11:24:46 +01:00
B YI 2125b87cae fix two typos (#4048) 2019-11-08 09:27:33 +01:00
Arkadiy Paronyan cca9ab436c Allow import withouth state verification (#4031)
* Allow import without state verification

* Explicit None

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>
2019-11-07 15:25:41 +01:00
Arkadiy Paronyan e73436d818 Decrease peer reputation on bad transactions (#4035)
* Decrease reputation on bad transactions

* Don't punish on duplicate transactions
2019-11-07 12:00:14 +01:00
Pierre Krieger cc09bfbd09 Apply breaking changes of new libp2p versions (#3877)
* Apply breaking changes of new libp2p versions

* Oops, forgot to update version

* Fix tests

* Fix imports with WASM

* Fix WASM for real

* Update core/network/src/debug_info.rs

Co-Authored-By: Roman Borschel <romanb@users.noreply.github.com>

* Fix compilation
2019-11-06 14:00:12 +01:00
Arkadiy Paronyan 1aaf31a34d Less verbose console output (#4029) 2019-11-06 10:06:11 +01:00
Gavin Wood 350e72dac5 Revert "Header-only sync for old forks (#3942)" (#4022)
This reverts commit 172359adad.
2019-11-05 16:05:36 +01:00
Arkadiy Paronyan 172359adad Header-only sync for old forks (#3942)
* Header-only sync for old forks

* Simplified blocks-count

* Update core/consensus/common/src/block_import.rs

Co-Authored-By: Marcio Diaz <marcio.diaz@gmail.com>
2019-11-05 12:17:12 +01:00
Arkadiy Paronyan b92674d88a Traffic statistics (#4017)
* Network stats

* Fixed tests
2019-11-05 12:15:59 +01:00
Arkadiy Paronyan 4ca6f8d1b2 Enable parallel block download (#4014) 2019-11-05 10:27:49 +01:00
Caio d5f61d5217 Implement Debug for some structures (#3941)
* Implement Debug for some structures

`NetworkConfiguration`, `TransportConfig`, `NodeKeyConfig` and `Secret`.

Needs a new release of the `rust-libp2p` crate. This PR is just a
reminder.

* Explicitly separate `std` and `core`

* Add manual implementation for Secret
2019-11-01 15:28:19 +01:00
Gavin Wood 26f4084f95 Add events for im_online (#3991)
* Add AllGood event for im_online

* Another event just in case.

* Bump runtime
2019-11-01 11:07:46 +00:00
Pierre Krieger eb3c950cfe Remove NetworkSpecialization::on_event (#3976) 2019-10-31 15:38:53 +00:00
Jim Posen a167f37b91 Create opaque struct for StorageProof. (#3834)
Passing around Vec<Vec<u8>> everywhere is gross and confusing and
breaks encapsulation.
2019-10-31 11:02:29 +01:00
Pierre Krieger fbe9cfad46 Too many addresses for a node is now a debug! rather than warn! (#3938)
* Too many addresses for a node is now a debug! rather than warn!

* I managed to fail this change
2019-10-28 13:12:31 +01:00
Max Inden 743a34bc1d core/finality-grandpa: Request block sync from network after import timeout (#3800)
* core/finality-grandpa: Pass Grandpa msg sender up to UntilImported

* core/finality-grandpa: Track senders to maybe later request blocks

* core/finality-grandpa: Make BlockStatus pub only within crate

* core/finality-grandpa: Abstract NetworkBridge with BlockSyncRequester

* core/finality-grandpa: Pass BlockSyncRequester to UntilImported

* core/finality-grandpa: Track block number of pending within UntilImported

* core/finality-grandpa: Request block sync on long wait

* core/finality-grandpa: Adjust unit tests to previous changes

* core/finality-grandpa: Fix line length

* core/finality-grandpa: Add comment explaining in & out vote combination

* core/finality-grandpa: Log after, not before, timeout expired

The UntilImported component should log whenever waiting for a specific
block to be imported surpassed a defined constant timeout. Without this
patch the code would log whenever the current time was below the
timeout.

* core/finality-grandpa: Collect senders as HashSet for deduplication

* Revert "core/finality-grandpa: Track senders to maybe later request blocks"

This reverts commit 61ac9dd715612d5fdbf7b8f00b84e450f282ade0.

* Revert "core/finality-grandpa: Pass Grandpa msg sender up to UntilImported"

This reverts commit afdc9646a6c314f99a9d19242f1878f85980e70d.

* core/network/sync: Ask for block from all peers if none provided

When requesting an explicit fork sync, try to sync from all known peers,
when no specific peers were provided.

* core/network/sync: Request specific fork sync from peers ahead or on par

When making an explicit fork sync request without specifying any peers,
make sure to only request it from the locally known peers that are
either ahead or on a par compared to the block number we are looking
for.

* grandpa: fix tests

* grandpa: fix warnings

* grandpa: add test for block sync request on until_imported

* grandpa: rename Environment field inner to client

* grandpa: fix minor nits

* grandpa: minor nits in until_imported

* grandpa: copy docs for set_sync_fork_request

* grandpa: remove stale TODO on UntilImported
2019-10-24 16:01:14 +01:00
Pierre Krieger eabdcbdd4f Remove support for secp256k1 for network keys (#3897) 2019-10-23 22:05:05 +02:00
Bastian Köcher d82216673c Throw an error if a bootnode is registered with two different peer ids (#3891)
* Throw an error if a bootnode is registered with two different peer ids

* Rename error

* Fix compilation :(

* Review feedback
2019-10-23 17:17:12 +02:00
Weiliang Li 5a07887535 gossip: futures 03 Receiver (#3832)
* gossip: futures 03 receiver

* fix gossip test

* use tokio 01

* add comment

* Update core/finality-grandpa/src/communication/mod.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* fix format

* rename

* remove tokio 01 runtime

* minor fix

* make stable happy
2019-10-23 08:23:47 +02:00
Arkadiy Paronyan 3963bb58ff Persist block announcements (#3826)
* Persist block announcements

* Renamed sync requests to fork targets

* Fixed pruning detection condition
2019-10-18 16:44:39 +02:00
Demi Obenour 4da48dd9f5 Bump dependencies (#3787)
* Update dependencies, respecting semver

* Bump dependencies

* Don’t patch tiny-bip39 dependency
2019-10-09 12:29:28 -04:00
Weiliang Li 520009973f gossip: save sender for kept messages (#3738) 2019-10-08 16:16:24 +01:00
Wei Tang 18fa1d15ce Cumulative fixes to make working with consensus-pow easier (#3617)
* consensus-pow: add difficulty data to auxiliary

* Timestamp api

* Implement FinalityProofProvider for ()

* Add DifficultyApi

* Remove assumption that Difficulty is u128

* Use a separate trait for add instead of hard-code it as Saturating

* Some convenience functions to work with PowVerifier

* Try to fix mining unstability

* Fix generic resolution

* Unused best_header variable

* Fix hash calculation

* Remove artificial sleep

* Tweak proposer waiting time

* Revert sleep removal

The reason why it was there is because when mine_loop returns, it means an error
happened. In that case, we'd better sleep for a moment before trying again,
because immediately trying would most likely just fail.

* Pass sync oracle to mining

So that it does not mine when major syncing

* Expose build time as a parameter

Instead of hardcode it as previously 100ms.

* Update lock file

* Fix compile

* Support skipping check_inherents for ancient blocks

For PoW, older blocks are secured by the work, and can mostly be considered to
be finalized. Thus we can save both code complexity and validation time by
skipping checking inherents for them.

* Move difficulty fetch function out of loop

To make things faster

* Remove seed from mining

Each engine can use its own Rng source.

* Better comments

* Add TotalDifficulty definition for U256 and u128

* Update core/consensus/pow/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Rename TotalDifficulty::add -> increment

* Use SelectChain to fetch the best header/hash

* Update lock file
2019-10-03 11:02:20 +08:00
DemiMarie-parity 62b97a090d Upgrade dependencies whenever “easy” (#3556)
* Update all dependencies

* Upgrade dependencies whenever “easy”

“easy” means that there are no major changes required.

* Fix build and bump paste dependency to 0.1.6

* Remove dead code

* Re-add = dependency for futures-preview

* Add missing std features for runtime-io

* Remove git dependencies

as updated versions have been published to crates.io

* try to debug bug

* For sr-io, "std" should imply "no_oom" and "no_panic_handler".

Otherwise, rustc complains (correctly) about duplicate lang items.

* Add missing "runtime-io/std" features

* Fix compilation errors

* Prevent duplicate lang items

Rust does not allow duplicate lang items.  When compiled without the
`std` feature, `sr-io` defines two lang items.  Therefore, `sr-io`
compiled without `feature = "std"` must not be linked with `std`.

However, `pwasm-utils` and `wasmi-validation` both bring in `std` unless
compiled with `default-features = "false"`.  This caused a duplicate
lang item error.  Building both with `default-features = "false"`
prevents this error.  When building with `feature = "std"`, they should
both be built with the `std` feature, so this feature needs to be
explicitly depended on.

* Bump `impl_version`

* Make tests pass

Three tests used 1 less gas than they had previously.

* Try to un-break build

* Add a Cargo.lock file

* Revert offchain code

* Revert "Revert offchain code"

This reverts commit d216d08cc6ca0344614669c1d24cde3aa5c0d4e2.

* Don’t try to send a body with a GET request

without adding a Transfer-Encoding or Content-Length header.

This has always been wrong, but hyperium/hyper#1925 hid the bug until
hyper was upgraded to 0.12.35.

* Change some more GET requests to POST requests

* Fix excess line width and remove an `extern crate`

* Delete commented-out extern crate

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Fix regression in Cargo.toml files

dev-dependencies need `default-features = false`, too.

* Bump parity-wasm dependency

* Bump `futures-preview`

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock files

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/service/src/chain_ops.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
2019-10-02 15:23:59 -04:00
Marcio Diaz f6bd58ac1f Optimize tree route to sync faster (#3665)
* Introduce HeaderMetadata and use it for tree_route. Add lowest_common_ancestor.
* Add tests.
2019-10-02 20:30:43 +02:00
Svyatoslav Nikolsky 387c31598d Prepare for asynchronous transaction validation in tx pool (#3650)
* async txpool API

* Update core/rpc/src/author/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/transaction-pool/graph/src/pool.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Pool -> Pool + ValidatedPool

* removed lost block_on when importing xt from network

* fix grumbles

* alias for future::Executor in rpc

* removed executor from Author RPCs

* Pool + SharedValidatedPool -> Pool

* fix compilation after merge

* another fix

* another fix
2019-10-01 12:14:25 +03:00
Tomasz Drwięga 667ee95f5d ChainSpec extensions (#3692)
* Add some chainspec tests and make sure we validate it.

* Manual implementation of Extension + Forks definitions.

* Move chain spec to separate crate.

* Allow using ChainSpec with extensions.

* Renames.

* Implement Extension derive.

* Implement Extension for Forks.

* Support specifying fork blocks.

* make for_blocks work

* Support forks correctly.

* Add a bunch of docs.

* Make fork blocks optional.

* Add missing docs.

* Fix build.

* Use struct for check_block params.

* Fix tests?

* Clean up.
2019-09-29 01:05:36 +08:00
Arkadiy Paronyan 3242d7f2b6 Explicit sync API for downloading important, possibly orphaned, forks (#3633)
* Explicit sync API

* Keep sync requests

* Don't request the finalized block we already have

* Dropping requests & docs

* Renamed a function
2019-09-28 14:02:36 +02:00
Toralf Wittner 3dd5a508b4 Remove debug_assert!ion. (#3678) 2019-09-25 09:35:45 +02:00
Toralf Wittner 7611c08679 Fix invalid debug assertion. (#3663)
The current `debug_assert_eq` is clearly wrong as the code handles zero
response blocks. What should have been expressed is that we expect 0 or
1 response blocks.
2019-09-24 13:53:16 +02:00
Toralf Wittner af0d71d389 Add block announce validator. (#3346)
* Add `BlockAnnounceValidator` trait.

* Add associated data to block announcement.

* Make tests compile.

* Move validator into `sync.rs`.

* Smaller changes.

* Update core/network/src/protocol.rs

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update core/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/network/src/test/sync.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Formatting.

* Remove assoc. data from `BlockImportNotification`.

* Use `Option<Vec<u8>>` for associated data.

* Update core/network/src/protocol/sync.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix type error.
2019-09-24 10:48:21 +02:00
Robert Habermeier c200ce757b Fixing BABE epochs to change between blocks (#3583)
* always fetch epoch from runtime

* node integration tests don't test light nodes

* give stand-in full node a FULL role

* rejig babe APIs

* introduce next-epoch-descriptor type

* overhaul srml-BABE epoch logic

* ensure VRF outputs end up in the right epoch-randomness

* rewrite `do_initialize` to remove unnecessary loop

* begin accounting for next epoch in epoch function

* slots passes header to epoch_data

* pass slot_number to SlotWorker::epoch_data

* begin extracting epoch-change logic into its own module

* aux methods for block weight

* aux methods for genesis configuration

* comment-out most, refactor header-check pipeline

* mostly flesh out verifier again

* reinstantiate babe BlockImport implementation

* reinstate import-queue instantiation

* reintroduce slot-worker implementation

* reinstate pretty much all the rest

* move fork-choice logic to BlockImport

* fix some, but not all errors

* patch test-runtime

* make is_descendent of slightly more generic

* get skeleton compiling when passing is_descendent_of

* make descendent-of-builder more succinct

* restore ordering of authority_index / slot_number

* start fiddling with tests

* fix warnings

* improve initialization architecture and handle genesis

* tests use correct block-import

* fix BABE tests

* fix some compiler errors

* fix node-cli compilation

* all crates compile

* bump runtime versions and fix some warnings

* tweak fork-tree search implementation

* do backtracking search in fork-tree

* node-cli integration tests now work

* fix broken assumption in test_connectivity

* babe tests fail for the right reasons.

* test genesis epoch logic for epoch_changes

* test that epochs can change between blocks

* First BABE SRML test

* Testing infrastructure for BABE

Also includes a trivial additional test.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* A little more test progress

* More work on BABE testing

* Try to get the tests working

* Implement `UintAuthorityId`-based test mocks

* Fix compilation errors

* Adjust to upstream changes

* Block numbers are ignored in BABE epoch calculation

* authority_index() should ignore invalid authorities

* Fix compile error

* Add tests that session transitions happen

* Check if BABE produces logs

It currently does not.

* Fix test suite

This was really nasty, due to a type confusion that showed up as an
off-by-1 buffer error.

* Add additional tests

Most of these were derived from the current output, so they are only
useful to guard against regressions.

* Make the tests more readable

Also bump impl_version.

* Fix excessive line width

* Remove unused imports

* Update srml/babe/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* try to fix imports

* Fix build errors in test suite

* tests did not pass

* Try to get at least one digest to be output

Currently, the code emits either no digests (if I don’t call
`Session::rotate_session()` or two digests (if I do), which is wrong.

* More tests

They still don’t work, but this should help debugging.

* fix silly error

* Don’t even try to compile a broken test

* remove broken check_epoch test and add one for genesis epoch

* Check that the length of the pre-digests is correct

* Bump `impl_version`

* use epoch_for_descendent_of even for genesis

* account for competing block 1s

* finish srml-babe docs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* address grumbles
2019-09-23 16:03:05 +02:00
Hernando Castano e434dbdfc5 Use ThreadPool::spawn_ok() instead of ThreadPool::spawn() (#3656)
* Bump futures-timer to v0.3

* Bump futures-preview to v0.3.0-alpha.18

* Replace ThreadPoll::spawn() with ThreadPoll::spawn_ok()
2019-09-20 15:26:14 +02:00
Svyatoslav Nikolsky eba6dd73c6 Blockchain cache pruning strategy (#3395)
* blockchain cache pruning strategy

* added some internal docs to cache_pruning_strategy

* Update core/client/db/src/cache/mod.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>
2019-09-20 09:49:15 +03:00
Arkadiy Paronyan 84d0c790f3 Send block status with announcement (#3607)
* Send block status with announcement

* Fixed tests

* Whitespace

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Additional comment

* Update comment

Co-Authored-By: André Silva <andre.beat@gmail.com>
2019-09-17 11:19:46 +02:00
Weiliang Li 6679c8b051 Update mod.rs (#3625)
Add `import_notification_stream` for test client
2019-09-17 11:16:22 +02:00
Svyatoslav Nikolsky 849e824652 Fetching multiple remote values using single message (#3612)
* multiple remote values in single message

* keys_str: String ->keys_str: closure
2019-09-16 13:30:21 +03:00
Svyatoslav Nikolsky 010395e620 Never panic during execution proof check (#3504)
* do not panic during execution proof check

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* BackendTrustLevel enum

* up runtime version

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* fixed some grumbles

* Update core/state-machine/src/testing.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update core/state-machine/src/lib.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* mov where

* spaces -> tabs (to restart build)
2019-09-11 10:27:32 +03:00
Svyatoslav Nikolsky 81af1d140a finalize -> finalize_with_ancestors (#3576) 2019-09-09 14:22:58 +01:00
André Silva ae8bc414ab grandpa: don't require justification of consensus changes on full node (#3540)
* grandpa: don't request justification for consensus changes on full node

* sync: finalize justification/finality proof requests in-order

* sync: ignore result of try_finalize_root on justification import
2019-09-09 07:28:35 +02:00
Max Inden 027d88796b core/authority-discovery: Enable authorities to discover each other (#3452)
With the *authority-discovery* module an authoritative node makes itself
discoverable and is able to discover other authorities. Once discovered, a node
can directly connect to other authorities instead of multi-hop gossiping
information.

1. **Making itself discoverable**

    1. Retrieve its external addresses

    2. Adds its network peer id to the addresses

    3. Sign the above

    4. Put the signature and the addresses on the libp2p Kademlia DHT

2. **Discovering other authorities**

    1. Retrieve the current set of authorities

    2. Start DHT queries for the ids of the authorities

    3. Validate the signatures of the retrieved key value pairs

    4. Add the retrieved external addresses as ~reserved~ priority nodes to the
       peerset


* node/runtime: Add authority-discovery as session handler

The srml/authority-discovery module implements the OneSessionHandler in
order to keep its authority set in sync. This commit adds the module to
the set of session handlers.

* core/network: Make network worker return Dht events on poll

Instead of network worker implement the Future trait, have it implement
the Stream interface returning Dht events.

For now these events are ignored in build_network_future but will be
used by the core/authority-discovery module in subsequent commits.

* *: Add scaffolding and integration for core/authority-discovery module

* core/authority-discovery: Implement module logic itself
2019-09-06 17:43:03 +02:00
André Silva 397855c611 client: only report block import to telemetry if new best (#3548)
* client: only report block import to telemetry if new best

* grandpa: fix tests

* consensus: derive Default for ImportedAux

* network: fix test
2019-09-04 17:14:42 +01:00
cheme e3d0c60a31 Implement change trie for child trie. (#3122)
* Initial implementation, some redundancy is awkward and there is
some useless computation (but there is a pending pr for that).
Next are tests.

* Minimal tests and fix extend child.

* implement iterator for change child trie.

* prune child trie.

* Fix pruning test.

* bump spec version.

* Avoid empty child trie (could also be checked before)

* tabs.

* Fix child digest overriding each others.

* Restore doc deleted on merge.

* Check correct child value on extrinsics build.

* Revert runtime version update.
2019-09-02 12:10:42 +03:00
cheme 816e132cd7 Add missing child trie rpc on_custom_message handler (#3522)
* missing on_custom for child read.

* fix and complete, add test cases.

* shorten line.

* replace vecs of key value tuple with maps.
2019-08-31 17:19:54 -07:00