With the *authority-discovery* module an authoritative node makes itself
discoverable and is able to discover other authorities. Once discovered, a node
can directly connect to other authorities instead of multi-hop gossiping
information.
1. **Making itself discoverable**
1. Retrieve its external addresses
2. Adds its network peer id to the addresses
3. Sign the above
4. Put the signature and the addresses on the libp2p Kademlia DHT
2. **Discovering other authorities**
1. Retrieve the current set of authorities
2. Start DHT queries for the ids of the authorities
3. Validate the signatures of the retrieved key value pairs
4. Add the retrieved external addresses as ~reserved~ priority nodes to the
peerset
* node/runtime: Add authority-discovery as session handler
The srml/authority-discovery module implements the OneSessionHandler in
order to keep its authority set in sync. This commit adds the module to
the set of session handlers.
* core/network: Make network worker return Dht events on poll
Instead of network worker implement the Future trait, have it implement
the Stream interface returning Dht events.
For now these events are ignored in build_network_future but will be
used by the core/authority-discovery module in subsequent commits.
* *: Add scaffolding and integration for core/authority-discovery module
* core/authority-discovery: Implement module logic itself
* Changes for the next libp2p release:
* Updates to the Kademlia APIs.
* Updated imports due to the extracted libp2p-swarm crate.
* ...
Still pending at least the following:
* rust-libp2p/#1189
* rust-libp2p/#1191
* rust-libp2p/#1194
* Use Quorum::One.
The previous choice was apparently arbitrary.
* Use libp2p-0.11 from crates.io. Address feedback.
* Correct imports after merge.
* Fix grammar and typo
* Extend network service
* Extend offchain API
* Support creating unsigned UncheckedExtrinsic
* Introduce srml/im-online
* Bump impl and spec version
* Fix web-wasm test
* Apply suggestions from code review
Remove parity-multiaddr dependency
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Replace transmute with from_raw_parts
* Replace PeerId.to_string() with .to_base58()
Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>
* Update Cargo.lock
* Bump impl and spec version (again)
It was updated in master in the meantime.
* Apply suggestions from code review
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Address comments
* Add public function is_online_in_current_session()
* Bump spec_version
* Fix doc tests
* Improve comments
* Remove superfluous line
* Name parameters consistently
* Implement comments
* Switch From to TryFrom
* Use Vec instead of HashSet
* Fix tests
* Revert me: local testing
* Fix check if already sent during session
We gossip each session, hence we need to check
if already sent in this session (not era).
* Fix typos
* Consistent terminology
* Revert "Revert me: local testing"
This reverts commit 73fbc29ff3e5ed71d99436318260b4f007e837f4.
* Introduce IsMember trait
* Implement misc comments
* Remove unused function
* Fix test
* Fix external_addresses being written
* Fix test
* Add necessary trait bound
* Do not increment version
* Update lib.rs
* Remove useless internal messages
* Remove NetworkService::disconnect_peer
* Remove NetworkMsg altogether
* Rename ProtocolMsg ServerToWorkerMsg
* Remove useless code
* Add example for parse_str_addr
* Move parse_str_addr and ProtocolId to config
* Don't reexport the content of config
* Rework the imports
* More reexports rework
* Add documentation
* Move finalization report to network future
* Move on_block_imported to worker
* get_value/put_value no longer locking
* local_peer_id() no longer locks
* Remove FetchFuture
* Service imports cleanup
* Produce the network state in the network task
* Merge network task and RPC network task
* Move network methods to NetworkWorker
* Remove Arc peers system from network
* add_reserved_peer now goes through the channel
* Remove Mutex around network swarm
* Remove the FnOnce alias traits
* Replace is_offline with num_connected
* Improve style of poll()
* Fix network tests
* Some doc in service module
* Remove macro export
* Minor doc changes
* Remove the synchronized() method of the import queue
* Line width
* Line widths
* Fix import queue tests
* Fix CLI tests
* Pass the peerset config to ProtocolBehaviour
* Don't pass the protocol versions
* Move fields from protocol_behaviour.rs to protocol.rs
* Remove LocalNetworkOut
* Move CustomProtos from protocol_behaviour.rs to protocol.rs
* Remove ProtocolBehaviour
* Inline poll()
* Force Behaviour to use Protocol
* Don't even attempt to have working tests
* Remove NetworkOut trait
* Line widths
* Add documentation at the root of substrate-network
* Apply suggestions from code review
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Grmlblbl
* Adjustments
* Link to Yamux specs
* Move network-libp2p into network
* Merge libp2p_service into service
* Don't expose RegisteredProtocol in the API
* Extract DiscoveryBehaviour from Behaviour
* Restore libp2p tests
* Add a test for discovery
* Line width
* Remove bandwidth check
* Fix gitlab
* Remove background thread from network
* Primarily use tokio-timer instead of tokio
* Minor style fix
* Reintroduce tokio as an optional dependency
* Renames
* Line width
* Ctxt -> Context
* Merge NetworkWorker and NetworkService
* send neighbor packets in more generic way
* integrate periodic neighbor-packet rebroadcaster
* integrate reporting
* attach callbacks to commit messages for rebroadcasting and reporting
* Tests for commit relay
* crunch up some nice warnings
* exit-scope sub-futures of grandpa
* address small grumbles
* some changes to commit handling
* Add `start_aura2`.
* .gitignore patch conflict files
and remove one that accidentally got committed
* Fix build
The tests still don’t work.
* Fix compilation errors
* Fix compile errors (again)
* Try (and fail) to fix tests
* Properly deserialize data
Previously, `DigestItem::Consensus` had no separate `DigestItemType`,
so it did not get properly serialized and deserialized.
* Add extra debug logging. Always allow old seals.
A `RUST_LOG=substrate_aura_consensus cargo test --all -- --nocapture \
tests::authoring_blocks` revealed that old seals were being and
rejected, causing the test to hang. As a temporary debug measure, allow
old seals unconditionally, so that CI can test if this fixes the
problem.
* Forcibly disable rejection of old seals
* Use old trait, but newer serialization
The old trait for `CompatibleDigestItem` actually worked. By changing
its implementation, one can ensure that all *new* seals have the modern
form, but *legacy* seals are still decoded correctly.
* Bump impl version
* Squash spurious deprecation warning
`rustc` should not be emitting a deprecation warning in deprecated
code, but it does, so silence it.
* Rip out unused Cargo feature
* Move AURA to aura_primitives
* Respond to code review
* Wrap overly-long line
* Reduce logging verbosity and add target
* Add dependency on `sr-primitives` to `aura_primitives`
* Fix build
It failed with a message about Cargo.lock being out of date.
* core: aura: rename aura engine id const
* core: aura: remove superfluous logging
* core: primitives: add removed semicolons
* core: aura: remove unused import
* core: network: style fix
* runtime: update wasm blobs
* runtime: bump impl_version
* core: primitives: tag all DigestItemType variants explicitly
* Update libp2p. Add support for ed25519 node (network) keys.
* Update networking to the changes from https://github.com/libp2p/rust-libp2p/pull/972.
* Add support for using ed25519 keys for libp2p networking.
* Add support for reading libp2p secret keys from (external) files.
* Adapt to changes from https://github.com/libp2p/rust-libp2p/pull/992
* More tests.
* Cosmetics
* Deduplicate tests.
* Remove quickcheck from tests that don't use extra random inputs.
* Remove quickcheck.
* Swap new/default impls for NetworkConfiguration.
* Use libp2p-0.5.0 from crates.io.
* Post-rebase update.
* Remove unnecessary wildcard pattern.
* Combine two overlapping tests.
* Limit the set of known extrinsics and blocks.
* Update Cargo.lock
* Always insert and pop oldest entry if needed.
Always inserting first ensures that the element's LRU position is
updated.
* rewrite network protocol/service to use channels
* remove use of unwrap
* re-introduce with_spec
* remove unnecessary mut
* remove unused param
* improve with_spec, add with_gossip
* rename job to task
* style: re-add comma
* remove extra string allocs
* rename use of channel
* turn TODO into FIXME
* remove mut in match
* remove Self in new
* pass headers by value to network service
* remove network sender from service
* remove TODO
* better expect
* rationalize use of network sender in ondemand
* core: fix predicate for dropping grandpa round messages
* core: grandpa: drop commits topic on authority set change
* core: gossip: only drop known messages based on expiration time
* core: grandpa: don't broadcast commit messages
* core: gossip: don't assume topics are header hashes
* core: gossip: expire messages more agressively
* core: grandpa: fix test environment
* core: gossip: fix tests
* core: gossip: track dead topics (and ignore messages)
* core: gossip: test dead topic pruning
* add beginnings of SRML grandpa library
* get srml-grandpa compiling
* tests for srml-grandpa
* add optional session integration to grandpa SRML
* start integration into node runtime
* Allow extracting pending change from header digest
* Make it compile on wasm
* make tests compile again
* Move Authority Key fetching into service, simplify service factory construction
* Generalize Authority Consensus Setup system
* Add Authority Setup Docs
* Allow CLI params to be extensible
- move params to structopts
- split parsing and default command execution
- add custom config to node
- extended parsing of custom config
- extending params via structop's flatten
* Minor fixes on cli extension params:
- added docs
- re-add actual app name, rather than node-name
- make strategy and subcommand optional
* better cli params
* synchronize GRANDPA and normal node authorities
* Implement grandpa::network for gossip consensus
* run_grandpa in Node
* Fix missed merge error
* Integrate grandpa import queue
* more specific type def
* link up linkhalf and import block
* make grandpa future send
* get compiling
* Fix new params convention and license header
* get it running
* rebuild node runtime WASM
* change logging level
* Update node/cli/src/params.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/cli/src/params.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/cli/src/lib.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/runtime/src/lib.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/cli/src/lib.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Clean up and Fixme for mutable config
* Move GrandpaService Integration into grandpa, feature gated but on per default
* Fixing grandpa runtime module test
* Update wasm runtime hashes for tests
* GRANDPA: use post-header hash when logging scheduled changes
* add an extra bit of logging to authorities
* fixing missing constrain
* remove old code
* move `NewAuthorities` to an event in srml-grandpa
* fix node-executor tests to use grandpa log
* Remove GossipConsensus from tests, use newly provided sync-feature, fixes tests
* Update to latest wasm runtimes
* address grumbles
* address grumbles
* only derive deserialize when using std
* Clean up use of Deserialize
* Move Roles to network::config
* Make network::config public
* Move NetworkConfig and NonReservedMode to config
* Move Params to config
* Move node_id() to NetworkManager and fix tests
* Rename Specialization to NetworkSpecialization
* docs: Remove safe-mix since is no longer local dependency
* docs: Fix existing packages link. Still only includes Substrate Core
* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme
* docs: Remove link to old docs
* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names
* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory
* Change example to use package name that opens all packages in RustDocs
* Fix list of packages names that are available to open in RustDocs
* fix typo
* fix ordering of CLI options and add missing backslash
* Generalize BlockImport
- move ImportBlock, BlockOrigin, ImportResult into shared sr-primitives
- let Consensus provide and traits again
- update consensus traits to latest development
- implement traits on client::Client, test_client::TestClient
- update RHD to use the new import_block API
* Move ImportBlock into consensus-common
* Send import notification in aura tests
* Integrating aura into service
* Make Signatures more generic
* Aura Block Production with the given key
* run aura on the thread pool
* start at exact step start in aura
* Add needed wasm blob, in leiu of better solutions.
* Make API ids consistent with traits and bring upstream for sharing.
* Add decrease_free_balance to Balances module
* Encode `Metadata` once instead of two times
* Bitops include xor
* Upgrade key module.
* Default pages to somewhat bigger.
* Introduce upgrade key into node
* Add `Created` event
* Make `on_message` take the message as `&mut Option<_>`
* Make `ConsensusGossip` implement `Specialization`
* Move `new_session` into `ConsensusGossip`
* Adds `construct_simple_protocol` macro for simplifying the creation of protocols
Pierre Krieger