* grandpa: initial implementation of minimal grandpa worker
* grandpa: extract grandpa observer future to function
* grandpa: add test for observer
* grandpa: start observer if no local key is defined
* grandpa: add minor comments
* grandpa: observer: log invalid commit
* grandpa: observer: persist voter set state on authority change and pause
* grandpa: observer: use commit processing callback
* grandpa: keep run_grandpa to avoid breaking public api
* grandpa: use grandpa::process_commit_validation_result
* grandpa: use finality-grandpa 0.7.2
* send neighbor packets in more generic way
* integrate periodic neighbor-packet rebroadcaster
* integrate reporting
* attach callbacks to commit messages for rebroadcasting and reporting
* Tests for commit relay
* crunch up some nice warnings
* exit-scope sub-futures of grandpa
* address small grumbles
* some changes to commit handling
* core: grandpa: migrate to grandpa 0.7
* core: grandpa: store current round votes and load them on startup
* core: grandpa: resend old persisted votes for the current round
* core: grandpa: store base and votes for last completed round
* core: grandpa: fix latest grandpa 0.7 changes
* core: grandpa: update to grandpa 0.7.1
* core: grandpa: persist votes for last two completed rounds
* core: grandpa: simplify VoterSetState usage
* core: grandpa: use Environment::update_voter_set_state
* core: grandpa: fix aux_schema test
* core: grandpa: add docs
* core: grandpa: add note about environment assumption
* core: grandpa: don't update voter set state on ignored votes
* core: grandpa: add test for v1 -> v2 aux_schema migration
* core: grandpa: add test for voter vote persistence
* core: grandpa: use grandpa 0.7.1 from crates.io
* core: grandpa: use try_init in test
* core: grandpa: add comment about block_import in test
* core: grandpa: avoid cloning HasVoted
* core: grandpa: add missing docs
* core: grandpa: cleanup up can_propose/prevote/precommit
* Consensus status packet
* Allow for repropagation after status
* More generic gossip
* add a basic view struct and gossip module
* move gossip stuff to the gossip module
* integrate view into gossip
* some reshuffling
* alter rules for keeping one commit at a time in view
* Allow sending addressed messages
* don't cast outgoing votes if we know that we voted before
* Handle one hop messages
* initial run at polite grandpa
* build WASM
* handle neighbor messages
* refactor validator's internals into an Inner struct
* gossip only knows to keep or discard messages. optimize should_send_to
* Periodic rebroadcast
* implement `should_send_to` and message_expired
* track peers' best received commit height
* Pass peer id to topic steam
* kill rebroadcasting network
* Notify about existing peers
* clean up network APIs a bunch
* implement gossip::send_message for direct messages
* refactor network trait
* implement gossip::send_message for direct messages
* get all non set-change tests passing
* treat unknown rebroadcasts as broadcasts
* get all other main tests passing
* remove unimplemented test
* everything compiles
* treat unknown rebroadcasts as broadcasts
* Rebradcast interval
* Apply suggestions from code review
Style
Co-Authored-By: arkpar <arkady.paronyan@gmail.com>
* Style
* some module docs
* address some grumbles + docs
* allow rebroadcast every few minutes
* send_topic && generic context
* some tests for view change
* more grumbles & tests
* use send_peer
* Add `start_aura2`.
* .gitignore patch conflict files
and remove one that accidentally got committed
* Fix build
The tests still don’t work.
* Fix compilation errors
* Fix compile errors (again)
* Try (and fail) to fix tests
* Properly deserialize data
Previously, `DigestItem::Consensus` had no separate `DigestItemType`,
so it did not get properly serialized and deserialized.
* Add extra debug logging. Always allow old seals.
A `RUST_LOG=substrate_aura_consensus cargo test --all -- --nocapture \
tests::authoring_blocks` revealed that old seals were being and
rejected, causing the test to hang. As a temporary debug measure, allow
old seals unconditionally, so that CI can test if this fixes the
problem.
* Forcibly disable rejection of old seals
* Use old trait, but newer serialization
The old trait for `CompatibleDigestItem` actually worked. By changing
its implementation, one can ensure that all *new* seals have the modern
form, but *legacy* seals are still decoded correctly.
* Bump impl version
* Squash spurious deprecation warning
`rustc` should not be emitting a deprecation warning in deprecated
code, but it does, so silence it.
* Rip out unused Cargo feature
* Move AURA to aura_primitives
* Respond to code review
* Wrap overly-long line
* Reduce logging verbosity and add target
* Add dependency on `sr-primitives` to `aura_primitives`
* Fix build
It failed with a message about Cargo.lock being out of date.
* core: aura: rename aura engine id const
* core: aura: remove superfluous logging
* core: primitives: add removed semicolons
* core: aura: remove unused import
* core: network: style fix
* runtime: update wasm blobs
* runtime: bump impl_version
* core: primitives: tag all DigestItemType variants explicitly
* Support skipping the interactive purge prompt
Skip it via `cargo run -- purge-chain -y`.
* Fix typos
* Add Grandpa telemetry
* Address comments
* Revert unnecessary version bump
* Version bump to make CI run
* Remove unnecessary cast
* Do not bump version
* Make use of parity-codec "derive" crate feature
* Bump impl_version
* Update wasm files and Cargo.lock files
* Transferred to parity_codec derive feature for the rest of the crates
* Update wasm binaries and lock files
* Update core and node wasm binaries
* core: implement logic for tracking dag of possible pending changes
* core: move pending justifications dag to its own crate
* core: remove unnecessary clone bounds on dag
* core: request justifications in-order from the dag
* core: dag: rename changes variables to node
* core: dag: allow finalizing blocks not part of dag
* core: dag: track best finalized number
* core: dag: add more tests
* core: sync: clean up pending justifications dag
* core: dag: derive codec decode encode
* core: dag: better error support
* core: dag: add finalization guarded by predicate
* core: grandpa: track multiple authority set changes in dag
* core: dag: add pre-order iterator
* core: grandpa: request justifications on startup
* core: dag: rearrange order of definitions
* core: rename util/dag to util/fork_tree
* core: fork_tree: add docs
* core: fork_tree: add more tests
* core: fork_tree: fix issues found in tests
* core: grandpa: fix authorities tests
* core: grandpa: add docs for is_descendent_of
* core: sync: add docs for PendingJustifications
* core: sync: add test for justification requests across forks
* core: sync: don't resend import or finality notifications in tests
* core: grandpa: add test for importing multiple change blocks
* core: grandpa: fix logic for checking if a block enacts a change
* core: grandpa: fix authorities tests
* rewrite network protocol/service to use channels
* remove use of unwrap
* re-introduce with_spec
* remove unnecessary mut
* remove unused param
* improve with_spec, add with_gossip
* rename job to task
* style: re-add comma
* remove extra string allocs
* rename use of channel
* turn TODO into FIXME
* remove mut in match
* remove Self in new
* pass headers by value to network service
* remove network sender from service
* remove TODO
* better expect
* rationalize use of network sender in ondemand
* core: grandpa: move consensus_changes to its own submodule
* core: grandpa: move justification to its own submodule
* core: grandpa: move environment to its own submodule
* core: grandpa: move block import to its own submodule
* core: fix predicate for dropping grandpa round messages
* core: grandpa: drop commits topic on authority set change
* core: gossip: only drop known messages based on expiration time
* core: grandpa: don't broadcast commit messages
* core: gossip: don't assume topics are header hashes
* core: gossip: expire messages more agressively
* core: grandpa: fix test environment
* core: gossip: fix tests
* core: gossip: track dead topics (and ignore messages)
* core: gossip: test dead topic pruning
Went through the TODOs, removed a bunch, which are outdated or nothing more than a regular comment, documented a bunch more as actual tickets and made them FIXMEs and unified their structure (`FIXME #TICKETNO DESC` for local tickets, `FIXME: DESC LINK` for external tickets) for easier in-editor support. Further more remove unnecessary remarks and related old code that I noticed in that instance.
* announce blocks we vote on to peers (missing network impl)
* Implemented 'announce'
* Fixed test
* improve announce docs and add logging
* Track recently announced blocks
* core: keep known gossip messages for twice their expiration
* core: test expiration of known gossip messages
* core: only broadcast grandpa votes if authority in current set
* core: only broadcast grandpa commits if authority in current set
* grandpa finality proof
* prove GrandpaApi::grandpa_authorities using parent block + some docs
* create justification when consensus data is changed
* generate justifications periodically
* test for ConsensusChanges
* BlockAuthorityId convenience type
* Rename AuthorityId -> Ed25519AuthorityId to make it more precise
* Generalize AuthorityId up to substrate-client
* Fix in client-db
* rename: BlockAuthorityId -> AuthorityIdFor
* typo: should be digest item
* Fix test-runtime authorityId mismatch
One states that AuthorityId is u64 while the other states that it's Ed25519AuthorityId.
* Fix more u64 - Ed25519AuthorityId mismatch
* Fix compile of most of the srml modules
* Continue to pin aura and grandpa with ed25519 and fix compile
* Add MaybeHash trait
* Fix node-runtime compile
* Fix network tests
* node: remove grandpa authority flags
* node: exit-guard grandpa and aura spawned futures
* node: wait for futures to stop running on shutdown
* core: run connectivity tests on same ports
* core: pass on_exit future when starting aura and grandpa
* node: add issue number to todo
* core: fix aura and grandpa tests
* core: support broadcasting consensus message to all peers
* grandpa: remove liveness oracle
* node: always start grandpa on full nodes
* core: don't check for grandpa justifications on failed block imports
* core: fix network connectivity test
* make offline-reporting infrastructure more generic
* add a listener-trait for watching when the timestamp has been set
* prevent inclusion of empty offline reports
* add test for exclusion
* generate aura-offline reports
* ability to slash many times for being offline "multiple" times
* Logic for punishing validators for missing aura steps
* stub tests
* pave way for verification of timestamp vs slot
* alter aura import queue to wait for timestamp
* check timestamp matches seal
* do inherent check properly
* service compiles
* all tests compile
* test srml-aura logic
* aura tests pass
* everything builds
* some more final tweaks to block authorship for aura
* switch to manual delays before step
* restore substrate-consensus-aura to always std and address grumbles
* update some state roots in executor tests
* node-executor tests pass
* get most tests passing
* address grumbles
* core: make block justification optional
* runtime: update wasm binaries
* core: optionally pass justification on finalize_block
* finality-grandpa: add channel to trigger authority set changes
this will allow the `BlockImport` to trigger an authority set change when
importing a change block that provides a justification (when syncing)
* finality-grandpa: move finalize_block to free function
* finality-grandpa: add GrandpaOracle for auth set liveness checking
this will be used by `BlockImport` to check whether the authority set for a
given block is still live, if the authority set isn't live then importing a
change block requires a justification.
* finality-grandpa: store justification on finalized transition blocks
* finality-grandpa: check justification on authority set change blocks
* finality-grandpa: poll grandpa liveness oracle every 10 seconds
* finality-grandpa: spawn grandpa oracle in service setup
* core: support multiple subscriptions per consensus gossip topic
* finality-grandpa: create and verify justifications
* finality-grandpa: update to local branch of grandpa
* finality-grandpa: update to finality-grandpa v0.5.0
* finality-grandpa: move grandpa oracle code
* finality-grandpa: fix canonality check
* finality-grandpa: clean up error handling
* finality-grandpa: fix canonical_at_height
* finality-grandpa: fix tests
* runtime: update wasm binaries
* core: add tests for finalizing block with justification
* finality-grandpa: improve validation of justifications
* core: remove unused IncompleteJustification block import error
* core: test multiple subscribers for same consensus gossip topic
* Revert "finality-grandpa: improve validation of justifications"
This reverts commit 51eb2c58c2219801e876af6d6c9371bdd9ff2477.
* finality-grandpa: fix commit validation
* finality-grandpa: fix commit ancestry validation
* finality-grandpa: use grandpa v0.5.1
* finality-grandpa: add docs
* finality-grandpa: fix failing test
* finality-grandpa: only allow a pending authority set change per fork
* finality-grandpa: fix validator set transition test
* get compiling with latest version of grandpa
* generalize UntilImported to prepare for waiting for commit message targets
* extract until_imported out to own module
* logic for blocking commits until enough blocks imported
* add tests for commit message blocking logic
* pass through commit mesage round number as well
* extract communication streams to own module
* add Error implementation for ExitOrError
* introduce stream adapter for checking commit messages
* output sink for commits
* implement the unimplemented
* remove extra line
* update to latest version of grandpa api
* update finality-grandpa to 0.4.0
* Use filter_map earlier when checking incoming commits messages
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* address some grumbles
* add beginnings of SRML grandpa library
* get srml-grandpa compiling
* tests for srml-grandpa
* add optional session integration to grandpa SRML
* start integration into node runtime
* Allow extracting pending change from header digest
* Make it compile on wasm
* make tests compile again
* Move Authority Key fetching into service, simplify service factory construction
* Generalize Authority Consensus Setup system
* Add Authority Setup Docs
* Allow CLI params to be extensible
- move params to structopts
- split parsing and default command execution
- add custom config to node
- extended parsing of custom config
- extending params via structop's flatten
* Minor fixes on cli extension params:
- added docs
- re-add actual app name, rather than node-name
- make strategy and subcommand optional
* better cli params
* synchronize GRANDPA and normal node authorities
* Implement grandpa::network for gossip consensus
* run_grandpa in Node
* Fix missed merge error
* Integrate grandpa import queue
* more specific type def
* link up linkhalf and import block
* make grandpa future send
* get compiling
* Fix new params convention and license header
* get it running
* rebuild node runtime WASM
* change logging level
* Update node/cli/src/params.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/cli/src/params.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/cli/src/lib.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/runtime/src/lib.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Update node/cli/src/lib.rs
Co-Authored-By: rphmeier <rphmeier@gmail.com>
* Clean up and Fixme for mutable config
* Move GrandpaService Integration into grandpa, feature gated but on per default
* Fixing grandpa runtime module test
* Update wasm runtime hashes for tests
* GRANDPA: use post-header hash when logging scheduled changes
* add an extra bit of logging to authorities
* fixing missing constrain
* remove old code
* move `NewAuthorities` to an event in srml-grandpa
* fix node-executor tests to use grandpa log
* Remove GossipConsensus from tests, use newly provided sync-feature, fixes tests
* Update to latest wasm runtimes
* address grumbles
* address grumbles
* only derive deserialize when using std
* Clean up use of Deserialize
* Revert "start to refactor block import wrapper a bit"
This reverts commit b919de49e1a82422a0870b66100832d531174771.
* Refactor grandpa stuff to get tests almost compiling
* Fixes after rebase
* Fixes merge compile errors
André Silva