* Finality notification: Optimize calculation of stale heads
While looking into some problem on Versi where a collator seemed to be stuck. I found out that it
was not stuck but there was a huge gap between last finalized and best block. This lead to a lot
leaves and it was basically trapped inside some loop of reading block headers from the db to find
the stale heads. While looking into this I found out that `leaves` already supports the feature to
give us the stale heads relative easily. However, the semantics change a little bit. Instead of
returning all stale heads of blocks that are not reachable anymore after finalizing a block, we
currently only return heads with a number lower than the finalized block. This should be no problem,
because these other leaves that are stale will be returned later when a block gets finalized which
number is bigger than the block number of these leaves.
While doing that, I also changed `tree_route` of the `FinalityNotification` to include the
`old_finalized`. Based on the comment I assumed that this was already part of it. However, if
wanted, I can revert this change.
* FMT
* Update client/service/src/client/client.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Do not include the last finalized block
* Rename function
* FMT
* Fix tests
* Update figure
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* primitives/runtime: initial changes on supporting multiple Justifications
* primitives/runtime: make Justifications strongly typed
* Encode/decode Justifications
* primitives/runtime: add Justification type
* backend: apply_finality and finalize_block takes a single Justification
* manual-seal: create engine id and let rpc take encoded justification
* backend: skeleton functions for appending justifications
* backend: initial implementation append_justification
Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k
* backend: guard against duplicate consensus engine id
* client/db: add check for block finality
* client/api: add append_justification to in_mem db
* client/light: add no-op append_justification
* network: fix decode call for Justification
* network: only send a single Justification in BlockData
* network: minor comment update
* protocol: update field names to distinguish single justification
* client: further field renames to plural
* client: update function names to plural justifications
* client/db: upgrade existing database for new format
* network: remove dependency on grandpa crate
* db: fix check for finalized block
* grandpa: check for multiple grandpa justifications hwne importing
* backend: update Finalizer trait to take multiple Justifications
* db: remove debugging statements in migration code
* manual-seal: update note about engine id
* db: fix check for finalized block
* client: update variable name to reflect it is now plural
* grandpa: fix incorrect empty Justications in test
* primitives: make Justifications opaque to avoid being empty
* network: fix detecting empty Justification
* runtime: doc strings for Justifications functions
* runtime: add into_justifications
* primitives: check for duplicates in when adding to Justifications
* network/test: use real grandpa engine id in test
* client: fix reviewer comments
* primitives: rename Justifications::push to append
* backend: revert changes to Finalizer trait
* backend: revert mark_finalized
* backend: revert changes to finalize_block
* backend: revert finalized_blocks
* db: add a quick early return for performance
* client: minor reviewer comments
* service/test: use local ConsensusEngineId
* network: add link to issue for sending multiple Justifications
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Apply suggestions from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* network: tweaks to review suggestions
* network: revert change to BlockData for backwards compatibility
* Apply suggestion from code review
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* primitives: update doc comment for Justifications
* client/db/upgrade: avoid grandpa crate dependency
* consensus: revert to single Justification for import_justification
* primitives: improve justifications docs
* style cleanups
* use and_then
* client: rename JUSTIFICATIONS db column
* network: revert to using FRNK in network-test
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* chore/error: remove from str conversion and add deprecation notifications
* fixup changes
* fix test looking for gone ::Msg variant
* another test fix
* one is duplicate, the other is not, so duplicates reported are n-1
* darn spaces
Co-authored-by: Andronik Ordian <write@reusable.software>
* remove pointless doc comments of error variants without any value
* low hanging fruits (for a tall person)
* moar error type variants
* avoid the storage modules for now
They are in need of a refactor, and the pain is rather large
removing all String error and DefaultError occurences.
* chore remove pointless error generic
* fix test for mocks, add a bunch of non_exhaustive
* max line width
* test fixes due to error changes
* fin
* error outputs... again
* undo stderr adjustments
* Update client/consensus/slots/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove closure clutter
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* more error types
* introduce ApiError
* extract Mock error
* ApiError refactor
* even more error types
* the last for now
* chore unused deps
* another extraction
* reduce should panic, due to extended error messages
* error test happiness
* shift error lines by one
* doc tests
* white space
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Into -> From
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove pointless codec
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* avoid pointless self import
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:
- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)
refs #4679
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
* Run offchain workers at particular hash, not number.
* Don't run if not new best.
* Don't run if not new best.
* Update client/service/src/builder.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Update client/service/src/builder.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Update client/service/src/builder.rs
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Adding script for rename, could be applicable for nodes on top of it, too
* add stderr and gitlab ci features
* apply script
* fix now minor details in expected stderr
* Update the Cargo.lock
* fix name: sc-transaction -> sc-tracing
* fix rename in script, too
* Move transaction pool to primitives
* move backend, errors into primitives
* remove unused client depencies
* Move rpc-api into primitives
* Move peerset back to client
* Move rpc/api back to client, move palette/support/rpc into utils
* move support-rpc into subfolder
* move system-rpc into utils
* move transaction-pool and -graph back into client
* fix broken imports
* Clean up test primitives
* Make support test utils independent of frame
* remove unnecessary node dependencies from service
* Reactivate dependency script:
- only enforce the now achieved status quo will remain
- allow for primitives to depend on /client for now without failing
- more discriptive error message so people understand, what it wants
- minor fix to differentiative between ../client and /client (which may be a subfolder)
- don't allow this to fail anylonger.
* fix doc comment
* 'Should not' rather than 'must not'.
* Revert unwanted dependency changes
* fix faulty import
* fixup derive_more version
* fix wrong import path
Falco Hirschenberger