* Add a new host function for reporting fatal errors
* Fix one of the wasmtime executor tests
* Have `#[runtime_interface(wasm_only)]` actually mean WASM-only, and not no_std-only
* Print out errors through `Display` instead of `Debug`
* Switch one more trait to require `Error` for its error instead of only `Debug`
* Align to review comments
* Add stale branches heads to finality notifications
Warning. Previous implementation was sending a notification for
each block between the previous (explicitly) finalized block and
the new finalized one (with an hardcoded limit of 256).
Now finality notification is sent only for the new finalized head and it
contains the hash of the new finalized head, new finalized head header,
a list of all the implicitly finalized blocks and a list of stale
branches heads (i.e. the branches heads that are not part of the
canonical chain anymore).
* Add implicitly finalized blocks list to `ChainEvent::Finalized` message
The list contains all the blocks between the previously finalized block
up to the parent of the currently finalized one, sorted by block number.
`Finalized` messages handler, part of the `MaintainedTransactionPool`
implementation for `BasicPool`, still propagate full set of finalized
blocks to the txpool by iterating over implicitly finalized blocks list.
* Rust fmt
* Greedy evaluation of `stale_heads` during finalization
* Fix outdated assumption in a comment
* Removed a test optimization that is no more relevant
The loop was there to prevent sending to
`peer.network.on_block_finalized` the full list of finalized blocks.
Now only the finalized heads are received.
* Last finalized block lookup not required anymore
* Tests for block finality notifications payloads
* Document a bit tricky condition to avoid duplicate finalization notifications
* More idiomatic way to skip an iterator entry
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Cargo fmt iteration
* Typo fix
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix potential failure when a finalized orphan block is imported
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Remove transaction-pool `test-helpers` feature
`test-helpers` feature is a bad idea in general, because once the feature is enabled somewhere in
the workspace, it is enabled anywhere. While removing the feature, the tests were also rewritten to
get rid off other "only test" related code.
Contributes towards: https://github.com/paritytech/substrate/issues/9727
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix benches
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Account for transaction priority when enforcing limits
* Improve `enforce_limits` comment
* Explanation comment on why comparison impl is not used for limit enforcement
* Yep
* Make it compile
* Make the benchmark work
* Some stuff
* Optimize transaction pool `BestIterator`
* Some docs
* Fix more warnings
* Fix compilation
* FMT
* sp-utils => sc-utils
* cargo fmt
* These files are now in the client so should be licensed as GPL3
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Comment out browser stuff
* Remove browser stuff
* Remove more wasm transport code
* Remove ExtTransport and rework how telemetry initialises.
* Change (most) wasm-timer using code to use std::time
* Rename CI-job
* Aura does not compile for wasm
* Remove testing in the browser on CI
* Update README
* Leave `StreamSink` be
* fmt
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* Do not call `initialize_block` before any runtime api
Before this change we always called `initialize_block` before calling
into the runtime. There was already support with `skip_initialize` to skip
the initialization. Almost no runtime_api requires that
`initialize_block` is called before. Actually this only leads to higher
execution times most of the time, because all runtime modules are
initialized and this is especially expensive when the block contained a
runtime upgrade.
TLDR: Do not call `initialize_block` before calling a runtime api.
* Change `validate_transaction` interface
* Fix rpc test
* Fixes and comments
* Some docs
* Transaction pool: Remove futures-diagnose and thread pool
This pr removes `futures-diagnose` as this isn't used anymore. Besides
that the pr also removes the thread pool that was used to validate the
transactions in the background. Instead of this thread pool we now spawn
two separate long running tasks that we use to validate the
transactions. All tasks of the transaction pool are now also spawned as
essential tasks. This means, if any of these tasks is stopping, the node
will stop as well.
* Update client/transaction-pool/src/api.rs
* Transaction pool: Ensure that we prune transactions properly
There was a bug in the transaction pool that we didn't pruned
transactions properly because we called `prune_known`, instead of `prune`.
This bug was introduced by:
https://github.com/paritytech/substrate/pull/4629
This is required to have stale extrinsics being removed properly, so
that they don't fill up the tx pool.
* Fix compilation
* Fix benches
* ...
`ready_at` returns when we have processed the requested block. However,
on startup we already have processed the best block and there
are no transactions in the pool on startup anyway. So, we can set `updated_at`
to the best block on startup.
Besides that `ready_at` now returns early when there are no ready nor
any future transactions in the pool.
* Make grandpa work
* Introduce `SharedData`
* Add test and fix bugs
* Switch to `SharedData`
* Make grandpa tests working
* More Babe work
* Make it async
* Fix fix
* Use `async_trait` in sc-consensus-slots
This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.
* Make grandpa tests compile
* More Babe tests work
* Fix network test
* Start fixing service test
* Finish service-test
* Fix sc-consensus-aura
* Fix fix fix
* More fixes
* Make everything compile *yeah*
* Fix build when we have Rust 1.51
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix warning
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* chore/error: remove from str conversion and add deprecation notifications
* fixup changes
* fix test looking for gone ::Msg variant
* another test fix
* one is duplicate, the other is not, so duplicates reported are n-1
* darn spaces
Co-authored-by: Andronik Ordian <write@reusable.software>
* remove pointless doc comments of error variants without any value
* low hanging fruits (for a tall person)
* moar error type variants
* avoid the storage modules for now
They are in need of a refactor, and the pain is rather large
removing all String error and DefaultError occurences.
* chore remove pointless error generic
* fix test for mocks, add a bunch of non_exhaustive
* max line width
* test fixes due to error changes
* fin
* error outputs... again
* undo stderr adjustments
* Update client/consensus/slots/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove closure clutter
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* more error types
* introduce ApiError
* extract Mock error
* ApiError refactor
* even more error types
* the last for now
* chore unused deps
* another extraction
* reduce should panic, due to extended error messages
* error test happiness
* shift error lines by one
* doc tests
* white space
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Into -> From
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* remove pointless codec
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* avoid pointless self import
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* implement events handling, implement parent_id for spans & events
* add events to sp_io::storage
* update test
* add tests
* adjust limit
* let tracing crate handle parent_ids
* re-enable current-id tracking
* add test for threads with CurrentSpan
* fix log level
* remove redundant check for non wasm traces
* remove duplicate definition in test
* Adding conditional events API
* prefer explicit parent_id over current,
enhance test
* limit changes to client::tracing event implementation
* remove From impl due to fallback required on parent_id
* make tracing codecable
* replace with global tracing
* new tracing interface
* impl TracingSubscriber in client
* implement access to global TracingSubscriber from primitives
* span for wasm
* increment towards Wasm Tracing Subscriber implementation
* increment, remove sp-tracing from runtime-interface
* increment, it compiles
* attained original functionality with new mechanism
* implement remaining TracingSubscriber functions
* remove spans from decl_module
* add handling for encoded values
* Revert "replace with global tracing"
This reverts commit 8824a60deea54d9b437407a21c8ceaf6a1902ee5.
* Wasm Side Tracing
* tracing on wasm
* enable tracing wasm on node-runtime
* export all the macros in std
* tracing subscriber on wasm-side only
* pass spans and events over and record them
* reactivate previous code and cleanup
* further cleaning up
* extend the span macros, activate through executive
* tracking the actual extrinsic, too
* style
* fixing tests
* spaces -> tabs
* attempting to reactivate params
* activate our tests in CI
* some passing
* tests passing
* with core lazy
* global tracer for wasm side with pass over
* fixing metadata referencing
* remove const_fn feature requirement
* reenable dispatch traces
* reset client tracing
* further cleaning up
* fixing runtime-test
* move tracing-build setup into runtime-test
* Merge DebugWriter from tracing and frame-support, move to sp-std
* remove dangling fixme
* Docs for tracing primitives
* cleaning up a bit more
* Wasm interface docs
* optimise docs.rs setup
* adding tracing flags to uncomment
* remove brace
* fixing imports
* fixing broken syntax
* add required modules
* nicer formatting
* better target management
* adding low level storage tracing events into frame
* add custom Debug impl for WasmMetadata
* cloning profiler
* adding info about cloning profiler
* using in-scope for within calls
* proper time tracing, cleaning up println
* allow to disable tracing on runtime_interface-macro
* disable tracing for wasm-tracing-interface
* simplify wasm-tracing-api
* update client to new interface
* fixing docs and tests for sp-tracing
* update integration tests
* re-activating enter_span
* dropping FIXME, it's documented
* fix formatting
* fix formatting
* fix imports
* more debug info
* inform wasm about it being disabled by returning 1
* only one tracer, but enabled multi-all support
* make trait pub again for tests
* Apply suggestions from code review
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
* fixing wasm doc tests for proper usage
* remove unnecessary import
* fixing formatting
* minor style fixes
* downgrading wabt
* update error message for UI
* Fix interface test
* next attempt to fix macros
* geee
* revert tracing on hashed for future PR
* remove local macros, use originals
* we are able to convert to static items
* implement more WasmValue types
* adding support to convert str, debug and encoded values
* more minor fixes
* revert unsafe 'static making
* fix indentation
* remove commented lines
* bump all them tracing versions
* cleaning up docs and info
* document new flag
* the new layered system handles span cloning better
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: David <dvdplm@gmail.com>
* Revalidate transactions only on latest best block
We should revalidate transactions only on the latest best block and not
on any arbitrary block. The revalidation before failed when there were
multiple blocks on the height given to the revalidation function, but no
block was imported as best block.
* Update test-utils/runtime/transaction-pool/src/lib.rs
Co-authored-by: Jaco Greeff <jacogr@gmail.com>
* Fix tests
* Only process best blocks in the transaction pool
Co-authored-by: Jaco Greeff <jacogr@gmail.com>
* Remove service components and add build_network, build_offchain_workers etc
* Improve transaction pool api
* Remove commented out line
* Add PartialComponents
* Add BuildNetworkParams, documentation
* Remove unused imports in tests
* Apply suggestions from code review
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Remove unused imports in node-bench
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Simplify a few chain components creation APIs related to the service
* Fix basic-authorship doc tests
* Remove DefaultQueue
* Update client/service/src/builder.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Move ExecutionExtensions comment around
* Remove unused BlakeTwo256
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Improve transaction submission
Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.
Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.
* Make sure we don't submit the same transaction twice from the network concurrently
* Remove added listener call
* Feedback
* Ignore banned on resubmit
* Fix tx-pool returning the same transaction multiple times
This fixes a bug that lead to returning the same transaction multiple
times when iterating the `ready` iterator. Internally the transaction
was kept in the `best` list and could be duplicated in that list be
re-inserting it again. This `best` list is using a `TransactionRef`
which internally uses a `insertion_id`. This `insertion_id` could lead
to the same transaction being inserted multiple times into the `best`
list.
* Update client/transaction-pool/src/testing/pool.rs
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Seperate out the complexity in ServiceBuilder::build_common into seperate functions
* Fix line widths
* Move some functions to their respective crates
This pr fixes a bug with the transaction pool not sending certain events
like finalized and also fixes the order of events. The problem with the
finalized event was that we did not extracted pruned extrinsics if there
were not ready transactions in the pool. However this is wrong, if we
have a re-org, a tx is clearly not ready anymore and we still need to
send a pruned event for it because it is in a new block included. This
also lead to sending "ready" events and tx being re-validated. The
listener also only send the "finalized" event if it has seen a block as
being included, which did not happen before with the old code.
The second fix of the pr is the order of events. If we prune and retract the
same transaction in the same block, we first need to send the "retract"
event and after that the "pruned" event, because finalization takes
longer and this would lead to the UI showing "retract" while it actually
is included.
Falco Hirschenberger