mirror of
https://github.com/pezkuwichain/pezkuwi-subxt.git
synced 2026-06-11 16:31:07 +00:00
RPC to trace execution of specified block (#7780)
* Add filter reload handle
* add RPC, move logging module from cli to tracing
* remove dup fn
* working example
* Update client/rpc-api/src/system/mod.rs
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Prefer "set" to "reload"
* Re-enable the commented out features of the logger
* Remove duplicate code
* cleanup
* unneeded lvar
* Bump to latest patch release
* Add new CLI option to disable log filter reloading,
Move profiling CLI options to SharedParams
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Applied suggestions from reviews
* Fix calls to init_logger()
* Handle errors when parsing logging directives
* Deny `system_setLogFilter` RPC by default
* One more time
* Don't ignore parse errors for log directives set via CLI or RPC
* Improve docs
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/cli/src/config.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* fix merge errors
* include default directives with system_setLogFilter RPC,
implement system_rawSetLogFilter RPC to exclude defaults
* docs etc...
* update test
* refactor: rename fn
* Add a test for system_set_log_filter – NOTE: the code should likely change to return an error when bad directives are passed
* Update client/cli/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Address review grumbles
* Add doc note on panicking behaviour
* print all invalid directives before panic
* change RPCs to: addLogFilter and resetLogFilter
* make CLI log directives default
* add comments
* restore previous behaviour to panic when hard-coded directives are invalid
* change/refactor directive parsing
* fix line width
* add test for log filter reloading
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* finish up suggestions from code review
* improve test
* change expect message
* change fn name
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* add docs, remove unused fn
* propagate Err on invalid log directive
* Update tracing-subscriber version
* Improve docs for `disable_log_reloading` CLI param
* WIP implementation: RPC and trace capturing
* WIP
* fix incorrect number of digest items
* return errors
* add From impl for Values, rename structs
* fixes
* implement option to choose targets for traces
* rename fn
* fix EnvFilter and add root span
* fix root span
* add docs, remove unnecessary traits
* fix regression on parent_id introduced in a9c73113a8
* fix line width
* remove unused
* include block hash, parent hash & targets in response
* move types from sp-tracing into sp-rpc
move block and parent hash into root of BlockTrace
* switch from log::trace to tracing::trace in state-machine
* use unsigned integer type to represent Ext::id in traces
* ensure id is unique by implementing Subscriber
tracing_subscriber::FmtSubscriber does not guarantee
unique ids
* indentation
* fix typo
* update types
* add sp_io::storage events
* Change response format
- update types
- record distinct timestamps
- sort spans by first entered
* convert to HexDisplay, refactor
* Sort out fallout from merge
* Update client/rpc-api/src/state/mod.rs
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Exit early unless the node runs with --rpc-methods=Unsafe
* Better error handling
* Use wasm-timer
* revert trace alteration in `state-machine` and remove events in `sp_io::storage`
Resolve in follow-up PR
* Review feedback: less collects
* Without Arcs
* Fix span exit
* typo
* cleanup
* Add a few debug messages to tracing module
* Structure traces state-machine/ext; Dispatchable extrinsics spans not working
* Correctly encode Option storage values
* Remove test field for Put and Get
* Try out some changes to dispatch macro
* Add various log messages in dispatch
* Add span dispatch span to new proc macro
* Remove debug messages in dispatch
* Trivial clean up
* Structure remaining state-machine traces (ChangesRoot*)
* Removed unnesecary tracing targets
* Remove log
* New cargo.lock post merge
* Add logging for wasm_overrides
* remove temp logs
* remove temp logs
* remove unused dep
* remove temp logs
* add logging to wasm_overrides
* add logging to state_tracing
* add logging for spans to substrate (includes timings)
* Skip serializing some event fields; Remove most storage traces
* Bring back all ext.rs traces
* Do not skip bool values in events
* Skip serializing span values
* Serialize span values; remove some trace events in ext
* Remove more trace events
* Delete commented out traces
* Remove all unused traces
* Add event filtering
* Fix typo
* wip - change response types to be more efficient
missing import
type
* Serialize struct fields as camelCase
* Add back in event filtering
* Remove name field from event
* Sort spans by time entered
* Sort spans in ASCending order
* Add storage keys target param to rpc
* Limit payload size; improve hash fields; include storage keys
- cleanup event_key_filter
- better block hash representation
- limit payload size
- cleanup based on andrews comments
* Error when serialized payload is to big
* Import MAX_PAYLOAD from rpc-servers
* Clean up ext.rs
* Misc. cleaning and comments
* Strict ordering span Id; no span sort; adjust for rpc base payload
* Add RPC docs to rpc-api/src/state/mod
* Make params bullet points
* Update primitives/rpc/src/tracing.rs
* Put all tracing logic within trace
* Remove attr.record in new_span
* Add back value record in new_span
* restore result collection in ext
* Revert "Add back value record in new_span"
This reverts commit baf1a735f23e5eef1bf6264adfabb788910fa661.
* 🤦
* more 🤦
* Update docs; Try fix line width issues
* Improve docs
* Improve docs
* Remove default key filters + add key recs to docs
* Try restore old traces
* Add back doc comment
* Clean up newlines in ext.rs
* More new line remova;
l
* Use FxHashMap
* Try use EnvFilter directives for event filtering
* Remove directive, filter events by fields
* Use trace metadata correctly
* Try EnvFilter directive with all default targets
* Revert "Try EnvFilter directive with all default targets"
This reverts commit 4cc6ebc721d207c3b846444174f89d45038525ac.
* Clean up clippy warning
* Incorporate Niklas feedback
* Update trace/log macro calls to have better syntx
* Use Ordering::Relaxed
* Improve patch and filter doc comment
* Clean up `BlockSubscriber::new`
* Try optimize `BlockSubscriber::enabled`
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
* Use contains_key
* use heuristic for payload size
* Add error tupe for client::tracing::block
* Minor tweaks
* Make a note about `--features with-tracing`
* Add CURL example to RPC docs
* Link to substrate-archibe wasm
* Trivial doc clean up based on David feedback
* Explicit result type name
* Respect line length
* Use the error
* Don't print timings when spans close
* Fix failing sc-rpc-api
* Update sp-tracing inner-line doc
* Update client/tracing/src/block/mod.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/call_executor.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/service/src/client/call_executor.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/tracing/src/block/mod.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Update client/tracing/src/block/mod.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Address some review grumbles
* Update primitives/state-machine/src/ext.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Use result_encoded structure fields in ext.rs
* Use value key for ext put
* Add notes about tracing key names matter
Co-authored-by: Matt <mattrutherford@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
This commit is contained in:
mattrutherford
Generated
+32
-3
@@ -7777,6 +7777,7 @@ dependencies = [
|
||||
"sp-runtime",
|
||||
"sp-session",
|
||||
"sp-state-machine",
|
||||
"sp-tracing",
|
||||
"sp-transaction-pool",
|
||||
"sp-utils",
|
||||
"sp-version",
|
||||
@@ -7803,6 +7804,7 @@ dependencies = [
|
||||
"sp-core",
|
||||
"sp-rpc",
|
||||
"sp-runtime",
|
||||
"sp-tracing",
|
||||
"sp-transaction-pool",
|
||||
"sp-version",
|
||||
]
|
||||
@@ -8009,16 +8011,26 @@ dependencies = [
|
||||
"parking_lot 0.11.1",
|
||||
"regex",
|
||||
"rustc-hash",
|
||||
"sc-client-api",
|
||||
"sc-rpc-server",
|
||||
"sc-telemetry",
|
||||
"sc-tracing-proc-macro",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sp-api",
|
||||
"sp-block-builder",
|
||||
"sp-blockchain",
|
||||
"sp-core",
|
||||
"sp-rpc",
|
||||
"sp-runtime",
|
||||
"sp-storage",
|
||||
"sp-tracing",
|
||||
"thiserror",
|
||||
"tracing",
|
||||
"tracing-core",
|
||||
"tracing-log",
|
||||
"tracing-subscriber",
|
||||
"wasm-bindgen",
|
||||
"wasm-timer",
|
||||
"web-sys",
|
||||
]
|
||||
|
||||
@@ -8408,6 +8420,15 @@ version = "0.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c111b5bd5695e56cffe5129854aa230b39c93a305372fdbb2668ca2394eea9f8"
|
||||
|
||||
[[package]]
|
||||
name = "slog"
|
||||
version = "2.7.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8347046d4ebd943127157b94d63abb990fcf729dc4e9978927fdf4ac3c998d06"
|
||||
dependencies = [
|
||||
"erased-serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "smallvec"
|
||||
version = "0.6.14"
|
||||
@@ -8971,9 +8992,11 @@ dependencies = [
|
||||
name = "sp-rpc"
|
||||
version = "3.0.0"
|
||||
dependencies = [
|
||||
"rustc-hash",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sp-core",
|
||||
"tracing-core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -9135,6 +9158,7 @@ dependencies = [
|
||||
"sp-std",
|
||||
"sp-trie",
|
||||
"thiserror",
|
||||
"tracing",
|
||||
"trie-db",
|
||||
"trie-root",
|
||||
]
|
||||
@@ -9200,8 +9224,13 @@ dependencies = [
|
||||
name = "sp-tracing"
|
||||
version = "3.0.0"
|
||||
dependencies = [
|
||||
"erased-serde",
|
||||
"log",
|
||||
"parity-scale-codec",
|
||||
"parking_lot 0.10.2",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"slog",
|
||||
"sp-std",
|
||||
"tracing",
|
||||
"tracing-core",
|
||||
@@ -10206,9 +10235,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "tracing-attributes"
|
||||
version = "0.1.13"
|
||||
version = "0.1.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a8a9bd1db7706f2373a190b0d067146caa39350c486f3d455b0e33b431f94c07"
|
||||
checksum = "c42e6fa53307c8a17e4ccd4dc81cf5ec38db9209f59b222210375b54ee40d1e2"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
|
||||
@@ -30,3 +30,4 @@ serde = { version = "1.0.101", features = ["derive"] }
|
||||
serde_json = "1.0.41"
|
||||
sp-transaction-pool = { version = "3.0.0", path = "../../primitives/transaction-pool" }
|
||||
sp-rpc = { version = "3.0.0", path = "../../primitives/rpc" }
|
||||
sp-tracing = { version = "3.0.0", path = "../../primitives/tracing" }
|
||||
|
||||
@@ -136,4 +136,118 @@ pub trait StateApi<Hash> {
|
||||
fn unsubscribe_storage(
|
||||
&self, metadata: Option<Self::Metadata>, id: SubscriptionId
|
||||
) -> RpcResult<bool>;
|
||||
|
||||
/// The `state_traceBlock` RPC provides a way to trace the re-execution of a single
|
||||
/// block, collecting Spans and Events from both the client and the relevant WASM runtime.
|
||||
/// The Spans and Events are conceptually equivalent to those from the [Tracing][1] crate.
|
||||
///
|
||||
/// The structure of the traces follows that of the block execution pipeline, so meaningful
|
||||
/// interpretation of the traces requires an understanding of the Substrate chain's block
|
||||
/// execution.
|
||||
///
|
||||
/// [Link to conceptual map of trace structure for Polkadot and Kusama block execution.][2]
|
||||
///
|
||||
/// [1]: https://crates.io/crates/tracing
|
||||
/// [2]: https://docs.google.com/drawings/d/1vZoJo9jaXlz0LmrdTOgHck9_1LsfuQPRmTr-5g1tOis/edit?usp=sharing
|
||||
///
|
||||
/// ## Node requirements
|
||||
///
|
||||
/// - Fully synced archive node (i.e. a node that is not actively doing a "major" sync).
|
||||
/// - [Tracing enabled WASM runtimes](#creating-tracing-enabled-wasm-runtimes) for all runtime versions
|
||||
/// for which tracing is desired.
|
||||
///
|
||||
/// ## Node recommendations
|
||||
///
|
||||
/// - Use fast SSD disk storage.
|
||||
/// - Run node flags to increase DB read speed (i.e. `--state-cache-size`, `--db-cache`).
|
||||
///
|
||||
/// ## Creating tracing enabled WASM runtimes
|
||||
///
|
||||
/// - Checkout commit of chain version to compile with WASM traces
|
||||
/// - [diener][1] can help to peg commit of substrate to what the chain expects.
|
||||
/// - Navigate to the `runtime` folder/package of the chain
|
||||
/// - Add feature `with-tracing = ["frame-executive/with-tracing", "sp-io/with-tracing"]`
|
||||
/// under `[features]` to the `runtime` packages' `Cargo.toml`.
|
||||
/// - Compile the runtime with `cargo build --release --features with-tracing`
|
||||
/// - Tracing-enabled WASM runtime should be found in `./target/release/wbuild/{{chain}}-runtime`
|
||||
/// and be called something like `{{your_chain}}_runtime.compact.wasm`. This can be
|
||||
/// renamed/modified however you like, as long as it retains the `.wasm` extension.
|
||||
/// - Run the node with the wasm blob overrides by placing them in a folder with all your runtimes,
|
||||
/// and passing the path of this folder to your chain, e.g.:
|
||||
/// - `./target/release/polkadot --wasm-runtime-overrides /home/user/my-custom-wasm-runtimes`
|
||||
///
|
||||
/// You can also find some pre-built tracing enabled wasm runtimes in [substrate-archive][2]
|
||||
///
|
||||
/// [Source.][3]
|
||||
///
|
||||
/// [1]: https://crates.io/crates/diener
|
||||
/// [2]: https://github.com/paritytech/substrate-archive/tree/master/wasm-tracing
|
||||
/// [3]: https://github.com/paritytech/substrate-archive/wiki
|
||||
///
|
||||
/// ## RPC Usage
|
||||
///
|
||||
/// The RPC allows for two filtering mechanisms: tracing targets and storage key prefixes.
|
||||
/// The filtering of spans and events takes place after they are all collected; so while filters
|
||||
/// do not reduce time for actual block re-execution, they reduce the response payload size.
|
||||
///
|
||||
/// Note: storage events primarily come from _primitives/state-machine/src/ext.rs_.
|
||||
/// The default filters can be overridden, see the [params section](#params) for details.
|
||||
///
|
||||
/// ### `curl` example
|
||||
///
|
||||
/// ```text
|
||||
/// curl \
|
||||
/// -H "Content-Type: application/json" \
|
||||
/// -d '{"id":1, "jsonrpc":"2.0", "method": "state_traceBlock", \
|
||||
/// "params": ["0xb246acf1adea1f801ce15c77a5fa7d8f2eb8fed466978bcee172cc02cf64e264"]}' \
|
||||
/// http://localhost:9933/
|
||||
/// ```
|
||||
///
|
||||
/// ### Params
|
||||
///
|
||||
/// - `block_hash` (param index 0): Hash of the block to trace.
|
||||
/// - `targets` (param index 1): String of comma separated (no spaces) targets. Specified
|
||||
/// targets match with trace targets by prefix (i.e if a target is in the beginning
|
||||
/// of a trace target it is considered a match). If an empty string is specified no
|
||||
/// targets will be filtered out. The majority of targets correspond to Rust module names,
|
||||
/// and the ones that do not are typically "hardcoded" into span or event location
|
||||
/// somewhere in the Substrate source code. ("Non-hardcoded" targets typically come from frame
|
||||
/// support macros.)
|
||||
/// - `storage_keys` (param index 2): String of comma separated (no spaces) hex encoded
|
||||
/// (no `0x` prefix) storage keys. If an empty string is specified no events will
|
||||
/// be filtered out. If anything other than an empty string is specified, events
|
||||
/// will be filtered by storage key (so non-storage events will **not** show up).
|
||||
/// You can specify any length of a storage key prefix (i.e. if a specified storage
|
||||
/// key is in the beginning of an events storage key it is considered a match).
|
||||
/// Example: for balance tracking on Polkadot & Kusama you would likely want
|
||||
/// to track changes to account balances with the frame_system::Account storage item,
|
||||
/// which is a map from `AccountId` to `AccountInfo`. The key filter for this would be
|
||||
/// the storage prefix for the map:
|
||||
/// `26aa394eea5630e07c48ae0c9558cef7b99d880ec681799c0cf30e8886371da9`
|
||||
/// Additionally you would want to track the extrinsic index, which is under the
|
||||
/// `:extrinsic_index` key. The key for this would be the aforementioned string as bytes
|
||||
/// in hex: `3a65787472696e7369635f696e646578`.
|
||||
/// The following are some resources to learn more about storage keys in substrate:
|
||||
/// [substrate storage][1], [transparent keys in substrate][2],
|
||||
/// [querying substrate storage via rpc][3].
|
||||
///
|
||||
/// [1]: https://substrate.dev/docs/en/knowledgebase/advanced/storage#storage-map-key
|
||||
/// [2]: https://www.shawntabrizi.com/substrate/transparent-keys-in-substrate/
|
||||
/// [3]: https://www.shawntabrizi.com/substrate/querying-substrate-storage-via-rpc/
|
||||
///
|
||||
/// ### Maximum payload size
|
||||
///
|
||||
/// The maximum payload size allowed is 15mb. Payloads over this size will return a
|
||||
/// object with a simple error message. If you run into issues with payload size you can
|
||||
/// narrow down the traces using a smaller set of targets and/or storage keys.
|
||||
///
|
||||
/// If you are having issues with maximum payload size you can use the flag
|
||||
/// `-lstate_tracing=trace` to get some logging during tracing.
|
||||
#[rpc(name = "state_traceBlock")]
|
||||
fn trace_block(
|
||||
&self,
|
||||
block: Hash,
|
||||
targets: Option<String>,
|
||||
storage_keys: Option<String>,
|
||||
) -> FutureResult<sp_rpc::tracing::TraceBlockResponse>;
|
||||
}
|
||||
|
||||
@@ -28,7 +28,7 @@ use log::error;
|
||||
use pubsub::PubSubMetadata;
|
||||
|
||||
/// Maximal payload accepted by RPC servers.
|
||||
const MAX_PAYLOAD: usize = 15 * 1024 * 1024;
|
||||
pub const MAX_PAYLOAD: usize = 15 * 1024 * 1024;
|
||||
|
||||
/// Default maximum number of connections for WS RPC servers.
|
||||
const WS_MAX_CONNECTIONS: usize = 100;
|
||||
|
||||
@@ -41,6 +41,7 @@ sc-tracing = { version = "3.0.0", path = "../tracing" }
|
||||
hash-db = { version = "0.15.2", default-features = false }
|
||||
parking_lot = "0.11.1"
|
||||
lazy_static = { version = "1.4.0", optional = true }
|
||||
sp-tracing = { version = "3.0.0", path = "../../primitives/tracing" }
|
||||
|
||||
[dev-dependencies]
|
||||
assert_matches = "1.3.0"
|
||||
|
||||
@@ -40,7 +40,9 @@ use self::error::{Error, FutureResult};
|
||||
|
||||
pub use sc_rpc_api::state::*;
|
||||
pub use sc_rpc_api::child_state::*;
|
||||
use sc_client_api::{ExecutorProvider, StorageProvider, BlockchainEvents, Backend, ProofProvider};
|
||||
use sc_client_api::{
|
||||
ExecutorProvider, StorageProvider, BlockchainEvents, Backend, BlockBackend, ProofProvider
|
||||
};
|
||||
use sp_blockchain::{HeaderMetadata, HeaderBackend};
|
||||
|
||||
const STORAGE_KEYS_PAGED_MAX_COUNT: u32 = 1000;
|
||||
@@ -165,6 +167,14 @@ pub trait StateBackend<Block: BlockT, Client>: Send + Sync + 'static
|
||||
_meta: Option<crate::Metadata>,
|
||||
id: SubscriptionId,
|
||||
) -> RpcResult<bool>;
|
||||
|
||||
/// Trace storage changes for block
|
||||
fn trace_block(
|
||||
&self,
|
||||
block: Block::Hash,
|
||||
targets: Option<String>,
|
||||
storage_keys: Option<String>,
|
||||
) -> FutureResult<sp_rpc::tracing::TraceBlockResponse>;
|
||||
}
|
||||
|
||||
/// Create new state API that works on full node.
|
||||
@@ -176,9 +186,10 @@ pub fn new_full<BE, Block: BlockT, Client>(
|
||||
where
|
||||
Block: BlockT + 'static,
|
||||
BE: Backend<Block> + 'static,
|
||||
Client: ExecutorProvider<Block> + StorageProvider<Block, BE> + ProofProvider<Block> + HeaderBackend<Block>
|
||||
Client: ExecutorProvider<Block> + StorageProvider<Block, BE> + ProofProvider<Block>
|
||||
+ HeaderMetadata<Block, Error = sp_blockchain::Error> + BlockchainEvents<Block>
|
||||
+ CallApiAt<Block> + ProvideRuntimeApi<Block> + Send + Sync + 'static,
|
||||
+ CallApiAt<Block> + HeaderBackend<Block>
|
||||
+ BlockBackend<Block> + ProvideRuntimeApi<Block> + Send + Sync + 'static,
|
||||
Client::Api: Metadata<Block>,
|
||||
{
|
||||
let child_backend = Box::new(
|
||||
@@ -346,6 +357,23 @@ impl<Block, Client> StateApi<Block::Hash> for State<Block, Client>
|
||||
) -> RpcResult<bool> {
|
||||
self.backend.unsubscribe_runtime_version(meta, id)
|
||||
}
|
||||
|
||||
/// Re-execute the given block with the tracing targets given in `targets`
|
||||
/// and capture all state changes.
|
||||
///
|
||||
/// Note: requires the node to run with `--rpc-methods=Unsafe`.
|
||||
/// Note: requires runtimes compiled with wasm tracing support, `--features with-tracing`.
|
||||
fn trace_block(
|
||||
&self, block: Block::Hash,
|
||||
targets: Option<String>,
|
||||
storage_keys: Option<String>
|
||||
) -> FutureResult<sp_rpc::tracing::TraceBlockResponse> {
|
||||
if let Err(err) = self.deny_unsafe.check_if_safe() {
|
||||
return Box::new(result(Err(err.into())))
|
||||
}
|
||||
|
||||
self.backend.trace_block(block, targets, storage_keys)
|
||||
}
|
||||
}
|
||||
|
||||
/// Child state backend API.
|
||||
|
||||
@@ -27,9 +27,10 @@ use jsonrpc_pubsub::{typed::Subscriber, SubscriptionId, manager::SubscriptionMan
|
||||
use rpc::{Result as RpcResult, futures::{stream, Future, Sink, Stream, future::result}};
|
||||
|
||||
use sc_rpc_api::state::ReadProof;
|
||||
use sc_client_api::backend::Backend;
|
||||
use sp_blockchain::{Result as ClientResult, Error as ClientError, HeaderMetadata, CachedHeaderMetadata, HeaderBackend};
|
||||
use sc_client_api::BlockchainEvents;
|
||||
use sp_blockchain::{
|
||||
Result as ClientResult, Error as ClientError, HeaderMetadata, CachedHeaderMetadata,
|
||||
HeaderBackend
|
||||
};
|
||||
use sp_core::{
|
||||
Bytes, storage::{well_known_keys, StorageKey, StorageData, StorageChangeSet,
|
||||
ChildInfo, ChildType, PrefixedStorageKey},
|
||||
@@ -43,7 +44,10 @@ use sp_api::{Metadata, ProvideRuntimeApi, CallApiAt};
|
||||
|
||||
use super::{StateBackend, ChildStateBackend, error::{FutureResult, Error, Result}, client_err};
|
||||
use std::marker::PhantomData;
|
||||
use sc_client_api::{CallExecutor, StorageProvider, ExecutorProvider, ProofProvider};
|
||||
use sc_client_api::{
|
||||
Backend, BlockBackend, BlockchainEvents, CallExecutor, StorageProvider, ExecutorProvider,
|
||||
ProofProvider
|
||||
};
|
||||
|
||||
/// Ranges to query in state_queryStorage.
|
||||
struct QueryStorageRange<Block: BlockT> {
|
||||
@@ -69,7 +73,7 @@ pub struct FullState<BE, Block: BlockT, Client> {
|
||||
impl<BE, Block: BlockT, Client> FullState<BE, Block, Client>
|
||||
where
|
||||
BE: Backend<Block>,
|
||||
Client: StorageProvider<Block, BE> + HeaderBackend<Block>
|
||||
Client: StorageProvider<Block, BE> + HeaderBackend<Block> + BlockBackend<Block>
|
||||
+ HeaderMetadata<Block, Error = sp_blockchain::Error>,
|
||||
Block: BlockT + 'static,
|
||||
{
|
||||
@@ -221,9 +225,11 @@ impl<BE, Block: BlockT, Client> FullState<BE, Block, Client>
|
||||
impl<BE, Block, Client> StateBackend<Block, Client> for FullState<BE, Block, Client> where
|
||||
Block: BlockT + 'static,
|
||||
BE: Backend<Block> + 'static,
|
||||
Client: ExecutorProvider<Block> + StorageProvider<Block, BE> + ProofProvider<Block> + HeaderBackend<Block>
|
||||
Client: ExecutorProvider<Block> + StorageProvider<Block, BE>
|
||||
+ ProofProvider<Block> + HeaderBackend<Block>
|
||||
+ HeaderMetadata<Block, Error = sp_blockchain::Error> + BlockchainEvents<Block>
|
||||
+ CallApiAt<Block> + ProvideRuntimeApi<Block>
|
||||
+ BlockBackend<Block>
|
||||
+ Send + Sync + 'static,
|
||||
Client::Api: Metadata<Block>,
|
||||
{
|
||||
@@ -527,12 +533,26 @@ impl<BE, Block, Client> StateBackend<Block, Client> for FullState<BE, Block, Cli
|
||||
) -> RpcResult<bool> {
|
||||
Ok(self.subscriptions.cancel(id))
|
||||
}
|
||||
|
||||
fn trace_block(
|
||||
&self,
|
||||
block: Block::Hash,
|
||||
targets: Option<String>,
|
||||
storage_keys: Option<String>,
|
||||
) -> FutureResult<sp_rpc::tracing::TraceBlockResponse> {
|
||||
Box::new(result(
|
||||
sc_tracing::block::BlockExecutor::new(self.client.clone(), block, targets, storage_keys)
|
||||
.trace_block()
|
||||
.map_err(|e| invalid_block::<Block>(block, None, e.to_string()))
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
impl<BE, Block, Client> ChildStateBackend<Block, Client> for FullState<BE, Block, Client> where
|
||||
Block: BlockT + 'static,
|
||||
BE: Backend<Block> + 'static,
|
||||
Client: ExecutorProvider<Block> + StorageProvider<Block, BE> + HeaderBackend<Block>
|
||||
Client: ExecutorProvider<Block> + StorageProvider<Block, BE>
|
||||
+ HeaderBackend<Block> + BlockBackend<Block>
|
||||
+ HeaderMetadata<Block, Error = sp_blockchain::Error> + BlockchainEvents<Block>
|
||||
+ CallApiAt<Block> + ProvideRuntimeApi<Block>
|
||||
+ Send + Sync + 'static,
|
||||
|
||||
@@ -474,6 +474,15 @@ impl<Block, F, Client> StateBackend<Block, Client> for LightState<Block, F, Clie
|
||||
) -> RpcResult<bool> {
|
||||
Ok(self.subscriptions.cancel(id))
|
||||
}
|
||||
|
||||
fn trace_block(
|
||||
&self,
|
||||
_block: Block::Hash,
|
||||
_targets: Option<String>,
|
||||
_storage_keys: Option<String>,
|
||||
) -> FutureResult<sp_rpc::tracing::TraceBlockResponse> {
|
||||
Box::new(result(Err(client_err(ClientError::NotAvailableOnLightClient))))
|
||||
}
|
||||
}
|
||||
|
||||
impl<Block, F, Client> ChildStateBackend<Block, Client> for LightState<Block, F, Client>
|
||||
|
||||
@@ -34,6 +34,7 @@ pub use sc_rpc_api::system::*;
|
||||
pub use self::helpers::{SystemInfo, Health, PeerInfo, NodeRole, SyncState};
|
||||
pub use self::gen_client::Client as SystemClient;
|
||||
|
||||
/// Early exit for RPCs that require `--rpc-methods=Unsafe` to be enabled
|
||||
macro_rules! bail_if_unsafe {
|
||||
($value: expr) => {
|
||||
if let Err(err) = $value.check_if_safe() {
|
||||
|
||||
@@ -81,15 +81,24 @@ where
|
||||
Block: BlockT,
|
||||
B: backend::Backend<Block>,
|
||||
{
|
||||
let code = self.wasm_override
|
||||
let code = if let Some(d) = self.wasm_override
|
||||
.as_ref()
|
||||
.map::<sp_blockchain::Result<Option<RuntimeCode>>, _>(|o| {
|
||||
let spec = self.runtime_version(id)?.spec_version;
|
||||
Ok(o.get(&spec, onchain_code.heap_pages))
|
||||
})
|
||||
.transpose()?
|
||||
.flatten()
|
||||
.unwrap_or(onchain_code);
|
||||
.flatten() {
|
||||
log::debug!(target: "wasm_overrides", "using WASM override for block {}", id);
|
||||
d
|
||||
} else {
|
||||
log::debug!(
|
||||
target: "wasm_overrides",
|
||||
"No WASM override available for block {}, using onchain code",
|
||||
id
|
||||
);
|
||||
onchain_code
|
||||
};
|
||||
|
||||
Ok(code)
|
||||
}
|
||||
|
||||
@@ -161,7 +161,19 @@ where
|
||||
Some("wasm") => {
|
||||
let wasm = WasmBlob::new(fs::read(&path).map_err(handle_err)?);
|
||||
let version = Self::runtime_version(executor, &wasm, Some(128))?;
|
||||
log::info!(
|
||||
target: "wasm_overrides",
|
||||
"Found wasm override in file: `{:?}`, version: {}",
|
||||
path.to_str(),
|
||||
version,
|
||||
);
|
||||
if let Some(_duplicate) = overrides.insert(version.spec_version, wasm) {
|
||||
log::info!(
|
||||
target: "wasm_overrides",
|
||||
"Found duplicate spec version for runtime in file: `{:?}`, version: {}",
|
||||
path.to_str(),
|
||||
version,
|
||||
);
|
||||
duplicates.push(format!("{}", path.display()));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,22 +15,32 @@ targets = ["x86_64-unknown-linux-gnu"]
|
||||
[dependencies]
|
||||
ansi_term = "0.12.1"
|
||||
atty = "0.2.13"
|
||||
erased-serde = "0.3.9"
|
||||
lazy_static = "1.4.0"
|
||||
log = { version = "0.4.8" }
|
||||
once_cell = "1.4.1"
|
||||
parking_lot = "0.11.1"
|
||||
regex = "1.4.2"
|
||||
rustc-hash = "1.1.0"
|
||||
erased-serde = "0.3.9"
|
||||
serde = "1.0.101"
|
||||
serde_json = "1.0.41"
|
||||
thiserror = "1.0.21"
|
||||
tracing = "0.1.25"
|
||||
tracing-core = "0.1.17"
|
||||
tracing-log = "0.1.1"
|
||||
tracing-subscriber = "0.2.15"
|
||||
sp-tracing = { version = "3.0.0", path = "../../primitives/tracing" }
|
||||
sp-rpc = { version = "3.0.0", path = "../../primitives/rpc" }
|
||||
sp-block-builder = { version = "3.0.0", path = "../../primitives/block-builder" }
|
||||
sp-storage = { version = "3.0.0", path = "../../primitives/storage" }
|
||||
sp-runtime = { version = "3.0.0", path = "../../primitives/runtime" }
|
||||
sp-blockchain = { version = "3.0.0", path = "../../primitives/blockchain" }
|
||||
sp-api = { version = "3.0.0", path = "../../primitives/api" }
|
||||
sp-core = { version = "3.0.0", path = "../../primitives/core" }
|
||||
sc-telemetry = { version = "3.0.0", path = "../telemetry" }
|
||||
sc-client-api = { version = "3.0.0", path = "../api" }
|
||||
sc-tracing-proc-macro = { version = "3.0.0", path = "./proc-macro" }
|
||||
sc-rpc-server = { version = "3.0.0", path = "../rpc-servers" }
|
||||
wasm-timer = "0.2"
|
||||
|
||||
[target.'cfg(target_os = "unknown")'.dependencies]
|
||||
wasm-bindgen = "0.2.67"
|
||||
|
||||
@@ -0,0 +1,338 @@
|
||||
// Copyright 2021 Parity Technologies (UK) Ltd.
|
||||
// This file is part of Substrate.
|
||||
|
||||
// Substrate is free software: you can redistribute it and/or modify
|
||||
// it under the terms of the GNU General Public License as published by
|
||||
// the Free Software Foundation, either version 3 of the License, or
|
||||
// (at your option) any later version.
|
||||
|
||||
// Substrate is distributed in the hope that it will be useful,
|
||||
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
// GNU General Public License for more details.
|
||||
|
||||
// You should have received a copy of the GNU General Public License
|
||||
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
//! Utilities for tracing block execution
|
||||
|
||||
use std::{collections::HashMap, sync::{Arc, atomic::{AtomicU64, Ordering}}, time::Instant};
|
||||
|
||||
use parking_lot::Mutex;
|
||||
use tracing::{Dispatch, dispatcher, Subscriber, Level, span::{Attributes, Record, Id}};
|
||||
use tracing_subscriber::CurrentSpan;
|
||||
|
||||
use sc_client_api::BlockBackend;
|
||||
use sc_rpc_server::MAX_PAYLOAD;
|
||||
use sp_api::{Core, Metadata, ProvideRuntimeApi, Encode};
|
||||
use sp_blockchain::HeaderBackend;
|
||||
use sp_runtime::{
|
||||
generic::BlockId,
|
||||
traits::{Block as BlockT, Header},
|
||||
};
|
||||
use sp_rpc::tracing::{BlockTrace, Span, TraceError, TraceBlockResponse};
|
||||
use sp_tracing::{WASM_NAME_KEY, WASM_TARGET_KEY, WASM_TRACE_IDENTIFIER};
|
||||
use sp_core::hexdisplay::HexDisplay;
|
||||
use crate::{SpanDatum, TraceEvent, Values};
|
||||
|
||||
// Heuristic for average event size in bytes.
|
||||
const AVG_EVENT: usize = 600 * 8;
|
||||
// Heuristic for average span size in bytes.
|
||||
const AVG_SPAN: usize = 100 * 8;
|
||||
// Estimate of the max base RPC payload size when the Id is bound as a u64. If strings
|
||||
// are used for the RPC Id this may need to be adjusted. Note: The base payload
|
||||
// does not include the RPC result.
|
||||
//
|
||||
// The estimate is based on the JSONRPC response message which has the following format:
|
||||
// `{"jsonrpc":"2.0","result":[],"id":18446744073709551615}`.
|
||||
//
|
||||
// We care about the total size of the payload because jsonrpc-server will simply ignore
|
||||
// messages larger than `sc_rpc_server::MAX_PAYLOAD` and the caller will not get any
|
||||
// response.
|
||||
const BASE_PAYLOAD: usize = 100;
|
||||
// Default to only pallet, frame support and state related traces
|
||||
const DEFAULT_TARGETS: &str = "pallet,frame,state";
|
||||
const TRACE_TARGET: &str = "block_trace";
|
||||
// The name of a field required for all events.
|
||||
const REQUIRED_EVENT_FIELD: &str = "method";
|
||||
|
||||
/// Tracing Block Result type alias
|
||||
pub type TraceBlockResult<T> = Result<T, Error>;
|
||||
|
||||
/// Tracing Block error
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[allow(missing_docs)]
|
||||
#[non_exhaustive]
|
||||
pub enum Error {
|
||||
#[error("Invalid block Id: {0}")]
|
||||
InvalidBlockId(#[from] sp_blockchain::Error),
|
||||
#[error("Missing block component: {0}")]
|
||||
MissingBlockComponent(String),
|
||||
#[error("Dispatch error: {0}")]
|
||||
Dispatch(String)
|
||||
}
|
||||
|
||||
struct BlockSubscriber {
|
||||
targets: Vec<(String, Level)>,
|
||||
next_id: AtomicU64,
|
||||
current_span: CurrentSpan,
|
||||
spans: Mutex<HashMap<Id, SpanDatum>>,
|
||||
events: Mutex<Vec<TraceEvent>>,
|
||||
}
|
||||
|
||||
impl BlockSubscriber {
|
||||
fn new(targets: &str) -> Self {
|
||||
let next_id = AtomicU64::new(1);
|
||||
let mut targets: Vec<_> = targets
|
||||
.split(',')
|
||||
.map(crate::parse_target)
|
||||
.collect();
|
||||
// Ensure that WASM traces are always enabled
|
||||
// Filtering happens when decoding the actual target / level
|
||||
targets.push((WASM_TRACE_IDENTIFIER.to_owned(), Level::TRACE));
|
||||
BlockSubscriber {
|
||||
targets,
|
||||
next_id,
|
||||
current_span: CurrentSpan::default(),
|
||||
spans: Mutex::new(HashMap::new()),
|
||||
events: Mutex::new(Vec::new()),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Subscriber for BlockSubscriber {
|
||||
fn enabled(&self, metadata: &tracing::Metadata<'_>) -> bool {
|
||||
if !metadata.is_span() && !metadata.fields().field(REQUIRED_EVENT_FIELD).is_some() {
|
||||
return false;
|
||||
}
|
||||
for (target, level) in &self.targets {
|
||||
if metadata.level() <= level && metadata.target().starts_with(target) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
fn new_span(&self, attrs: &Attributes<'_>) -> Id {
|
||||
let id = Id::from_u64(self.next_id.fetch_add(1, Ordering::Relaxed));
|
||||
let mut values = Values::default();
|
||||
attrs.record(&mut values);
|
||||
let parent_id = attrs.parent().cloned()
|
||||
.or_else(|| self.current_span.id());
|
||||
let span = SpanDatum {
|
||||
id: id.clone(),
|
||||
parent_id,
|
||||
name: attrs.metadata().name().to_owned(),
|
||||
target: attrs.metadata().target().to_owned(),
|
||||
level: *attrs.metadata().level(),
|
||||
line: attrs.metadata().line().unwrap_or(0),
|
||||
start_time: Instant::now(),
|
||||
values,
|
||||
overall_time: Default::default()
|
||||
};
|
||||
|
||||
self.spans.lock().insert(id.clone(), span);
|
||||
id
|
||||
}
|
||||
|
||||
fn record(&self, span: &Id, values: &Record<'_>) {
|
||||
let mut span_data = self.spans.lock();
|
||||
if let Some(s) = span_data.get_mut(span) {
|
||||
values.record(&mut s.values);
|
||||
}
|
||||
}
|
||||
|
||||
fn record_follows_from(&self, _span: &Id, _follows: &Id) {
|
||||
// Not currently used
|
||||
unimplemented!("record_follows_from is not implemented");
|
||||
}
|
||||
|
||||
fn event(&self, event: &tracing::Event<'_>) {
|
||||
let mut values = crate::Values::default();
|
||||
event.record(&mut values);
|
||||
let parent_id = event.parent().cloned()
|
||||
.or_else(|| self.current_span.id());
|
||||
let trace_event = TraceEvent {
|
||||
name: event.metadata().name().to_owned(),
|
||||
target: event.metadata().target().to_owned(),
|
||||
level: *event.metadata().level(),
|
||||
values,
|
||||
parent_id,
|
||||
};
|
||||
self.events.lock().push(trace_event);
|
||||
}
|
||||
|
||||
fn enter(&self, id: &Id) {
|
||||
self.current_span.enter(id.clone());
|
||||
}
|
||||
|
||||
fn exit(&self, span: &Id) {
|
||||
if self.spans.lock().contains_key(span) {
|
||||
self.current_span.exit();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Holds a reference to the client in order to execute the given block.
|
||||
/// Records spans & events for the supplied targets (eg. "pallet,frame,state") and
|
||||
/// only records events with the specified hex encoded storage key prefixes.
|
||||
/// Note: if `targets` or `storage_keys` is an empty string then nothing is
|
||||
/// filtered out.
|
||||
pub struct BlockExecutor<Block: BlockT, Client> {
|
||||
client: Arc<Client>,
|
||||
block: Block::Hash,
|
||||
targets: Option<String>,
|
||||
storage_keys: Option<String>,
|
||||
}
|
||||
|
||||
impl<Block, Client> BlockExecutor<Block, Client>
|
||||
where
|
||||
Block: BlockT + 'static,
|
||||
Client: HeaderBackend<Block> + BlockBackend<Block> + ProvideRuntimeApi<Block>
|
||||
+ Send + Sync + 'static,
|
||||
Client::Api: Metadata<Block>,
|
||||
{
|
||||
/// Create a new `BlockExecutor`
|
||||
pub fn new(
|
||||
client: Arc<Client>,
|
||||
block: Block::Hash,
|
||||
targets: Option<String>,
|
||||
storage_keys: Option<String>,
|
||||
) -> Self {
|
||||
Self { client, block, targets, storage_keys }
|
||||
}
|
||||
|
||||
/// Execute block, record all spans and events belonging to `Self::targets`
|
||||
/// and filter out events which do not have keys starting with one of the
|
||||
/// prefixes in `Self::storage_keys`.
|
||||
pub fn trace_block(&self) -> TraceBlockResult<TraceBlockResponse> {
|
||||
tracing::debug!(target: "state_tracing", "Tracing block: {}", self.block);
|
||||
// Prepare the block
|
||||
let id = BlockId::Hash(self.block);
|
||||
let mut header = self.client.header(id)
|
||||
.map_err(|e| Error::InvalidBlockId(e))?
|
||||
.ok_or_else(|| Error::MissingBlockComponent("Header not found".to_string()))?;
|
||||
let extrinsics = self.client.block_body(&id)
|
||||
.map_err(|e| Error::InvalidBlockId(e))?
|
||||
.ok_or_else(|| Error::MissingBlockComponent("Extrinsics not found".to_string()))?;
|
||||
tracing::debug!(target: "state_tracing", "Found {} extrinsics", extrinsics.len());
|
||||
let parent_hash = *header.parent_hash();
|
||||
let parent_id = BlockId::Hash(parent_hash);
|
||||
// Remove all `Seal`s as they are added by the consensus engines after building the block.
|
||||
// On import they are normally removed by the consensus engine.
|
||||
header.digest_mut().logs.retain(|d| d.as_seal().is_none());
|
||||
let block = Block::new(header, extrinsics);
|
||||
|
||||
let targets = if let Some(t) = &self.targets { t } else { DEFAULT_TARGETS };
|
||||
let block_subscriber = BlockSubscriber::new(targets);
|
||||
let dispatch = Dispatch::new(block_subscriber);
|
||||
|
||||
{
|
||||
let dispatcher_span = tracing::debug_span!(
|
||||
target: "state_tracing",
|
||||
"execute_block",
|
||||
extrinsics_len = block.extrinsics().len(),
|
||||
);
|
||||
let _guard = dispatcher_span.enter();
|
||||
if let Err(e) = dispatcher::with_default(&dispatch, || {
|
||||
let span = tracing::info_span!(
|
||||
target: TRACE_TARGET,
|
||||
"trace_block",
|
||||
);
|
||||
let _enter = span.enter();
|
||||
self.client.runtime_api().execute_block(&parent_id, block)
|
||||
}) {
|
||||
return Err(Error::Dispatch(format!("Failed to collect traces and execute block: {:?}", e).to_string()));
|
||||
}
|
||||
}
|
||||
|
||||
let block_subscriber = dispatch.downcast_ref::<BlockSubscriber>()
|
||||
.ok_or(Error::Dispatch(
|
||||
"Cannot downcast Dispatch to BlockSubscriber after tracing block".to_string()
|
||||
))?;
|
||||
let spans: Vec<_> = block_subscriber.spans
|
||||
.lock()
|
||||
.drain()
|
||||
// Patch wasm identifiers
|
||||
.filter_map(|(_, s)| patch_and_filter(SpanDatum::from(s), targets))
|
||||
.collect();
|
||||
let events: Vec<_> = block_subscriber.events
|
||||
.lock()
|
||||
.drain(..)
|
||||
.filter(|e| self.storage_keys
|
||||
.as_ref()
|
||||
.map(|keys| event_key_filter(e, keys))
|
||||
.unwrap_or(false)
|
||||
)
|
||||
.map(|s| s.into())
|
||||
.collect();
|
||||
tracing::debug!(target: "state_tracing", "Captured {} spans and {} events", spans.len(), events.len());
|
||||
|
||||
let approx_payload_size = BASE_PAYLOAD + events.len() * AVG_EVENT + spans.len() * AVG_SPAN;
|
||||
let response = if approx_payload_size > MAX_PAYLOAD {
|
||||
TraceBlockResponse::TraceError(TraceError {
|
||||
error:
|
||||
"Payload likely exceeds max payload size of RPC server.".to_string()
|
||||
})
|
||||
} else {
|
||||
TraceBlockResponse::BlockTrace(BlockTrace {
|
||||
block_hash: block_id_as_string(id),
|
||||
parent_hash: block_id_as_string(parent_id),
|
||||
tracing_targets: targets.to_string(),
|
||||
storage_keys: self.storage_keys.clone().unwrap_or_default(),
|
||||
spans,
|
||||
events,
|
||||
})
|
||||
};
|
||||
|
||||
Ok(response)
|
||||
}
|
||||
}
|
||||
|
||||
fn event_key_filter(event: &TraceEvent, storage_keys: &str) -> bool {
|
||||
event.values.string_values.get("key")
|
||||
.and_then(|key| Some(check_target(storage_keys, key, &event.level)))
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// Filter out spans that do not match our targets and if the span is from WASM update its `name`
|
||||
/// and `target` fields to the WASM values for those fields.
|
||||
//
|
||||
// The `tracing` crate requires trace metadata to be static. This does not work for wasm code in
|
||||
// substrate, as it is regularly updated with new code from on-chain events. The workaround for this
|
||||
// is for substrate's WASM tracing wrappers to put the `name` and `target` data in the `values` map
|
||||
// (normally they would be in the static metadata assembled at compile time). Here, if a special
|
||||
// WASM `name` or `target` key is found in the `values` we remove it and put the key value pair in
|
||||
// the span's metadata, making it consistent with spans that come from native code.
|
||||
fn patch_and_filter(mut span: SpanDatum, targets: &str) -> Option<Span> {
|
||||
if span.name == WASM_TRACE_IDENTIFIER {
|
||||
span.values.bool_values.insert("wasm".to_owned(), true);
|
||||
if let Some(n) = span.values.string_values.remove(WASM_NAME_KEY) {
|
||||
span.name = n;
|
||||
}
|
||||
if let Some(t) = span.values.string_values.remove(WASM_TARGET_KEY) {
|
||||
span.target = t;
|
||||
}
|
||||
if !check_target(targets, &span.target, &span.level) {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
Some(span.into())
|
||||
}
|
||||
|
||||
/// Check if a `target` matches any `targets` by prefix
|
||||
fn check_target(targets: &str, target: &str, level: &Level) -> bool {
|
||||
for (t, l) in targets.split(',').map(crate::parse_target) {
|
||||
if target.starts_with(t.as_str()) && level <= &l {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
fn block_id_as_string<T: BlockT>(block_id: BlockId<T>) -> String {
|
||||
match block_id {
|
||||
BlockId::Hash(h) => HexDisplay::from(&h.encode()).to_string(),
|
||||
BlockId::Number(n) => HexDisplay::from(&n.encode()).to_string()
|
||||
}
|
||||
}
|
||||
@@ -29,6 +29,7 @@
|
||||
#![warn(missing_docs)]
|
||||
|
||||
pub mod logging;
|
||||
pub mod block;
|
||||
|
||||
use rustc_hash::FxHashMap;
|
||||
use std::fmt;
|
||||
@@ -86,7 +87,7 @@ pub trait TraceHandler: Send + Sync {
|
||||
#[derive(Debug)]
|
||||
pub struct TraceEvent {
|
||||
/// Name of the event.
|
||||
pub name: &'static str,
|
||||
pub name: String,
|
||||
/// Target of the event.
|
||||
pub target: String,
|
||||
/// Level of the event.
|
||||
@@ -123,13 +124,13 @@ pub struct SpanDatum {
|
||||
/// Holds associated values for a tracing span
|
||||
#[derive(Default, Clone, Debug)]
|
||||
pub struct Values {
|
||||
/// HashMap of `bool` values
|
||||
/// FxHashMap of `bool` values
|
||||
pub bool_values: FxHashMap<String, bool>,
|
||||
/// HashMap of `i64` values
|
||||
/// FxHashMap of `i64` values
|
||||
pub i64_values: FxHashMap<String, i64>,
|
||||
/// HashMap of `u64` values
|
||||
/// FxHashMap of `u64` values
|
||||
pub u64_values: FxHashMap<String, u64>,
|
||||
/// HashMap of `String` values
|
||||
/// FxHashMap of `String` values
|
||||
pub string_values: FxHashMap<String, String>,
|
||||
}
|
||||
|
||||
@@ -265,7 +266,7 @@ impl<S: Subscriber> Layer<S> for ProfilingLayer {
|
||||
parent_id: attrs.parent().cloned().or_else(|| self.current_span.id()),
|
||||
name: attrs.metadata().name().to_owned(),
|
||||
target: attrs.metadata().target().to_owned(),
|
||||
level: attrs.metadata().level().clone(),
|
||||
level: *attrs.metadata().level(),
|
||||
line: attrs.metadata().line().unwrap_or(0),
|
||||
start_time: Instant::now(),
|
||||
overall_time: ZERO_DURATION,
|
||||
@@ -285,9 +286,9 @@ impl<S: Subscriber> Layer<S> for ProfilingLayer {
|
||||
let mut values = Values::default();
|
||||
event.record(&mut values);
|
||||
let trace_event = TraceEvent {
|
||||
name: event.metadata().name(),
|
||||
name: event.metadata().name().to_owned(),
|
||||
target: event.metadata().target().to_owned(),
|
||||
level: event.metadata().level().clone(),
|
||||
level: *event.metadata().level(),
|
||||
values,
|
||||
parent_id: event.parent().cloned().or_else(|| self.current_span.id()),
|
||||
};
|
||||
@@ -304,7 +305,6 @@ impl<S: Subscriber> Layer<S> for ProfilingLayer {
|
||||
}
|
||||
|
||||
fn on_exit(&self, span: &Id, _ctx: Context<S>) {
|
||||
self.current_span.exit();
|
||||
let end_time = Instant::now();
|
||||
let span_datum = {
|
||||
let mut span_data = self.span_data.lock();
|
||||
@@ -312,6 +312,8 @@ impl<S: Subscriber> Layer<S> for ProfilingLayer {
|
||||
};
|
||||
|
||||
if let Some(mut span_datum) = span_datum {
|
||||
// If `span_datum` is `None` we don't exit (we'd be exiting the parent span)
|
||||
self.current_span.exit();
|
||||
span_datum.overall_time += end_time - span_datum.start_time;
|
||||
if span_datum.name == WASM_TRACE_IDENTIFIER {
|
||||
span_datum.values.bool_values.insert("wasm".to_owned(), true);
|
||||
@@ -330,9 +332,7 @@ impl<S: Subscriber> Layer<S> for ProfilingLayer {
|
||||
};
|
||||
}
|
||||
|
||||
fn on_close(&self, span: Id, ctx: Context<S>) {
|
||||
self.on_exit(&span, ctx)
|
||||
}
|
||||
fn on_close(&self, _span: Id, _ctx: Context<S>) {}
|
||||
}
|
||||
|
||||
/// TraceHandler for sending span data to the logger
|
||||
@@ -385,6 +385,32 @@ impl TraceHandler for LogTraceHandler {
|
||||
}
|
||||
}
|
||||
|
||||
impl From<TraceEvent> for sp_rpc::tracing::Event {
|
||||
fn from(trace_event: TraceEvent) -> Self {
|
||||
let data = sp_rpc::tracing::Data {
|
||||
string_values: trace_event.values.string_values
|
||||
};
|
||||
sp_rpc::tracing::Event {
|
||||
target: trace_event.target,
|
||||
data,
|
||||
parent_id: trace_event.parent_id.map(|id| id.into_u64())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<SpanDatum> for sp_rpc::tracing::Span {
|
||||
fn from(span_datum: SpanDatum) -> Self {
|
||||
let wasm = span_datum.values.bool_values.get("wasm").is_some();
|
||||
sp_rpc::tracing::Span {
|
||||
id: span_datum.id.into_u64(),
|
||||
parent_id: span_datum.parent_id.map(|id| id.into_u64()),
|
||||
name: span_datum.name,
|
||||
target: span_datum.target,
|
||||
wasm,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -555,7 +581,7 @@ mod tests {
|
||||
break;
|
||||
}
|
||||
}
|
||||
// gard2 and span2 dropped / exited
|
||||
// guard2 and span2 dropped / exited
|
||||
});
|
||||
|
||||
// wait for Event to be dispatched and stored
|
||||
|
||||
@@ -177,6 +177,9 @@ where
|
||||
};
|
||||
let builder = FmtSubscriber::builder().with_env_filter(env_filter);
|
||||
|
||||
#[cfg(not(target_os = "unknown"))]
|
||||
let builder = builder.with_span_events(format::FmtSpan::NONE);
|
||||
|
||||
#[cfg(not(target_os = "unknown"))]
|
||||
let builder = builder.with_writer(std::io::stderr as _);
|
||||
|
||||
|
||||
@@ -15,6 +15,8 @@ targets = ["x86_64-unknown-linux-gnu"]
|
||||
[dependencies]
|
||||
serde = { version = "1.0.101", features = ["derive"] }
|
||||
sp-core = { version = "3.0.0", path = "../core" }
|
||||
tracing-core = "0.1.17"
|
||||
rustc-hash = "1.1.0"
|
||||
|
||||
[dev-dependencies]
|
||||
serde_json = "1.0.41"
|
||||
|
||||
@@ -21,6 +21,7 @@
|
||||
|
||||
pub mod number;
|
||||
pub mod list;
|
||||
pub mod tracing;
|
||||
|
||||
/// A util function to assert the result of serialization and deserialization is the same.
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
// This file is part of Substrate.
|
||||
|
||||
// Copyright (C) 2020 Parity Technologies (UK) Ltd.
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! Types for working with tracing data
|
||||
|
||||
use serde::{Serialize, Deserialize};
|
||||
|
||||
use rustc_hash::FxHashMap;
|
||||
|
||||
/// Container for all related spans and events for the block being traced.
|
||||
#[derive(Serialize, Deserialize, Clone, Debug)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct BlockTrace {
|
||||
/// Hash of the block being traced
|
||||
pub block_hash: String,
|
||||
/// Parent hash
|
||||
pub parent_hash: String,
|
||||
/// Module targets that were recorded by the tracing subscriber.
|
||||
/// Empty string means record all targets.
|
||||
pub tracing_targets: String,
|
||||
/// Storage key targets used to filter out events that do not have one of the storage keys.
|
||||
/// Empty string means do not filter out any events.
|
||||
pub storage_keys: String,
|
||||
/// Vec of tracing spans
|
||||
pub spans: Vec<Span>,
|
||||
/// Vec of tracing events
|
||||
pub events: Vec<Event>,
|
||||
}
|
||||
|
||||
/// Represents a tracing event, complete with recorded data.
|
||||
#[derive(Serialize, Deserialize, Clone, Debug)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct Event {
|
||||
/// Event target
|
||||
pub target: String,
|
||||
/// Associated data
|
||||
pub data: Data,
|
||||
/// Parent id, if it exists
|
||||
pub parent_id: Option<u64>,
|
||||
}
|
||||
|
||||
/// Represents a single instance of a tracing span.
|
||||
///
|
||||
/// Exiting a span does not imply that the span will not be re-entered.
|
||||
#[derive(Serialize, Deserialize, Clone, Debug)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct Span {
|
||||
/// id for this span
|
||||
pub id: u64,
|
||||
/// id of the parent span, if any
|
||||
pub parent_id: Option<u64>,
|
||||
/// Name of this span
|
||||
pub name: String,
|
||||
/// Target, typically module
|
||||
pub target: String,
|
||||
/// Indicates if the span is from wasm
|
||||
pub wasm: bool,
|
||||
}
|
||||
|
||||
/// Holds associated values for a tracing span.
|
||||
#[derive(Serialize, Deserialize, Default, Clone, Debug)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct Data {
|
||||
/// HashMap of `String` values recorded while tracing
|
||||
pub string_values: FxHashMap<String, String>,
|
||||
}
|
||||
|
||||
/// Error response for the `state_traceBlock` RPC.
|
||||
#[derive(Serialize, Deserialize, Default, Clone, Debug)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct TraceError {
|
||||
/// Error message
|
||||
pub error: String,
|
||||
}
|
||||
|
||||
/// Response for the `state_traceBlock` RPC.
|
||||
#[derive(Serialize, Deserialize, Clone, Debug)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub enum TraceBlockResponse {
|
||||
/// Error block tracing response
|
||||
TraceError(TraceError),
|
||||
/// Successful block tracing response
|
||||
BlockTrace(BlockTrace)
|
||||
}
|
||||
@@ -29,6 +29,7 @@ rand = { version = "0.7.2", optional = true }
|
||||
sp-externalities = { version = "0.9.0", path = "../externalities", default-features = false }
|
||||
smallvec = "1.4.1"
|
||||
sp-std = { version = "3.0.0", default-features = false, path = "../std" }
|
||||
tracing = { version = "0.1.22", optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
hex-literal = "0.3.1"
|
||||
@@ -52,4 +53,5 @@ std = [
|
||||
"parking_lot",
|
||||
"rand",
|
||||
"sp-panic-handler",
|
||||
"tracing"
|
||||
]
|
||||
|
||||
@@ -201,11 +201,22 @@ where
|
||||
let _guard = guard();
|
||||
let result = self.overlay.storage(key).map(|x| x.map(|x| x.to_vec())).unwrap_or_else(||
|
||||
self.backend.storage(key).expect(EXT_NOT_ALLOWED_TO_FAIL));
|
||||
trace!(target: "state", "{:04x}: Get {}={:?}",
|
||||
self.id,
|
||||
HexDisplay::from(&key),
|
||||
result.as_ref().map(HexDisplay::from)
|
||||
|
||||
// NOTE: be careful about touching the key names – used outside substrate!
|
||||
trace!(
|
||||
target: "state",
|
||||
method = "Get",
|
||||
ext_id = self.id,
|
||||
key = %HexDisplay::from(&key),
|
||||
result = ?result.as_ref().map(HexDisplay::from),
|
||||
result_encoded = %HexDisplay::from(
|
||||
&result
|
||||
.as_ref()
|
||||
.map(|v| EncodeOpaqueValue(v.clone()))
|
||||
.encode()
|
||||
),
|
||||
);
|
||||
|
||||
result
|
||||
}
|
||||
|
||||
@@ -354,17 +365,27 @@ where
|
||||
}
|
||||
|
||||
fn place_storage(&mut self, key: StorageKey, value: Option<StorageValue>) {
|
||||
trace!(target: "state", "{:04x}: Put {}={:?}",
|
||||
self.id,
|
||||
HexDisplay::from(&key),
|
||||
value.as_ref().map(HexDisplay::from)
|
||||
);
|
||||
let _guard = guard();
|
||||
if is_child_storage_key(&key) {
|
||||
warn!(target: "trie", "Refuse to directly set child storage key");
|
||||
return;
|
||||
}
|
||||
|
||||
// NOTE: be careful about touching the key names – used outside substrate!
|
||||
trace!(
|
||||
target: "state",
|
||||
method = "Put",
|
||||
ext_id = self.id,
|
||||
key = %HexDisplay::from(&key),
|
||||
value = ?value.as_ref().map(HexDisplay::from),
|
||||
value_encoded = %HexDisplay::from(
|
||||
&value
|
||||
.as_ref()
|
||||
.map(|v| EncodeOpaqueValue(v.clone()))
|
||||
.encode()
|
||||
),
|
||||
);
|
||||
|
||||
self.mark_dirty();
|
||||
self.overlay.set_storage(key, value);
|
||||
}
|
||||
|
||||
@@ -46,7 +46,9 @@ pub use std_reexport::*;
|
||||
#[cfg(feature = "std")]
|
||||
pub use execution::*;
|
||||
#[cfg(feature = "std")]
|
||||
pub use log::{debug, warn, trace, error as log_error};
|
||||
pub use log::{debug, warn, error as log_error};
|
||||
#[cfg(feature = "std")]
|
||||
pub use tracing::trace;
|
||||
|
||||
/// In no_std we skip logs for state_machine, this macro
|
||||
/// is a noops.
|
||||
|
||||
@@ -24,6 +24,11 @@ tracing = { version = "0.1.25", default-features = false }
|
||||
tracing-core = { version = "0.1.17", default-features = false }
|
||||
log = { version = "0.4.8", optional = true }
|
||||
tracing-subscriber = { version = "0.2.15", optional = true, features = ["tracing-log"] }
|
||||
parking_lot = { version = "0.10.0", optional = true }
|
||||
erased-serde = { version = "0.3.9", optional = true }
|
||||
serde = { version = "1.0.101", optional = true }
|
||||
serde_json = { version = "1.0.41", optional = true }
|
||||
slog = { version = "2.5.2", features = ["nested-values"], optional = true }
|
||||
|
||||
[features]
|
||||
default = [ "std" ]
|
||||
@@ -39,4 +44,9 @@ std = [
|
||||
"sp-std/std",
|
||||
"log",
|
||||
"tracing-subscriber",
|
||||
"parking_lot",
|
||||
"erased-serde",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"slog"
|
||||
]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
Substrate tracing primitives and macros.
|
||||
|
||||
To trace functions or invidual code in Substrate, this crate provides [`within_span`]
|
||||
To trace functions or individual code in Substrate, this crate provides [`within_span`]
|
||||
and [`enter_span`]. See the individual docs for how to use these macros.
|
||||
|
||||
Note that to allow traces from wasm execution environment there are
|
||||
|
||||
@@ -28,14 +28,36 @@
|
||||
//! Additionally, we have a const: `WASM_TRACE_IDENTIFIER`, which holds a span name used
|
||||
//! to signal that the 'actual' span name and target should be retrieved instead from
|
||||
//! the associated Fields mentioned above.
|
||||
//!
|
||||
//! Note: The `tracing` crate requires trace metadata to be static. This does not work
|
||||
//! for wasm code in substrate, as it is regularly updated with new code from on-chain
|
||||
//! events. The workaround for this is for the wasm tracing wrappers to put the
|
||||
//! `name` and `target` data in the `values` map (normally they would be in the static
|
||||
//! metadata assembled at compile time).
|
||||
|
||||
#![cfg_attr(not(feature = "std"), no_std)]
|
||||
|
||||
#[cfg(feature = "std")]
|
||||
use tracing;
|
||||
pub use tracing::{
|
||||
debug, debug_span, error, error_span, event, info, info_span, Level, span, Span,
|
||||
trace, trace_span, warn, warn_span,
|
||||
};
|
||||
|
||||
pub use crate::types::{
|
||||
WasmEntryAttributes, WasmFieldName, WasmFields, WasmLevel, WasmMetadata, WasmValue,
|
||||
WasmValuesSet
|
||||
};
|
||||
#[cfg(feature = "std")]
|
||||
pub use crate::types::{
|
||||
WASM_NAME_KEY, WASM_TARGET_KEY, WASM_TRACE_IDENTIFIER
|
||||
};
|
||||
|
||||
/// Tracing facilities and helpers.
|
||||
///
|
||||
/// This is modeled after the `tracing`/`tracing-core` interface and uses that more or
|
||||
/// less directly for the native side. Because of certain optimisations the these crates
|
||||
/// have done, the wasm implementation diverges slightly and is optimised for thtat use
|
||||
/// have done, the wasm implementation diverges slightly and is optimised for that use
|
||||
/// case (like being able to cross the wasm/native boundary via scale codecs).
|
||||
///
|
||||
/// One of said optimisations is that all macros will yield to a `noop` in non-std unless
|
||||
@@ -86,23 +108,9 @@
|
||||
/// and call `set_tracing_subscriber` at the very beginning of your execution –
|
||||
/// the default subscriber is doing nothing, so any spans or events happening before
|
||||
/// will not be recorded!
|
||||
///
|
||||
|
||||
mod types;
|
||||
|
||||
#[cfg(feature = "std")]
|
||||
use tracing;
|
||||
|
||||
pub use tracing::{
|
||||
debug, debug_span, error, error_span, info, info_span, trace, trace_span, warn, warn_span,
|
||||
span, event, Level, Span,
|
||||
};
|
||||
|
||||
pub use crate::types::{
|
||||
WasmMetadata, WasmEntryAttributes, WasmValuesSet, WasmValue, WasmFields, WasmLevel, WasmFieldName
|
||||
};
|
||||
|
||||
|
||||
/// Try to init a simple tracing subscriber with log compatibility layer.
|
||||
/// Ignores any error. Useful for testing.
|
||||
#[cfg(feature = "std")]
|
||||
@@ -112,12 +120,6 @@ pub fn try_init_simple() {
|
||||
.with_writer(std::io::stderr).try_init();
|
||||
}
|
||||
|
||||
#[cfg(feature = "std")]
|
||||
pub use crate::types::{
|
||||
WASM_NAME_KEY, WASM_TARGET_KEY, WASM_TRACE_IDENTIFIER
|
||||
};
|
||||
|
||||
|
||||
/// Runs given code within a tracing span, measuring it's execution time.
|
||||
///
|
||||
/// If tracing is not enabled, the code is still executed. Pass in level and name or
|
||||
|
||||
@@ -53,8 +53,6 @@ impl From<&tracing_core::Level> for WasmLevel {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
impl core::default::Default for WasmLevel {
|
||||
fn default() -> Self {
|
||||
WasmLevel::TRACE
|
||||
|
||||
Reference in New Issue
Block a user