feat: Rebrand Polkadot/Substrate references to PezkuwiChain
This commit systematically rebrands various references from Parity Technologies' Polkadot/Substrate ecosystem to PezkuwiChain within the kurdistan-sdk. Key changes include: - Updated external repository URLs (zombienet-sdk, parity-db, parity-scale-codec, wasm-instrument) to point to pezkuwichain forks. - Modified internal documentation and code comments to reflect PezkuwiChain naming and structure. - Replaced direct references to with or specific paths within the for XCM, Pezkuwi, and other modules. - Cleaned up deprecated issue and PR references in various and files, particularly in and modules. - Adjusted image and logo URLs in documentation to point to PezkuwiChain assets. - Removed or rephrased comments related to external Polkadot/Substrate PRs and issues. This is a significant step towards fully customizing the SDK for the PezkuwiChain ecosystem.
This commit is contained in:
@@ -0,0 +1,113 @@
|
||||
# The `benchmark block` command
|
||||
|
||||
The whole benchmarking process in Bizinikiwi aims to predict the resource usage of an unexecuted block. This command
|
||||
measures how accurate this prediction was by executing a block and comparing the predicted weight to its actual resource
|
||||
usage. It can be used to measure the accuracy of the pallet benchmarking.
|
||||
|
||||
In the following it will be explained once for PezkuwiChain and once for Bizinikiwi.
|
||||
|
||||
## PezkuwiChain # 1
|
||||
<sup>(Also works for Kusama, zagros and pezkuwichain)</sup>
|
||||
|
||||
|
||||
Suppose you either have a synced PezkuwiChain node or downloaded a snapshot from [Polkachu]. This example uses a pruned
|
||||
ParityDB snapshot from the 2022-4-19 with the last block being 9939462. For pruned snapshots you need to know the number
|
||||
of the last block (to be improved [here]). Pruned snapshots normally store the last 256 blocks, archive nodes can use
|
||||
any block range.
|
||||
|
||||
In this example we will benchmark just the last 10 blocks:
|
||||
```sh
|
||||
cargo run --profile=production -- benchmark block --from 9939453 --to 9939462 --db paritydb
|
||||
```
|
||||
|
||||
Output:
|
||||
```pre
|
||||
Block 9939453 with 2 tx used 4.57% of its weight ( 26,458,801 of 579,047,053 ns)
|
||||
Block 9939454 with 3 tx used 4.80% of its weight ( 28,335,826 of 590,414,831 ns)
|
||||
Block 9939455 with 2 tx used 4.76% of its weight ( 27,889,567 of 586,484,595 ns)
|
||||
Block 9939456 with 2 tx used 4.65% of its weight ( 27,101,306 of 582,789,723 ns)
|
||||
Block 9939457 with 2 tx used 4.62% of its weight ( 26,908,882 of 582,789,723 ns)
|
||||
Block 9939458 with 2 tx used 4.78% of its weight ( 28,211,440 of 590,179,467 ns)
|
||||
Block 9939459 with 4 tx used 4.78% of its weight ( 27,866,077 of 583,260,451 ns)
|
||||
Block 9939460 with 3 tx used 4.72% of its weight ( 27,845,836 of 590,462,629 ns)
|
||||
Block 9939461 with 2 tx used 4.58% of its weight ( 26,685,119 of 582,789,723 ns)
|
||||
Block 9939462 with 2 tx used 4.60% of its weight ( 26,840,938 of 583,697,101 ns)
|
||||
```
|
||||
|
||||
### Output Interpretation
|
||||
|
||||
<sup>(Only results from reference hardware are relevant)</sup>
|
||||
|
||||
Each block is executed multiple times and the results are averaged. The percent number is the interesting part and
|
||||
indicates how much weight was used as compared to how much was predicted. The closer to 100% this is without exceeding
|
||||
100%, the better. If it exceeds 100%, the block is marked with "**OVER WEIGHT!**" to easier spot them. This is not good
|
||||
since then the benchmarking under-estimated the weight. This would mean that an honest validator would possibly not be
|
||||
able to keep up with importing blocks since users did not pay for enough weight. If that happens the validator could lag
|
||||
behind the chain and get slashed for missing deadlines. It is therefore important to investigate any overweight blocks.
|
||||
|
||||
In this example you can see an unexpected result; only < 5% of the weight was used! The measured blocks can be executed
|
||||
much faster than predicted. This means that the benchmarking process massively over-estimated the execution time. Since
|
||||
they are off by so much, it is an issue [`pezkuwi#5192`].
|
||||
|
||||
The ideal range for these results would be 85-100%.
|
||||
|
||||
## PezkuwiChain # 2
|
||||
|
||||
Let's take a more interesting example where the blocks use more of their predicted weight. Every day when validators pay
|
||||
out rewards, the blocks are nearly full. Using an archive node here is the easiest.
|
||||
|
||||
The PezkuwiChain blocks TODO-TODO for example contain large batch transactions for staking payout.
|
||||
|
||||
```sh
|
||||
cargo run --profile=production -- benchmark block --from TODO --to TODO --db paritydb
|
||||
```
|
||||
|
||||
```pre
|
||||
TODO
|
||||
```
|
||||
|
||||
## Bizinikiwi
|
||||
|
||||
It is also possible to try the procedure in Bizinikiwi, although it's a bit boring.
|
||||
|
||||
First you need to create some blocks with either a local or dev chain. This example will use the standard development
|
||||
spec. Pick a non existing directory where the chain data will be stored, eg `/tmp/dev`.
|
||||
```sh
|
||||
cargo run --profile=production -- --dev -d /tmp/dev
|
||||
```
|
||||
You should see after some seconds that it started to produce blocks:
|
||||
```pre
|
||||
…
|
||||
✨ Imported #1 (0x801d…9189)
|
||||
…
|
||||
```
|
||||
You can now kill the node with `Ctrl+C`. Then measure how long it takes to execute these blocks:
|
||||
```sh
|
||||
cargo run --profile=production -- benchmark block --from 1 --to 1 --dev -d /tmp/dev --pruning archive
|
||||
```
|
||||
This will benchmark the first block. If you killed the node at a later point, you can measure multiple blocks.
|
||||
```pre
|
||||
Block 1 with 1 tx used 72.04% of its weight ( 4,945,664 of 6,864,702 ns)
|
||||
```
|
||||
|
||||
In this example the block used ~72% of its weight. The benchmarking therefore over-estimated the effort to execute the
|
||||
block. Since this block is empty, its not very interesting.
|
||||
|
||||
## Arguments
|
||||
|
||||
- `--from` Number of the first block to measure (inclusive).
|
||||
- `--to` Number of the last block to measure (inclusive).
|
||||
- `--repeat` How often each block should be measured.
|
||||
- [`--db`]
|
||||
- [`--pruning`]
|
||||
|
||||
License: Apache-2.0
|
||||
|
||||
<!-- LINKS -->
|
||||
|
||||
[Polkachu]: https://polkachu.com/snapshots
|
||||
[here]: https://github.com/pezkuwichain/kurdistan-sdk/issues/6
|
||||
[pezkuwi#5192]: https://github.com/pezkuwichain/kurdistan-sdk/issues/154
|
||||
|
||||
[`--db`]: ../shared/README.md#arguments
|
||||
[`--pruning`]: ../shared/README.md#arguments
|
||||
@@ -0,0 +1,184 @@
|
||||
// This file is part of Bizinikiwi.
|
||||
|
||||
// Copyright (C) Parity Technologies (UK) Ltd.
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! Contains the core benchmarking logic.
|
||||
|
||||
use codec::DecodeAll;
|
||||
use pezframe_support::weights::constants::WEIGHT_REF_TIME_PER_NANOS;
|
||||
use pezframe_system::ConsumedWeight;
|
||||
use pezsc_block_builder::BlockBuilderApi;
|
||||
use pezsc_cli::{Error, Result};
|
||||
use pezsc_client_api::{
|
||||
Backend as ClientBackend, BlockBackend, HeaderBackend, StorageProvider, UsageProvider,
|
||||
};
|
||||
use pezsp_api::{ApiExt, Core, ProvideRuntimeApi};
|
||||
use pezsp_blockchain::Error::RuntimeApiError;
|
||||
use pezsp_runtime::{
|
||||
generic::BlockId,
|
||||
traits::{Block as BlockT, Header as HeaderT},
|
||||
DigestItem, OpaqueExtrinsic,
|
||||
};
|
||||
use pezsp_storage::StorageKey;
|
||||
|
||||
use clap::Args;
|
||||
use log::{info, warn};
|
||||
use serde::Serialize;
|
||||
use std::{fmt::Debug, marker::PhantomData, sync::Arc, time::Instant};
|
||||
use thousands::Separable;
|
||||
|
||||
use crate::shared::{StatSelect, Stats};
|
||||
|
||||
/// Log target for printing block weight info.
|
||||
const LOG_TARGET: &'static str = "benchmark::block::weight";
|
||||
|
||||
/// Parameters for modifying the benchmark behaviour.
|
||||
#[derive(Debug, Default, Serialize, Clone, PartialEq, Args)]
|
||||
pub struct BenchmarkParams {
|
||||
/// Number of the first block to consider.
|
||||
#[arg(long)]
|
||||
pub from: u32,
|
||||
|
||||
/// Last block number to consider.
|
||||
#[arg(long)]
|
||||
pub to: u32,
|
||||
|
||||
/// Number of times that the benchmark should be repeated for each block.
|
||||
#[arg(long, default_value_t = 10)]
|
||||
pub repeat: u32,
|
||||
}
|
||||
|
||||
/// Convenience closure for the [`Benchmark::run()`] function.
|
||||
pub struct Benchmark<Block, BA, C> {
|
||||
client: Arc<C>,
|
||||
params: BenchmarkParams,
|
||||
_p: PhantomData<(Block, BA, C)>,
|
||||
}
|
||||
|
||||
/// Helper for nano seconds.
|
||||
type NanoSeconds = u64;
|
||||
|
||||
impl<Block, BA, C> Benchmark<Block, BA, C>
|
||||
where
|
||||
Block: BlockT<Extrinsic = OpaqueExtrinsic>,
|
||||
BA: ClientBackend<Block>,
|
||||
C: ProvideRuntimeApi<Block>
|
||||
+ StorageProvider<Block, BA>
|
||||
+ UsageProvider<Block>
|
||||
+ BlockBackend<Block>
|
||||
+ HeaderBackend<Block>,
|
||||
C::Api: ApiExt<Block> + BlockBuilderApi<Block>,
|
||||
{
|
||||
/// Returns a new [`Self`] from the arguments.
|
||||
pub fn new(client: Arc<C>, params: BenchmarkParams) -> Self {
|
||||
Self { client, params, _p: PhantomData }
|
||||
}
|
||||
|
||||
/// Benchmark the execution speed of historic blocks and log the results.
|
||||
pub fn run(&self) -> Result<()> {
|
||||
if self.params.from == 0 {
|
||||
return Err("Cannot benchmark the genesis block".into());
|
||||
}
|
||||
|
||||
for i in self.params.from..=self.params.to {
|
||||
let block_num = BlockId::Number(i.into());
|
||||
let hash = self.client.expect_block_hash_from_id(&block_num)?;
|
||||
let consumed = self.consumed_weight(hash)?;
|
||||
|
||||
let block = self.client.block(hash)?.ok_or(format!("Block {} not found", block_num))?;
|
||||
let block = self.unsealed(block.block);
|
||||
let took = self.measure_block(&block, *block.header().parent_hash())?;
|
||||
|
||||
self.log_weight(i, block.extrinsics().len(), consumed, took);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Return the average *execution* aka. *import* time of the block.
|
||||
fn measure_block(&self, block: &Block, parent_hash: Block::Hash) -> Result<NanoSeconds> {
|
||||
let mut record = Vec::<NanoSeconds>::default();
|
||||
// Interesting part here:
|
||||
// Execute the block multiple times and collect stats about its execution time.
|
||||
for _ in 0..self.params.repeat {
|
||||
let block = block.clone();
|
||||
let runtime_api = self.client.runtime_api();
|
||||
let start = Instant::now();
|
||||
|
||||
runtime_api
|
||||
.execute_block(parent_hash, block.into())
|
||||
.map_err(|e| Error::Client(RuntimeApiError(e)))?;
|
||||
|
||||
record.push(start.elapsed().as_nanos() as NanoSeconds);
|
||||
}
|
||||
|
||||
let took = Stats::new(&record)?.select(StatSelect::Average);
|
||||
Ok(took)
|
||||
}
|
||||
|
||||
/// Returns the total nanoseconds of a [`pezframe_system::ConsumedWeight`] for a block number.
|
||||
///
|
||||
/// This is the post-dispatch corrected weight and is only available
|
||||
/// after executing the block.
|
||||
fn consumed_weight(&self, block_hash: Block::Hash) -> Result<NanoSeconds> {
|
||||
// Hard-coded key for System::BlockWeight. It could also be passed in as argument
|
||||
// for the benchmark, but I think this should work as well.
|
||||
let hash = array_bytes::hex2bytes(
|
||||
"26aa394eea5630e07c48ae0c9558cef734abf5cb34d6244378cddbf18e849d96",
|
||||
)?;
|
||||
let key = StorageKey(hash);
|
||||
|
||||
let mut raw_weight = &self
|
||||
.client
|
||||
.storage(block_hash, &key)?
|
||||
.ok_or(format!("Could not find System::BlockWeight for block: {}", block_hash))?
|
||||
.0[..];
|
||||
|
||||
let weight = ConsumedWeight::decode_all(&mut raw_weight)?;
|
||||
// Should be divisible, but still use floats in case we ever change that.
|
||||
Ok((weight.total().ref_time() as f64 / WEIGHT_REF_TIME_PER_NANOS as f64).floor()
|
||||
as NanoSeconds)
|
||||
}
|
||||
|
||||
/// Prints the weight info of a block to the console.
|
||||
fn log_weight(&self, num: u32, num_ext: usize, consumed: NanoSeconds, took: NanoSeconds) {
|
||||
// The ratio of weight that the block used vs what it consumed.
|
||||
// This should in general not exceed 100% (minus outliers).
|
||||
let percent = (took as f64 / consumed as f64) * 100.0;
|
||||
|
||||
let msg = format!(
|
||||
"Block {} with {: >5} tx used {: >6.2}% of its weight ({: >14} of {: >14} ns)",
|
||||
num,
|
||||
num_ext,
|
||||
percent,
|
||||
took.separate_with_commas(),
|
||||
consumed.separate_with_commas()
|
||||
);
|
||||
|
||||
if took <= consumed {
|
||||
info!(target: LOG_TARGET, "{}", msg);
|
||||
} else {
|
||||
warn!(target: LOG_TARGET, "{} - OVER WEIGHT!", msg);
|
||||
}
|
||||
}
|
||||
|
||||
/// Removes the consensus seal from the block.
|
||||
fn unsealed(&self, block: Block) -> Block {
|
||||
let (mut header, exts) = block.deconstruct();
|
||||
header.digest_mut().logs.retain(|item| !matches!(item, DigestItem::Seal(_, _)));
|
||||
Block::new(header, exts)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
// This file is part of Bizinikiwi.
|
||||
|
||||
// Copyright (C) Parity Technologies (UK) Ltd.
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! Contains the [`BlockCmd`] as entry point for the CLI to execute
|
||||
//! the *block* benchmark.
|
||||
|
||||
use pezsc_block_builder::BlockBuilderApi;
|
||||
use pezsc_cli::{CliConfiguration, ImportParams, Result, SharedParams};
|
||||
use pezsc_client_api::{Backend as ClientBackend, BlockBackend, StorageProvider, UsageProvider};
|
||||
use pezsp_api::{ApiExt, ProvideRuntimeApi};
|
||||
use pezsp_blockchain::HeaderBackend;
|
||||
use pezsp_runtime::{traits::Block as BlockT, OpaqueExtrinsic};
|
||||
|
||||
use clap::Parser;
|
||||
use std::{fmt::Debug, sync::Arc};
|
||||
|
||||
use super::bench::{Benchmark, BenchmarkParams};
|
||||
|
||||
/// Benchmark the execution time of historic blocks.
|
||||
///
|
||||
/// This can be used to verify that blocks do not use more weight than they consumed
|
||||
/// in their `WeightInfo`. Example:
|
||||
///
|
||||
/// Let's say you are on a Bizinikiwi chain and want to verify that the first 3 blocks
|
||||
/// did not use more weight than declared which would otherwise be an issue.
|
||||
/// To test this with a dev node, first create one with a temp directory:
|
||||
///
|
||||
/// $ bizinikiwi --dev -d /tmp/my-dev --wasm-execution compiled
|
||||
///
|
||||
/// And wait some time to let it produce 3 blocks. Then benchmark them with:
|
||||
///
|
||||
/// $ bizinikiwi benchmark-block --from 1 --to 3 --dev -d /tmp/my-dev
|
||||
/// --wasm-execution compiled --pruning archive
|
||||
///
|
||||
/// The output will be similar to this:
|
||||
///
|
||||
/// Block 1 with 1 tx used 77.34% of its weight ( 5,308,964 of 6,864,645 ns)
|
||||
/// Block 2 with 1 tx used 77.99% of its weight ( 5,353,992 of 6,864,645 ns)
|
||||
/// Block 3 with 1 tx used 75.91% of its weight ( 5,305,938 of 6,989,645 ns)
|
||||
///
|
||||
/// The percent number is important and indicates how much weight
|
||||
/// was used as compared to the consumed weight.
|
||||
/// This number should be below 100% for reference hardware.
|
||||
#[derive(Debug, Parser)]
|
||||
pub struct BlockCmd {
|
||||
#[allow(missing_docs)]
|
||||
#[clap(flatten)]
|
||||
pub shared_params: SharedParams,
|
||||
|
||||
#[allow(missing_docs)]
|
||||
#[clap(flatten)]
|
||||
pub import_params: ImportParams,
|
||||
|
||||
#[allow(missing_docs)]
|
||||
#[clap(flatten)]
|
||||
pub params: BenchmarkParams,
|
||||
|
||||
/// Enable the Trie cache.
|
||||
///
|
||||
/// This should only be used for performance analysis and not for final results.
|
||||
#[arg(long)]
|
||||
pub enable_trie_cache: bool,
|
||||
}
|
||||
|
||||
impl BlockCmd {
|
||||
/// Benchmark the execution time of historic blocks and compare it to their consumed weight.
|
||||
///
|
||||
/// Output will be printed to console.
|
||||
pub fn run<Block, BA, C>(&self, client: Arc<C>) -> Result<()>
|
||||
where
|
||||
Block: BlockT<Extrinsic = OpaqueExtrinsic>,
|
||||
BA: ClientBackend<Block>,
|
||||
C: BlockBackend<Block>
|
||||
+ ProvideRuntimeApi<Block>
|
||||
+ StorageProvider<Block, BA>
|
||||
+ UsageProvider<Block>
|
||||
+ HeaderBackend<Block>,
|
||||
C::Api: ApiExt<Block> + BlockBuilderApi<Block>,
|
||||
{
|
||||
// Put everything in the benchmark type to have the generic types handy.
|
||||
Benchmark::new(client, self.params.clone()).run()
|
||||
}
|
||||
}
|
||||
|
||||
// Boilerplate
|
||||
impl CliConfiguration for BlockCmd {
|
||||
fn shared_params(&self) -> &SharedParams {
|
||||
&self.shared_params
|
||||
}
|
||||
|
||||
fn import_params(&self) -> Option<&ImportParams> {
|
||||
Some(&self.import_params)
|
||||
}
|
||||
|
||||
fn trie_cache_maximum_size(&self) -> Result<Option<usize>> {
|
||||
if self.enable_trie_cache {
|
||||
Ok(self.import_params().map(|x| x.trie_cache_maximum_size()).unwrap_or_default())
|
||||
} else {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
// This file is part of Bizinikiwi.
|
||||
|
||||
// Copyright (C) Parity Technologies (UK) Ltd.
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
//! Crate to benchmark the execution time of historic blocks
|
||||
//! and compare it to their consumed weight.
|
||||
|
||||
mod bench;
|
||||
mod cmd;
|
||||
|
||||
pub use cmd::BlockCmd;
|
||||
Reference in New Issue
Block a user