* Experiments with common equivocation trait
* Improved equivocation trait
* Fix grandpa equivocation implementation
* Remove some cruft
* Remove some more cruft
* More generic naming
* Simplification of offences manipilation
* More refactory
* Some prograss with the encapsulation of offence report system
* Finally unit type works as a universal null report system
* Align substrate node code
* Further simplification
* Fix test utils
* Remove not required associated type
* Fix benches
* Rollback to prev field name
* Box big params
* Fix typo
* Remove new tag computation
* Remove default implementations
* Better docs
* Return 'Result' instead of bool
* Change offence report system return types
* Some renaming and documentation
* Improve documentation
* More abstract offence report system
* Rename 'consume_evidence' to 'process_evidence'
* Further docs refinements
* Doc for dummy offence report
* Fix rustdoc
* Fix after master merge
* Apply code review suggestions
* Improve docs
* Move import queue out of `sc-network`
Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.
This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.
* Move stuff to SyncingEngine
* Move `ChainSync` instanation to `SyncingEngine`
Some of the tests have to be rewritten
* Move peer hashmap to `SyncingEngine`
* Let `SyncingEngine` to implement `ChainSyncInterface`
* Introduce `SyncStatusProvider`
* Move `sync_peer_(connected|disconnected)` to `SyncingEngine`
* Implement `SyncEventStream`
Remove `SyncConnected`/`SyncDisconnected` events from
`NetworkEvenStream` and provide those events through
`ChainSyncInterface` instead.
Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
`SyncEventStream` object which they listen to for incoming sync peer
events.
* Introduce `ChainSyncInterface`
This interface provides a set of miscellaneous functions that other
subsystems can use to query, for example, the syncing status.
* Move event stream polling to `SyncingEngine`
Subscribe to `NetworkStreamEvent` and poll the incoming notifications
and substream events from `SyncingEngine`.
The code needs refactoring.
* Make `SyncingEngine` into an asynchronous runner
This commits removes the last hard dependency of syncing from
`sc-network` meaning the protocol now lives completely outside of
`sc-network`, ignoring the hardcoded peerset entry which will be
addressed in the future.
Code needs a lot of refactoring.
* Fix warnings
* Code refactoring
* Use `SyncingService` for BEEFY
* Use `SyncingService` for GRANDPA
* Remove call delegation from `NetworkService`
* Remove `ChainSyncService`
* Remove `ChainSync` service tests
They were written for the sole purpose of verifying that `NetworWorker`
continues to function while the calls are being dispatched to
`ChainSync`.
* Refactor code
* Refactor code
* Update client/finality-grandpa/src/communication/tests.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Fix warnings
* Apply review comments
* Fix docs
* Fix test
* cargo-fmt
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Update client/network/sync/src/engine.rs
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Add missing docs
* Refactor code
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* `BlockId` removal: `BlockBuilderProvider::new_block_at`
It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`
* fmt
* fix
* more fixes
* BlockId removal: refactor of runtime API
It changes the arguments of:
- `ApiExt` methods: `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`
It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* BlockId removal: refactor of runtime API - tests
- tests adjusted to new runtime API,
- some tests migrated from block number to block hash
* benchmarking-cli: BlockId(0) migrated to info().genesis_hash
`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.
* trivial: api.call(BlockId) -> api.call(Hash)
- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,
* CallApiAtParams::at changed to Block::Hash
* missed doc updated
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* ".git/.scripts/commands/fmt/fmt.sh"
* BlockId removal: Benchmark::consumed_weight
Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.
* at_hash renamed
* wrong merge fixed
* beefy worker: merged with master
* beefy: tests: missing block problem fixed
* Apply review suggestion
* fix
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
* BlockId removal: Client::runtime_version_at
It changes the arguments of `Client::runtime_version_at` from: `BlockId<Block>` to: `Block::Hash`
* Apply suggestions from code review
Co-authored-by: Anton <anton.kalyaev@gmail.com>
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: parity-processbot <>
* initial impl
* add template test
* linear fit proof size
* always record proof when tracking storage
* calculate worst case pov
* remove duplicate worst case
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* more comment output
* add cli for worst case map size
* update name
* clap does not support underscores
* rename
* expose worst case map values
* improve some comments
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* update template
* cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* fix fmt
* more fmt
* more fmt
* Dont panic when there is no proof
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix test features
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Whitelist :extrinsic_index
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Use whitelist when recording proof
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add logs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add PoV testing pallet
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Deploy PoV testing pallet
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Storage benches reside in the PoV pallet
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Linear regress PoV per component
Splits the PoV calculation into "measured" and "estimated".
The measured part is reported by the Proof recorder and linear
regressed over all components at once.
The estimated part is calculated as worst-case by using the max
PoV size per storage access and calculating one linear regress per
component. This gives each component a (possibly) independent PoV.
For now the measured size will always be lower than the PoV on
Polkadot since it is measured on an empty snapshot. The measured
part is therefor only used as diagnostic for debugging.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Put PoV into the weight templates
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* fmt
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Extra alanysis choise for PoV
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add+Fix tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Make benches faster
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Cleanup
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Use same template comments
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* ".git/.scripts/bench-bot.sh" pallet dev pallet_balances
* ".git/.scripts/bench-bot.sh" pallet dev pallet_democracy
* Update referenda mock BlockWeights
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Take measured value size into account
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* clippy
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* ".git/.scripts/bench-bot.sh" pallet dev pallet_scheduler
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* proof_size: None
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* ugly, but works
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* wup
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* WIP
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add pov_mode attribute to the benchmarks! macro
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Use pov_mode attribute in PoV benchmarking
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Scheduler, Whitelist: Add pov_mode attr
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update PoV weights
* Add CLI arg: default-pov-mode
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* fmt
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* fix
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Revert "Update PoV weights"
This reverts commit 2f3ac2387396470b118122a6ff8fa4ee12216f4b.
* Revert "WIP"
This reverts commit c34b538cd2bc45da4544e887180184e30957904a.
* Revert first approach
This reverts commit range 8ddaa2fffe5930f225a30bee314d0b7c94c344dd^..4c84f8748e5395852a9e0e25b0404953fee1a59e
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Clippy
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add extra benchmarks
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_alliance
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_whitelist
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_scheduler
* fmt
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Clippy
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Clippy 🤦
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add reference benchmarks
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix doc comments
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Undo logging
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Add 'Ignored' pov_mode
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Allow multiple attributes per benchmark
Turns out that the current benchmarking syntax does not support
multiple attributes per bench 🤦. Changing it to support that
since otherwise the `pov_mode` would conflict with the others.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Validate pov_mode syntax
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Ignore PoV for all contract benchmarks
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Test
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* test
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Bump macro recursion limit
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* fmt
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update contract weights
They dont have a PoV component anymore.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* fix test ffs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* pov_mode is unsupported in V2 syntax
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix pallet ui tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* update pallet ui
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix pallet ui tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Update weights
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Your Name <you@example.com>
* reduce exec time of fast-unstake benchmarks
* fix test
* fmt
* fix patch the tests
* fix patch the tests
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* add batch size as well
* update some benches to be better
* fix one last test
* fix
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* reduce time even more
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* fix tests
* nit
* remove
* improve the weight calc further
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* fix benchmarks
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* update
* fix
* fix
* fmt
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* lots of changes again...
* smaller input
* update
* fmt
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* cleanup
* small simplification
* fmt
* reduce exec time a bit
* fix
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* test
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* increase again
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
* review comments
* fmt
* fix
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake
Co-authored-by: command-bot <>
* integrity test for MaxCodeLen and CallStack::len()
* integrity test for MaxDebugBufferLen
* addressed review comments
* fix append_debug_buffer()
* ci fix
* updated code_len_limit formula after further discussion
* enlarged mem safe margin after discussion
* +doc to Config trait associated types
* Apply suggestions from code review
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
* more lil fixes from code review feedback
* lowered max call depth to satisfy mem limits
* fix node runtime pallet params to satisfy integrity check
* fix max call depth value calc
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
The macros in frame-benchmarking relied on having all the macros imported, which isn't a behavior
for a proper macro :D This pr fixes this by making all internal macro usages absolute.
* Make CLI state pruning optional again
The state pruning setting is stored in the database when it is created. In later runs it is fine to
drop the `--state-pruning` CLI argument as the setting is stored in the database. The state db will
only return an error if the stored state pruning doesn't match the state pruning given via CLI.
Recently we improved the state pruning CLI handling and accidentally made the state pruning value
always present (as we set some default value for the clap). If we could find out if a user has
passed a value or the default value was taken, we could keep the default value in the CLI interface,
but clap isn't supporting this right now. So, we need to go back and make `state_pruning` an
optional with the default written into the docs.
It also adds a test to ensure that we don't break this behavior again.
* More docs
* staking admin can set min commission
* ".git/.scripts/bench-bot.sh" pallet dev pallet_staking
* fmt
* fix for pr comments
Co-authored-by: command-bot <>
Gavin Wood