Commit Graph

190 Commits

Author SHA1 Message Date
Dcompoze 002d9260f9 Fix spelling mistakes across the whole repository (#3808)
**Update:** Pushed additional changes based on the review comments.

**This pull request fixes various spelling mistakes in this
repository.**

Most of the changes are contained in the first **3** commits:

- `Fix spelling mistakes in comments and docs`

- `Fix spelling mistakes in test names`

- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`

Other source code spelling mistakes are separated into individual
commits for easier reviewing:

- `Fix the spelling of 'authority'`

- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`

- `Fix the spelling of 'prev_enqueud_messages'`

- `Fix the spelling of 'endpoint'`

- `Fix the spelling of 'children'`

- `Fix the spelling of 'PenpalSiblingSovereignAccount'`

- `Fix the spelling of 'PenpalSudoAccount'`

- `Fix the spelling of 'insufficient'`

- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`

- `Fix the spelling of 'subtracted'`

- `Fix the spelling of 'CandidatePendingAvailability'`

- `Fix the spelling of 'exclusive'`

- `Fix the spelling of 'until'`

- `Fix the spelling of 'discriminator'`

- `Fix the spelling of 'nonexistent'`

- `Fix the spelling of 'subsystem'`

- `Fix the spelling of 'indices'`

- `Fix the spelling of 'committed'`

- `Fix the spelling of 'topology'`

- `Fix the spelling of 'response'`

- `Fix the spelling of 'beneficiary'`

- `Fix the spelling of 'formatted'`

- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`

- `Fix the spelling of 'succeeded'`

- `Fix the spelling of 'reopened'`

- `Fix the spelling of 'proposer'`

- `Fix the spelling of 'InstantiationNonce'`

- `Fix the spelling of 'depositor'`

- `Fix the spelling of 'expiration'`

- `Fix the spelling of 'phantom'`

- `Fix the spelling of 'AggregatedKeyValue'`

- `Fix the spelling of 'randomness'`

- `Fix the spelling of 'defendant'`

- `Fix the spelling of 'AquaticMammal'`

- `Fix the spelling of 'transactions'`

- `Fix the spelling of 'PassingTracingSubscriber'`

- `Fix the spelling of 'TxSignaturePayload'`

- `Fix the spelling of 'versioning'`

- `Fix the spelling of 'descendant'`

- `Fix the spelling of 'overridden'`

- `Fix the spelling of 'network'`

Let me know if this structure is adequate.

**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.

~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~

~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~

~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~

**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.

~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~
2024-03-26 13:57:57 +00:00
Davide Galassi 4c10fd2a41 Move cryptographic hashing procedures to crypto folder. (#2306)
Step towards https://github.com/paritytech/polkadot-sdk/issues/1975

As reported
https://github.com/paritytech/polkadot-sdk/issues/1975#issuecomment-1774534225
I'd like to encapsulate crypto related stuff in a dedicated folder.

Currently all cryptographic primitive wrappers are all sparsed in
`substrate/core` which contains "misc core" stuff.

To simplify the process, as the first step with this PR I propose to
move the cryptographic hashing there.

The `substrate/crypto` folder was already created to contains `ec-utils`
crate.

Notes:
- rename `sp-core-hashing` to `sp-crypto-hashing`
- rename `sp-core-hashing-proc-macro` to `sp-crypto-hashing-proc-macro`
- As the crates name is changed I took the freedom to restart fresh from
version 0.1.0 for both crates

---------

Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
2024-01-22 23:36:14 +00:00
Alexander Samusev cd63276d31 [ci] Update rust to 1.74 (#2545)
cc https://github.com/paritytech/ci_cd/issues/900

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
2023-12-07 09:57:43 +01:00
André Silva 1f023deab8 sp-core: Rename VrfOutput to VrfPreOutput (#2534)
This will make more sense after
https://github.com/paritytech/polkadot-sdk/pull/2524 since the
schnorrkel type for VRF outputs is also renamed in the latest version.
Can be reviewed independently though.

Can be merged after https://github.com/paritytech/polkadot-sdk/pull/1577
so that there is less pain for @davxy.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-12-06 02:19:56 +02:00
Bastian Köcher ca5f10567a sc-block-builder: Remove BlockBuilderProvider (#2099)
The `BlockBuilderProvider` was a trait that was defined in
`sc-block-builder`. The trait was implemented for `Client`. This
basically meant that you needed to import `sc-block-builder` any way to
have access to the block builder. So, this trait was not providing any
real value. This pull request is removing the said trait. Instead of the
trait it introduces a builder for creating a `BlockBuilder`. The builder
currently has the quite fabulous name `BlockBuilderBuilder` (I'm open to
any better name 😅). The rest of the pull request is about
replacing the old trait with the new builder.

# Downstream code changes

If you used `new_block` or `new_block_at` before you now need to switch
it over to the new `BlockBuilderBuilder` pattern:

```rust
// `new` requires a type that implements `CallApiAt`. 
let mut block_builder = BlockBuilderBuilder::new(client)
                // Then you need to specify the hash of the parent block the block will be build on top of
		.on_parent_block(at)
                // The block builder also needs the block number of the parent block. 
                // Here it is fetched from the given `client` using the `HeaderBackend`
                // However, there also exists `with_parent_block_number` for directly passing the number
		.fetch_parent_block_number(client)
		.unwrap()
                // Enable proof recording if required. This call is optional.
		.enable_proof_recording()
                // Pass the digests. This call is optional.
                .with_inherent_digests(digests)
		.build()
		.expect("Creates new block builder");
```

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>
2023-11-03 19:06:31 +01:00
Davide Galassi c7dbfc21b6 Babe epoch newtype (#1596)
Removal of verbatim duplication of BABE's `Epoch` struct in the client.

I think is better to have one single definition and wrap the primitive
`Epoch` in a newtype (required because we need to implement the `Epoch`
trait).
2023-09-17 22:06:19 +02:00
Nazar Mokrynskyi b728724318 Make claim_slot take &mut self (#1225)
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-08-29 11:37:02 +02:00
Bastian Köcher ecf8035da6 Set StateBackend::Transaction to PrefixedMemoryDB (#14612)
* Yep

* Try to get it working everywhere

* Make `from_raw_storage` start with an empty db

* More fixes!

* Make everything compile

* Fix `child_storage_root`

* Fix after merge

* Cleanups

* Update primitives/state-machine/src/overlayed_changes/mod.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Review comments

* Fix issues

* Silence warning

* FMT

* Clippy

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-08-17 10:49:38 +00:00
Bastian Köcher 5eb816d7a6 Removal of execution strategies (#14387)
* Start

* More work!

* Moar

* More changes

* More fixes

* More worrk

* More fixes

* More fixes to make it compile

* Adds `NoOffchainStorage`

* Pass the extensions

* Small basti making small progress

* Fix merge errors and remove `ExecutionContext`

* Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`

Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
`ExecutionExtension` which provides the default extensions.

* Fix compilation

* Register the global extensions inside runtime api instance

* Fixes

* Fix `generate_initial_session_keys` by passing the keystore extension

* Fix the grandpa tests

* Fix more tests

* Fix more tests

* Don't set any heap pages if there isn't an override

* Fix small fallout

* FMT

* Fix tests

* More tests

* Offchain worker custom extensions

* More fixes

* Make offchain tx pool creation reusable

Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
registered in the runtime externalities context. This factory will be required for a later pr to
make the creation of offchain transaction pools easier.

* Fixes

* Fixes

* Set offchain transaction pool in BABE before using it in the runtime

* Add the `offchain_tx_pool` to Grandpa as well

* Fix the nodes

* Print some error when using the old warnings

* Fix merge issues

* Fix compilation

* Rename `babe_link`

* Rename to `offchain_tx_pool_factory`

* Cleanup

* FMT

* Fix benchmark name

* Fix `try-runtime`

* Remove `--execution` CLI args

* Make clippy happy

* Forward bls functions

* Fix docs

* Update UI tests

* Update client/api/src/execution_extensions.rs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/cli/src/params/import_params.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Pass the offchain storage to the MMR RPC

* Update client/api/src/execution_extensions.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Review comments

* Fixes

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-07-11 14:21:38 +00:00
Bastian Köcher 83caca85b6 frame-benchmarking-cli: Remove native dispatch requirement (#14474)
* frame-benchmarking-cli: Remove native dispatch requirement

No need for this, we can just use the `WasmExecutor` directly.

* Fixes

* Pass benchmarking host functions

* Ensure we can pass custom host functions
2023-06-29 17:56:25 +02:00
Michal Kucharczyk 6a295e7c28 substrate-test-runtime migrated to "pure" frame runtime (#13737)
* substrate-test-runtime migrated to pure-frame based

* test block builder: helpers added

* simple renaming

* basic_authorship test adjusted

* block_building storage_proof test adjusted

* babe: tests: should_panic expected added

* babe: tests adjusted

ConsensusLog::NextEpochData is now added by pallet_babe as
pallet_babe::SameAuthoritiesForever trigger is used in runtime config.

* beefy: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digests stored during block execution are
checked against header digests:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(MmrRoot / AuthoritiesChange) is used.

* grandpa: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digest logs stored during block execution are
checked against header digest logs:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(ScheduledChange / ForcedChange and DigestItem::Other) is used.

* network:bitswap: test adjusted

The size of unchecked extrinsic was increased. The pattern used in test will
be placed at the end of scale-encoded buffer.

* runtime apis versions adjusted

* storage keys used in runtime adjusted

* wasm vs native tests removed

* rpc tests: adjusted

Transfer transaction processing was slightly improved, test was
adjusted.

* tests: sizes adjusted

Runtime extrinsic size was increased. Size of data read during block
execution was also increased due to usage of new pallets in runtime.

Sizes were adjusted in tests.

* cargo.lock update

cargo update -p substrate-test-runtime -p substrate-test-runtime-client

* warnings fixed

* builders cleanup: includes / std

* extrinsic validation cleanup

* txpool: benches performance fixed

* fmt

* spelling

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Apply code review suggestions

* Apply code review suggestions

* get rid of 1063 const

* renaming: UncheckedExtrinsic -> Extrinsic

* test-utils-runtime: further step to pure-frame

* basic-authorship: tests OK

* CheckSubstrateCall added + tests fixes

* test::Transfer call removed

* priority / propagate / no sudo+root-testing

* fixing warnings + format

* cleanup: build2/nonce + format

* final tests fixes

all tests are passing

* logs/comments removal

* should_not_accept_old_signatures test removed

* make txpool benches work again

* Cargo.lock reset

* format

* sudo hack removed

* txpool benches fix+cleanup

* .gitignore reverted

* rebase fixing + unsigned cleanup

* Cargo.toml/Cargo.lock cleanup

* force-debug feature removed

* mmr tests fixed

* make cargo-clippy happy

* network sync test uses unsigned extrinsic

* cleanup

* ".git/.scripts/commands/fmt/fmt.sh"

* push_storage_change signed call remove

* GenesisConfig cleanup

* fix

* fix

* GenesisConfig simplified

* storage_keys_works: reworked

* storage_keys_works: expected keys in vec

* storage keys list moved to substrate-test-runtime

* substrate-test: some sanity tests + GenesisConfigBuilder rework

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

* Review suggestions

* fix

* fix

* beefy: generate_blocks_and_sync block_num sync with actaul value

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* cargo update -p sc-rpc -p sc-transaction-pool

* Review suggestions

* fix

* doc added

* slot_duration adjusted for Babe::slot_duration

* small doc fixes

* array_bytes::hex used instead of hex

* tiny -> medium name fix

* Apply suggestions from code review

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* TransferData::try_from_unchecked_extrinsic -> try_from

* Update Cargo.lock

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-05-04 16:20:22 +00:00
Davide Galassi 3a90728de0 Refinements to VRF types (#14036)
* Allow extra signing data

* Fix tests after renaming

* Rename VrfSecret/VrfVerifier to VrfSecret/VrfPublic

* Further encrapsulation of 'transcript' type to the sr25519 implementation

* Keystore sr25519 pre-output

* Leave additional custom input field hidden in the associated VrfInput type

* Fix test

* More ergonomic output_bytes

* Trigger pipeline

* Define a separated type for vrf signature data

* Fix docs

* Fix doc

* Remove annotation

* Directly use dleq_proove and dleq_verify in sr25519

* Trigger CI

* Remove cruft before merge
2023-05-04 15:41:59 +02:00
Davide Galassi bb394e08ac VRF refactory (#13889)
* First iteration to encapsulate schnorrkel and merlin usage

* Remove schnorkel direct dependency from BABE pallet

* Remove schnorrkel direct dependency from BABE client

* Trivial renaming for VrfTranscript data and value

* Better errors

* Expose a function to get a schnorrkel friendly transcript

* Keep the vrf signature stuff together (preventing some clones around)

* Fix tests

* Remove vrf agnostic transcript and define it as an associated type for VrfSigner and VrfVerifier

* Fix babe pallet mock

* Inner types are required to be public for polkadot

* Update client/consensus/babe/src/verification.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Nit

* Remove Deref implementations

* make_bytes as a method

* Trigger CI

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-04-19 09:11:47 +00:00
André Silva e8e22b83b8 babe: replace usage of SharedEpochChanges with internal RPC (#13883)
* babe: replace usage of SharedEpochChanges with internal RPC

* babe-rpc: fix tests

* babe: use SinkExt::send instead of Sender::try_send

SinkExt::send provides backpressure in case the channel is full

* Update client/consensus/babe/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* babe: fix spawn

* babe: send handles backpressure

* babe: use testing::TaskExecutor

* babe-rpc: better error handling

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-04-18 09:38:04 +00:00
Sam Johnson b83bf4784e Globally upgrade to syn 2.x and latest quote and proc_macro2 1x versions (#13846)
* globally upgrade quote to latest 1.0.x (1.0.26)

* globally upgrade syn to final 1.0.x version (1.0.109)

* globally upgrade proc-macro2 to 1.0.56

* upgrade to syn v2.0.13 and fix everything except NestedMeta

* fix parse nested metadata code in decl_runtime_apis.rs

* Port more stuff to syn 2.0

* Make the rest compile

* Ignore error

* update to syn 2.0.14

---------

Co-authored-by: Bastian Köcher <info@kchr.de>
2023-04-12 18:42:22 +00:00
Davide Galassi 40e1704e1c Keystore overhaul (final) (#13683)
* Introduce keystore specialized sign methods

* Get rid of 'AppKey::UntypedGeneric' associated type.

Untyped generics are accessible using associated types 'Generic' associated type.
I.e. <T as AppKey>::Public::Generic

* Get rid of 'CryptoTypePublicPair'

* Trivial fix

* Small refactory of local keystore implementations

* Remove 'crypto_id' method from 'Public'

* Trivial rename of 'AppKey' to 'AppCrypto'

* Remove unused import

* Improve docs

* Better signature related errors for authority-discovery

* Apply review suggestion

* Apply review suggestions

Co-authored-by: Koute <koute@users.noreply.github.com>

* Authority discoverty signing error revisited

* Signing error revisited for babe and aura as well

* Further cleanup

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-03-24 13:46:02 +00:00
Davide Galassi 480396fe06 Keystore overhaul (iter 2) (#13634)
* Remove bloat about remote keystore

* Update docs and remove unused 'KeystoreRef' trait

* Use wherever possible, MemoryKeystore for testing

* Remove unrequired fully qualified method syntax for Keystore
2023-03-20 18:21:26 +00:00
Davide Galassi f110941b7f Keystore overhaul (#13615)
* Remove 'supported_keys' 'sign_with_any' and 'sign_with_all' from keystore trait

* Remove the aync keystore

* Renaming:
- SyncCryptoStore -> Keystore
- SyncCryptoStorePtr -> KeystorePtr
- KeyStore -> MemoryKeystore

* Fix authority discovery worker and tests

* Rename 'insert_unknown' to 'insert'

* Remove leftover
2023-03-17 11:24:14 +00:00
André Silva 13b0f24abd consensus: remove caching functionality from block import pipeline (#13551)
* consensus: remove caching functionality from block import pipeline

* client: update docs on Verifier::verify

* node: fix block production benchmark
2023-03-07 11:19:19 +00:00
Aaro Altonen 1a7f5be07f Extract syncing protocol from sc-network (#12828)
* Move import queue out of `sc-network`

Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.

This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.

* Move stuff to SyncingEngine

* Move `ChainSync` instanation to `SyncingEngine`

Some of the tests have to be rewritten

* Move peer hashmap to `SyncingEngine`

* Let `SyncingEngine` to implement `ChainSyncInterface`

* Introduce `SyncStatusProvider`

* Move `sync_peer_(connected|disconnected)` to `SyncingEngine`

* Implement `SyncEventStream`

Remove `SyncConnected`/`SyncDisconnected` events from
`NetworkEvenStream` and provide those events through
`ChainSyncInterface` instead.

Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
`SyncEventStream` object which they listen to for incoming sync peer
events.

* Introduce `ChainSyncInterface`

This interface provides a set of miscellaneous functions that other
subsystems can use to query, for example, the syncing status.

* Move event stream polling to `SyncingEngine`

Subscribe to `NetworkStreamEvent` and poll the incoming notifications
and substream events from `SyncingEngine`.

The code needs refactoring.

* Make `SyncingEngine` into an asynchronous runner

This commits removes the last hard dependency of syncing from
`sc-network` meaning the protocol now lives completely outside of
`sc-network`, ignoring the hardcoded peerset entry which will be
addressed in the future.

Code needs a lot of refactoring.

* Fix warnings

* Code refactoring

* Use `SyncingService` for BEEFY

* Use `SyncingService` for GRANDPA

* Remove call delegation from `NetworkService`

* Remove `ChainSyncService`

* Remove `ChainSync` service tests

They were written for the sole purpose of verifying that `NetworWorker`
continues to function while the calls are being dispatched to
`ChainSync`.

* Refactor code

* Refactor code

* Update client/finality-grandpa/src/communication/tests.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* Fix warnings

* Apply review comments

* Fix docs

* Fix test

* cargo-fmt

* Update client/network/sync/src/engine.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* Update client/network/sync/src/engine.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* Add missing docs

* Refactor code

---------

Co-authored-by: Anton <anton.kalyaev@gmail.com>
2023-03-06 16:33:38 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Michal Kucharczyk 5ef88dd398 BlockId removal: BlockBuilderProvider::new_block_at (#13401)
* `BlockId` removal: `BlockBuilderProvider::new_block_at`

It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`

* fmt

* fix

* more fixes
2023-02-21 18:36:00 +00:00
Michal Kucharczyk 7a10154188 BlockId removal: runtime-api refactor (#13255)
* BlockId removal: refactor of runtime API

It changes the arguments of:
- `ApiExt` methods:  `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`

It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* BlockId removal: refactor of runtime API - tests

- tests adjusted to new runtime API,
- some tests migrated from block number to block hash

* benchmarking-cli: BlockId(0) migrated to info().genesis_hash

`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.

* trivial: api.call(BlockId) -> api.call(Hash)

- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,

* CallApiAtParams::at changed to Block::Hash

* missed doc updated

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

* BlockId removal: Benchmark::consumed_weight

Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.

* at_hash renamed

* wrong merge fixed

* beefy worker: merged with master

* beefy: tests: missing block problem fixed

* Apply review suggestion

* fix

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-02-20 22:47:21 +00:00
Adrian Catangiu c21f292a02 BEEFY: implement equivocations detection, reporting and slashing (#13121)
* client/beefy: simplify self_vote logic

* client/beefy: migrate to new state version

* client/beefy: detect equivocated votes

* fix typos

* sp-beefy: add equivocation primitives

* client/beefy: refactor vote processing

* fix version migration for new rounds struct

* client/beefy: track equivocations and create proofs

* client/beefy: adjust tests for new voting logic

* sp-beefy: fix commitment ordering and equality

* client/beefy: simplify handle_vote() a bit

* client/beefy: add simple equivocation test

* client/beefy: submit equivocation proof - WIP

* frame/beefy: add equivocation report runtime api - part 1

* frame/beefy: report equivocation logic - part 2

* frame/beefy: add pluggable Equivocation handler - part 3

* frame/beefy: impl ValidateUnsigned for equivocations reporting

* client/beefy: submit report equivocation unsigned extrinsic

* primitives/beefy: fix tests

* frame/beefy: add default weights

* frame/beefy: fix tests

* client/beefy: fix tests

* frame/beefy-mmr: fix tests

* frame/beefy: cross-check session index with equivocation report

* sp-beefy: make test Keyring useable in pallet

* frame/beefy: add basic equivocation test

* frame/beefy: test verify equivocation results in slashing

* frame/beefy: test report_equivocation_old_set

* frame/beefy: add more equivocation tests

* sp-beefy: fix docs

* beefy: simplify equivocations and fix tests

* client/beefy: address review comments

* frame/beefy: add ValidateUnsigned to test/mock runtime

* client/beefy: fixes after merge master

* fix missed merge damage

* client/beefy: add test for reporting equivocations

Also validated there's no unexpected equivocations reported in the
other tests.

Signed-off-by: acatangiu <adrian@parity.io>

* sp-beefy: move test utils to their own file

* client/beefy: add negative test for equivocation reports

* sp-beefy: move back MmrRootProvider - used in polkadot-service

* impl review suggestions

* client/beefy: add equivocation metrics

---------

Signed-off-by: acatangiu <adrian@parity.io>
Co-authored-by: parity-processbot <>
2023-02-17 09:45:00 +00:00
Alexandru Vasile 4b2cc67fc6 Metadata V15: Derive TypeInfo for describing runtime types (#13272)
* scale_info: Derive TypeInfo for types present in runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cargo: Update Cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-02-02 19:14:40 +02:00
Dan Henton ae8ef86ec0 Use non-binary pronouns in comments. (#13209)
* use non binary pronouns in comments

* cargo fmt

* fix the use of "it" with "they" when dealing about an opperations identity
2023-01-27 12:31:17 +01:00
Davide Galassi e66c53089d Babe: bad epoch index with skipped epochs and warp sync (#13135)
* Detect and correct epoch-index for skipped epochs

* Code refactory

* Epoch index should be also be fixed for secondary claims with VRF

* Fix typo

* Make clippy happy

* Fix typo

* Trigger pipeline
2023-01-25 12:57:59 +01:00
Bastian Köcher a8d7cda774 Fix flaky BABE test (#13199)
The `authoring_blocks` test of BABE was calculating the slot based on the timestamp it sometimes
failed in CI. The problem is that we combine all the notifications and authoring futures in one big
future. This one big future may first polls one authoring future to build a block. Then it polls all
notification futures again to import the block. Then some other authoring future is polled and
builds on the imported block using the same slot and making the import fail. The solution is that we
just artificially increase the slot to make the test work.
2023-01-21 19:00:24 +00:00
Davide Galassi 7aa88e6a59 Stop keeping track of epoch changes for the sync gap (#13127)
* Stop keeping track of epoch changes data within the sync gap

* Fix docs

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Fix typo

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-01-12 20:36:31 +01:00
André Silva 017cf70378 babe: allow skipping over empty epochs (#11727)
* babe: allow skipping epochs in pallet

* babe: detect and skip epochs on client

* babe: cleaner epoch util functions

* babe: add test for runtime handling of skipped epochs

* babe: simpler implementation of client handling of skipped epochs

* babe: test client-side handling of skipped epochs

* babe: add comments on client-side skipped epochs

* babe: remove emptyline

* babe: make it resilient to forks

* babe: typo

* babe: overflow-safe math

* babe: add test for skipping epochs across different forks

* Fix tests

* FMT

Co-authored-by: Bastian Köcher <info@kchr.de>
2022-12-24 00:03:31 +00:00
Michal Kucharczyk c237b82690 BlockId removal: refactor: HeaderBackend::status (#12981)
It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

Co-authored-by: parity-processbot <>
2022-12-21 10:25:16 +00:00
Michal Kucharczyk 548955a73f BlockId removal: refactor: HeaderBackend::header (#12874)
* BlockId removal: refactor: HeaderBackend::header

It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`,
- `PeersClient::header`
- `ChainApi::block_header`

methods from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* non-trivial usages of haeder(block_id) refactored

This may required introduction of dedicated function:
header_for_block_num

* fmt

* fix

* doc fixed

* ".git/.scripts/fmt.sh"

* BlockId removal: refactor: HeaderBackend::expect_header

It changes the arguments of `HeaderBackend::expect_header` method from: `BlockId<Block>` to: `Block::Hash`

* ".git/.scripts/fmt.sh"

* readme updated

* ".git/.scripts/fmt.sh"

* fix

Co-authored-by: parity-processbot <>
2022-12-20 09:43:31 +00:00
Dmitry Markin e14d6a5c4e Clean up tests from runtime.block_on() and moving around Runtime and Handle (#12941) 2022-12-16 16:29:39 +03:00
Davide Galassi 391adaf443 Use LOG_TARGET in consensus related crates (#12875)
* Use shared LOG_TARGET in consensus related crates
* Rename target from "afg" to "grandpa"
2022-12-14 14:56:17 +01:00
Dmitry Markin 5eb84f9cc6 Upgrade tokio to 1.22.0 and replace async-std with tokio (#12646)
* Replace deprecated libp2p feature specs with correct ones

* Bump tokio to 1.21.2

* Replace async-std libp2p primitives with tokio ones

* minor: rustfmt

* Fix TestNet to run initialization in the tokio context

* Convert telemetry test from async-std to tokio

* Convert notifications tests from async-std to tokio

* Convert chain sync tests from async-std to tokio

* Ditch async-std completely

* Make executor mandatory

* Bump tokio to 1.22.0

* minor: rustfmt

* Explicitly use tokio runtime in tests

* Move more tests to explicit tokio runtime

* Explicitly set multithreaded runtime in tokio test

* minor: rustfmt

* minor: fix comment

* Replace async-std with tokio in MMR tests
2022-12-05 08:18:46 +00:00
alexgparity d20e495812 Reduce provisioner work (#12749)
* Move create_inherent_data call to use side

* Make provide_inherent_data async

* Fix tests

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Log errors

* Fix test

* Fix test

* fix

* Deduplicate test code

* fix

* flag

* Update client/consensus/slots/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Revert "Deduplicate test code"

This reverts commit ba46adbe089329c78cd69ccdb08e27ed67bd77cf.

* Fix test

* remove commented out code

* minor to start CI run

* start CI

* Update client/consensus/slots/src/lib.rs

Co-authored-by: Marcin S. <marcin@bytedude.com>

* Apply PR suggestions

* Apply PR suggestions

* Update client/consensus/slots/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* minor

* kickoff CI

* PR suggestions

* Compute remaining duration instead of using slot_info.duration

* Don't rely on sub implementation for Instant

* Apply PR suggestions

* Use saturating_duration_since

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: parity-processbot <>
2022-12-01 16:51:36 +01:00
Serban Iorga ff439ee335 pallet-mmr: move offchain logic to client-side gadget (#12753)
* Move MMR utils methods from pallet to primitives

Signed-off-by: Serban Iorga <serban@parity.io>

* Add method to MmrApi

* Move forks expanding logic from babe to primitives

* Implement MMR gadget

* Remove prunning logic from the MMR pallet

* Code review changes: 1st iteration

* Replace MaybeCanonEngine with CanonEngineBuilder

* fix mmr_leaves_count() for kitchen sink demo

* Update client/merkle-mountain-range/src/canon_engine.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Code review changes: 2nd iteration

* fix INDEXING_PREFIX

* impl review comments

* add documentation and minor rename

Signed-off-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
2022-11-29 14:39:52 +00:00
Qinxuan Chen ad6630ed71 Update some dependencies to prune duplicated crates with different version (#12560)
* sc-client-babe/sp-arithmetic-fuzzer: update num-bigint and num-rational to v0.4

* update lru 0.7.5 ==> v0.8.1

* pallet-example-offchain-worker: update lite-json v0.1.3 ==> v0.2.0

* update hyper 0.14.16 ==> 0.14.20, num-fromat 0.4.0 ==> 0.4.3

* pallet-mmr: update ckb-merkle-mountain-range v0.3.2 ==> v0.5.2

* update handlebars v4.2.2 ==> v4.3.5

* `runtime_cache_size` must always be at least 1

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* default cache size with .min(1)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* update hyper 0.14.20 ==> 0.14.22

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* update lru 0.8.0 ==> 0.8.1

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Fix Cargo.lock

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Bastian Köcher <info@kchr.de>
2022-11-09 11:46:00 +00:00
Davide Galassi 535c6f2e94 Epoch-Changes tree pruning was lagging by one epoch (#12567)
* Remove all not required nodes from the epoch-changes tree

Some outdated nodes were left there because of the predicate

* Test to excercise the fix

* Add a fork on genesis to the test

* Fix typo in comments
2022-11-09 09:50:36 +01:00
Michal Kucharczyk 1ed70004e7 BlockId removal: &Hash to Hash (#12626)
It changes &Block::Hash argument to Block::Hash.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
2022-11-07 21:42:16 +00:00
Davide Galassi 6201ccaf8f Normalize keystore type and its usage across tests (#12553)
* Normalize keystore type and usage across tests

* Extract peer index from array index
2022-10-24 17:55:25 +00:00
Michal Kucharczyk 749bcd1949 BlockId removal: refactor: Finalizer (#12528)
* BlockId removal: refactor: Finalizer

It changes the arguments of methods of `Finalizer` trait from:
block: `BlockId<Block>` to: hash: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* minor corrections

* failing test corrected

* minor rework
2022-10-20 06:44:04 +00:00
Davide Galassi 499ca74afe Relax Slots-based engines from Epochs (#12360)
Remove Epochs reference from slots subsystem
2022-09-27 13:16:30 +02:00
Davide Galassi bf97f2a702 Independence of Slot-based algorithms from system Timestamp (#12224)
* Remove timestamp from SlotInfo

* Expose as millis instead of secs

* Nits

* Fix test after field removal

* Yet another test fix

* On the fly timestamp computation

* Removed slot timestamp from logs

* Removed reference to timestamp from slots subsystem

* Slot based algorithm tests do not require timstamp inherent anymore

* Remove junk files

* Further tests cleanup

* Trigger pipeline

* Apply code suggestions

* Trigger pipeline

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2022-09-23 17:51:57 +00:00
Davide Galassi 6c7020bb16 Introduce 'intermediate_insert' method to hide implementation details (#12215)
Renaming from 'intermediate_take' to 'intermediate_remove'
2022-09-21 09:42:12 +00:00
Arkadiy Paronyan 86b122d7a7 Fix warp sync (#12281) 2022-09-16 14:16:37 +00:00
Michal Kucharczyk 214eb25f87 Remove CanAuthorWith trait (#12213)
* Remove CanAuthorWith trait

CanAuthotWith trait removed. Also all dependencies, parameters, type
paramers were removed. This is related to removal of native runtime.

* Remove commented code

* Fix code formatting

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job
2022-09-13 12:19:26 +00:00
Davide Galassi 83aec6f93a Fetch Babe configuration from runtime state (#11760)
* Fetch babe config data from runtime state

* Some renaming

* More renaming

* Final nits

* Fix tests and benches

* Rename  to  in BabeConfiguration

* Remove duplicate babe parameter description

Already specified over the 'PRIMARY_PROBABILITY' constant value

* trigger pipeline

* trigger pipeline
2022-09-05 17:41:32 +00:00
Squirrel a68a80fbae declone and close the door (#12035)
* declone and close the door

* cargo fmt

* remove brackets
2022-08-15 19:38:36 +00:00
Sebastian Kunert c831138c0a Remove retain_mut crate (#11926)
* Remove retain_mut crate

* Remove reain_mut crate from babe-consensus
2022-07-27 22:35:33 +00:00