Commit Graph

21 Commits

Author SHA1 Message Date
Bastian Köcher ca5f10567a sc-block-builder: Remove BlockBuilderProvider (#2099)
The `BlockBuilderProvider` was a trait that was defined in
`sc-block-builder`. The trait was implemented for `Client`. This
basically meant that you needed to import `sc-block-builder` any way to
have access to the block builder. So, this trait was not providing any
real value. This pull request is removing the said trait. Instead of the
trait it introduces a builder for creating a `BlockBuilder`. The builder
currently has the quite fabulous name `BlockBuilderBuilder` (I'm open to
any better name 😅). The rest of the pull request is about
replacing the old trait with the new builder.

# Downstream code changes

If you used `new_block` or `new_block_at` before you now need to switch
it over to the new `BlockBuilderBuilder` pattern:

```rust
// `new` requires a type that implements `CallApiAt`. 
let mut block_builder = BlockBuilderBuilder::new(client)
                // Then you need to specify the hash of the parent block the block will be build on top of
		.on_parent_block(at)
                // The block builder also needs the block number of the parent block. 
                // Here it is fetched from the given `client` using the `HeaderBackend`
                // However, there also exists `with_parent_block_number` for directly passing the number
		.fetch_parent_block_number(client)
		.unwrap()
                // Enable proof recording if required. This call is optional.
		.enable_proof_recording()
                // Pass the digests. This call is optional.
                .with_inherent_digests(digests)
		.build()
		.expect("Creates new block builder");
```

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>
2023-11-03 19:06:31 +01:00
Michal Kucharczyk ab3a3bc278 BlockId removal: tx-pool refactor (#1678)
It changes following APIs:
- trait `ChainApi`
-- `validate_transaction`

- trait `TransactionPool` 
--`submit_at`
--`submit_one`
--`submit_and_watch`

and some implementation details, in particular:
- impl `Pool` 
--`submit_at`
--`resubmit_at`
--`submit_one`
--`submit_and_watch`
--`prune_known`
--`prune`
--`prune_tags`
--`resolve_block_number`
--`verify`
--`verify_one`

- revalidation queue

All tests are also adjusted.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-27 11:58:39 +02:00
Juan 6a29a70a92 Replace system config Index for Nonce (#14290)
* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* wip

* remove index in lieu of nonce

* wip

* remove accountnonce in lieu of nonce

* add minor improvement

* rebase and merge conflicts
2023-07-14 06:56:48 +00:00
Michal Kucharczyk 6a295e7c28 substrate-test-runtime migrated to "pure" frame runtime (#13737)
* substrate-test-runtime migrated to pure-frame based

* test block builder: helpers added

* simple renaming

* basic_authorship test adjusted

* block_building storage_proof test adjusted

* babe: tests: should_panic expected added

* babe: tests adjusted

ConsensusLog::NextEpochData is now added by pallet_babe as
pallet_babe::SameAuthoritiesForever trigger is used in runtime config.

* beefy: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digests stored during block execution are
checked against header digests:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(MmrRoot / AuthoritiesChange) is used.

* grandpa: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digest logs stored during block execution are
checked against header digest logs:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(ScheduledChange / ForcedChange and DigestItem::Other) is used.

* network:bitswap: test adjusted

The size of unchecked extrinsic was increased. The pattern used in test will
be placed at the end of scale-encoded buffer.

* runtime apis versions adjusted

* storage keys used in runtime adjusted

* wasm vs native tests removed

* rpc tests: adjusted

Transfer transaction processing was slightly improved, test was
adjusted.

* tests: sizes adjusted

Runtime extrinsic size was increased. Size of data read during block
execution was also increased due to usage of new pallets in runtime.

Sizes were adjusted in tests.

* cargo.lock update

cargo update -p substrate-test-runtime -p substrate-test-runtime-client

* warnings fixed

* builders cleanup: includes / std

* extrinsic validation cleanup

* txpool: benches performance fixed

* fmt

* spelling

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Apply code review suggestions

* Apply code review suggestions

* get rid of 1063 const

* renaming: UncheckedExtrinsic -> Extrinsic

* test-utils-runtime: further step to pure-frame

* basic-authorship: tests OK

* CheckSubstrateCall added + tests fixes

* test::Transfer call removed

* priority / propagate / no sudo+root-testing

* fixing warnings + format

* cleanup: build2/nonce + format

* final tests fixes

all tests are passing

* logs/comments removal

* should_not_accept_old_signatures test removed

* make txpool benches work again

* Cargo.lock reset

* format

* sudo hack removed

* txpool benches fix+cleanup

* .gitignore reverted

* rebase fixing + unsigned cleanup

* Cargo.toml/Cargo.lock cleanup

* force-debug feature removed

* mmr tests fixed

* make cargo-clippy happy

* network sync test uses unsigned extrinsic

* cleanup

* ".git/.scripts/commands/fmt/fmt.sh"

* push_storage_change signed call remove

* GenesisConfig cleanup

* fix

* fix

* GenesisConfig simplified

* storage_keys_works: reworked

* storage_keys_works: expected keys in vec

* storage keys list moved to substrate-test-runtime

* substrate-test: some sanity tests + GenesisConfigBuilder rework

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

* Review suggestions

* fix

* fix

* beefy: generate_blocks_and_sync block_num sync with actaul value

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* cargo update -p sc-rpc -p sc-transaction-pool

* Review suggestions

* fix

* doc added

* slot_duration adjusted for Babe::slot_duration

* small doc fixes

* array_bytes::hex used instead of hex

* tiny -> medium name fix

* Apply suggestions from code review

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* TransferData::try_from_unchecked_extrinsic -> try_from

* Update Cargo.lock

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-05-04 16:20:22 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Michal Kucharczyk 57b1de3f47 txpool: LOG_TARGET const added (#13180)
* txpool: LOG_TARGET const added

part of: #12873

* LOG_TARGET added to tests mod

* txpool::api for api

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-01-19 22:21:41 +00:00
Michal Kucharczyk a1a9dff12c txpool: TestApi best_block fix (#12992)
This is a minor clean up.

There was a minor bug in TestApi::add_block: the best block flag for
given number was not cleared when the new best block with the same
number was added.

The usage of is_best_block argument in add_block method in txpool tests
were aligned with the BestBlock event.
2022-12-21 16:36:08 +00:00
Marcin S aa21e56744 Remove mem_info and references to parity-util-mem (#12795)
* Remove mem_info and some references to parity-util-mem

* [Draft] Finish removing references to `parity-util-mem`

* Upgrade dependencies

* Update scripts/ci/deny.toml

Co-authored-by: ordian <write@reusable.software>

* Fix Cargo.lock (remove unwanted dependency changes)

* Removed unused argument

* Run cargo fmt (didn't have pre-commit set up)

* Fix some CI errors

* Fix another CI error

* Remove unused dependency

Co-authored-by: ordian <write@reusable.software>
2022-12-06 14:55:10 +00:00
Michal Kucharczyk 453b9ef235 tx-pool: failing tests fixed (#12481) 2022-10-12 08:06:22 +00:00
Michal Kucharczyk 62bca87f3a Finalized block event triggers tx maintanance (#12305)
* finalized block event triggers tx maintanance

* tx-pool: enactment helper introduced

* tx-pool: ChainApi: added tree_route method

* enactment logic implemented + tests

Signed-off-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Some additional tests

* minor improvements

* trigger CI job

* fix compilation errors

ChainApi::tree_route return type changed to Result<Option<..>>, as some
implementations (tests) are not required to provide this tree route.

* formatting

* trait removed

* implementation slightly simplified

(thanks to @koute)

* get rid of Arc<> in EnactmentState return value

* minor improvement

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Apply suggestions from code review

* comment updated + formatting

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>

* formatting

* finalization notification bug fix

+ new test case
+ log::warn message when finalized block is being retracted by new event

* added error message on tree_route failure

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* use provided tree_route in Finalized event

* Option removed from ChainApi::tree_route

* doc added, test and logs improved

* handle_enactment aligned with original implementation

* use async-await

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* formatting + warn->debug

* compilation error fix

* enactment_state initializers added

* enactment_state: Option removed

* manual-seal: compilation & tests fix

* manual-seal: tests fixed

* tests cleanup

* another compilation error fixed

* TreeRoute::new added

* get rid of pub hack

* one more test added

* formatting

* TreeRoute::new doc added + formatting

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* (bool,Option) simplified to Option

* log message improved

* yet another review suggestions applied

* get rid of hash in handle_enactment

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/transaction-pool/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* minor corrections

* EnactmentState moved to new file

* File header corrected

* error formatting aligned with codebase

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* remove commented code

* small nits

Signed-off-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
2022-10-11 20:20:13 +00:00
Alexandru Vasile 5f18aaadcd rpc: Implement transaction RPC API (#12328)
* rpc/tx: Add transaction structures for serialization

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Add public facing `TransactionEvent`

To circumvent the fact that serde does not allow mixing
`#[serde(tag = "event")]` with
`#[serde(tag = "event", content = "block")]`
the public facing subscription structure is serialized
and deserialized to an intermmediate representation.

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Add trait for the `transaction` API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Convert RPC errors to transaction events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Implement `transaction` RPC methods

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* tx-pool: Propagate tx index to events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* tx-pool: Adjust testing to reflect tx index in events

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Convert tx-pool events for the new RPC spec

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Convert tx-pool `FinalityTimeout` event to `Dropped`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* service: Enable the `transaction` API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Add tests for tx event encoding and decoding

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* tx: Add indentation for subscriptions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Fix documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Serialize usize to hex

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* tx-pool: Rename closure parameters

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* service: Separate RPC spec versions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Use `H256` for testing block's hash

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Serialize numbers as string

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* tx-pool: Backward compatibility with RPC v1

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/rpc-spec-v2/src/transaction/transaction.rs

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* rpc/tx: Remove comment about serde clone

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* rpc/tx: Use RPC custom error code for invalid tx format

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/rpc-spec-v2/src/transaction/event.rs

Co-authored-by: James Wilson <james@jsdw.me>

* rpc/tx: Adjust internal structures for serialization/deserialization

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
2022-10-11 08:49:12 +00:00
Xavier Lau 86198c5471 Use array-bytes for All Array/Bytes/Hex Operations (#12190)
* Use `array-bytes` for All Array/Bytes/Hex Operations

Signed-off-by: Xavier Lau <xavier@inv.cafe>

* Reorder

* Self Review

* Format

* Fix Tests

* Bump `array-bytes`

* Optimize large test res

Signed-off-by: Xavier Lau <xavier@inv.cafe>
Co-authored-by: parity-processbot <>
2022-09-21 06:12:20 +00:00
Squirrel a68a80fbae declone and close the door (#12035)
* declone and close the door

* cargo fmt

* remove brackets
2022-08-15 19:38:36 +00:00
Sebastian Kunert 9f409dc0b8 Prepare for rust 1.62.1 (#11903)
* Update UI test output for rust 1.62.1

* switch ci to staging image to check that everything works

* fix artifacts node-bench-regression-guard

* Imeplement `scale_info::TypeInfo` manually to silence aggressive rust warning

* Fix more clippy lints

* Make clippy happy by relying on auto-deref were possible

* Add tracking issue to the comments

* pin ci image

Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-07-26 14:37:05 +02:00
Qinxuan Chen 41c84b0b41 Remove useless imports (#11136)
Because `TryInto`/`TryFrom` are in prelude by default from edition 2021

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2022-04-03 21:44:29 +02:00
Roman Gafiyatullin 4009148889 #10576: generic utility to unsubscribe from broadcast upon drop of the rx-side. (#10708)
* #10576: refactor `sc-utils::notification` and `sc-client-api::notifications`, so that they use common subscribe/unsubscribe routines

* Add some docs. Reorganise `sc-utils::notification`

* `sc-clent-api::notifications` and `sc-utils::notification` — ensure the SubscriptionGuard is dropped before the Rx-channel

* `sc-utils::pubsub::SubscriptionGuard` make it a bit more ergonomic.

Let the `Rx` to be put inside of the `SubscriptionGuard`, so that the latter shall guarantee the order:
- first unsubscribe;
- then drop the `Rx`.

* Being less zealous with splitting the modules into little pieces

* rework pubsub: the concrete usage should only define a good registry type

* sc-client-api::notifications: make it comply with the reworked pubsub

* cargo fmt

* make sc-client-api tests work

* Address the review notes

* cargo fmt

* Describe the behaviour of pubsub registry

* Doc-comments for module `sc-utils::pubsub`

* Fix: it used to send notifications regardless of the filter setup during subscription

* `sc-client-api::StorageNotifications` the API does not have to require mut-self-reference.

As a result `sc-service::Client` does not have to wrap its `storage_notifications` into a Mutex.

* cargo fmt

* Several changes addressing the notes by @bckhr.

- Remove the `impl Default for StorageNotifications<Block>`;
- no need for groupping the `remove_from` and `listen_from` into a separate `helpers` module;
- remove unnecessary import `use registry::SubscribeOp`.

* Add a doc-comment to the `sc-client::notifications::SubscribeOp`

* As per @bkchr note on the unproven assertion: behave gracefully upon receiving a duplicate subscription-ID.

* sc-utils::pubsub: log when a registry yields an ID that does point to an existing sink

* `sc-utils::notifications`: payload materialized lazily

* Update Cargo.lock (after adding `log` as a dependency to the `sc-utils`)

* `sc-client-api::notifications`: introduce a struct (instead of a type def) for the notification message

* Get rid of `sc-utils::pubsub::Channel` trait (instead just use the `sc-utils::mpsc`)

* The SubsID is no more generic: the fact it is a `Copy` is known — no need to pass it by ref

* sc-utils::pubsub internals do not have to be generic over the channel type

* Rename Hub::dispatch into Hub::send

* That method was unnecessary (`SubscriberSink::render_notification`)

* cargo fmt

* No need for a separate UnsubscribeGuard type

* Ditch the type-def of SubsID in the sc-utils::pubsub, instead — just use the crate::id_sequence::SeqID

* Return the <Registry as Dispatch>::Ret when sending an item

* Make the `Hub<M, R>::lock_registry(...)` method more ergonomic

* cargo doc links

* cargo doc links

* Use a simpler name for the type

* cargo doc links

* Derive `Default` rather than implement it

* Derive `Default` rather than implement it

* Remove an unnecessary usage of type_name

* Define a more cautious order between sinks.remove->registry.unsubscribe and registry.subscribe->sinks.insert

* Hub: lock_registry_for_tests->map_registry_for_tests — a safer choice for a public API

* Replace Mutex over the shared Registry with a ReentrableMutex+RefCell

* sc-utils::pubsub: add tests for a panicking registry

* Add the missing copyright headers

* Arc<Vec<_>> -> Arc<[_]>
2022-02-28 23:51:35 +02:00
Davide Galassi 7f3bb8d0da Add stale branches heads to finality notifications (#10639)
* Add stale branches heads to finality notifications

Warning. Previous implementation was sending a notification for
each block between the previous (explicitly) finalized block and
the new finalized one (with an hardcoded limit of 256).

Now finality notification is sent only for the new finalized head and it
contains the hash of the new finalized head, new finalized head header,
a list of all the implicitly finalized blocks and a list of stale
branches heads (i.e. the branches heads that are not part of the
canonical chain anymore).

* Add implicitly finalized blocks list to `ChainEvent::Finalized` message

The list contains all the blocks between the previously finalized block
up to the parent of the currently finalized one, sorted by block number.

`Finalized` messages handler, part of the `MaintainedTransactionPool`
implementation for `BasicPool`, still propagate full set of finalized
blocks to the txpool by iterating over implicitly finalized blocks list.

* Rust fmt

* Greedy evaluation of `stale_heads` during finalization

* Fix outdated assumption in a comment

* Removed a test optimization that is no more relevant

The loop was there to prevent sending to
`peer.network.on_block_finalized` the full list of finalized blocks.

Now only the finalized heads are received.

* Last finalized block lookup not required anymore

* Tests for block finality notifications payloads

* Document a bit tricky condition to avoid duplicate finalization notifications

* More idiomatic way to skip an iterator entry

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Cargo fmt iteration

* Typo fix

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix potential failure when a finalized orphan block is imported

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-01-27 18:21:58 +01:00
Bastian Köcher 424ddf33ee Remove transaction-pool test-helpers feature (#10571)
* Remove transaction-pool `test-helpers` feature

`test-helpers` feature is a bad idea in general, because once the feature is enabled somewhere in
the workspace, it is enabled anywhere. While removing the feature, the tests were also rewritten to
get rid off other "only test" related code.

Contributes towards: https://github.com/paritytech/substrate/issues/9727

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix benches

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2022-01-03 15:08:42 +00:00
Bastian Köcher f3662b4bba Happy new year 2022 (#10573) 2022-01-03 09:22:14 +01:00
Bastian Köcher 7b56ab15b4 Run cargo fmt on the whole code base (#9394)
* Run cargo fmt on the whole code base

* Second run

* Add CI check

* Fix compilation

* More unnecessary braces

* Handle weights

* Use --all

* Use correct attributes...

* Fix UI tests

* AHHHHHHHHH

* 🤦

* Docs

* Fix compilation

* 🤷

* Please stop

* 🤦 x 2

* More

* make rustfmt.toml consistent with polkadot

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-07-21 14:32:32 +00:00
Seun Lanlege 2ae9d36758 sc-transcation-pool refactor (#9228)
* Use TransactionPool trait

* sc-transaction-pool-primitives

* sc-transaction-pool-api

* TP

* bye sc_transaction_graph

* fix line widths

* fix import errors

* fix import errors

* fix import errors 🤦🏾‍♂️

* fix import errors 🤦🏾‍♂️🤦🏾‍♂️🤦🏾‍♂️

* remove sp-keyring
2021-07-08 13:33:34 +00:00