Commit Graph
75 Commits
Author SHA1 Message Date
Sebastian Kunert d21113c13f Delete full db directory with purge-chain subcommand (#1786)
Closes #1767 

Until now the `purge-chain` command would only remove the `full`
subfolder of the db folder. However there is also the `parachains` db
that currently remains and can cause problems on node restart.

Example wiht old code:
```
polkadot purge-chain --database paritydb --base-path /tmp/some-folder
Are you sure to remove "/tmp/some-folder/chains/polkadot/paritydb/full"? [y/N]: y
"/tmp/some-folder/chains/polkadot/paritydb/full" removed.
```
In this case `/tmp/some-folder/chains/polkadot/paritydb/parachains`
would remain and might cause problem on node restart because of version
conflicts as described in #1767. After this PR the whole
`/tmp/some-folder/chains/polkadot/paritydb` folder will be deleted.
2023-10-05 18:44:03 +02:00
Sebastian Kunert 9485b0b492 Point documentation links to monorepo (#1741) 2023-09-29 22:33:26 +02:00
Sebastian KunertandBastian Köcher 4902db2198 Use Extensions to register offchain worker custom extensions (#1719)
Closes #1671 

Adds a `type_id` function to the `Extension` trait, allowing to properly store an retrieve
boxed `Extensions`.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-29 09:37:51 +02:00
Sebastian Kunert fb0fd3e624 Use correct target dir in polkadot readme (#1643)
Closes #1372 

This fixes the target directories in the polkadot readme. I verified
manually the Ubuntu based install works on a fresh cloud machine. Build
from source also worked as described.

Since #1304 landed `--dev` also works again (not on v1.1.0 though), so I
would consider the install instructions fixed.
2023-09-22 12:19:09 +02:00
817c4503db Allow integrated relay chain light client (#2270)
* Add embedded light client to cli

* Prepare for light-client-worker

* First working version

* Clean up

* Remove unwanted logs

* Simplify subscription code

* Let jsonrpsee handle rpc management

* Simplify implementation

* Reorganize crate structure

* Use relay chain arg chainspec for light-client

* Clean up command line

* Add light client worker file

* Use smoldot master to avoid wasmtime conflict

* Remove sleep

* Improve naming of cli option

* Remove conflict with `validator`

* Improve docs

* Update smoldot, remove unwanted change

* Apply suggestions from code review

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

* Disable collation

* Reviewer comments

* Update smoldot and tokio-platform

* Update smoldot

* Update smoldot

* Adjust to new version

* Patch substrate

* Use constants

* Add readme entry, improve zombienet tests

* Apply suggestions from code review

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Make execution mode an enum

* Update smoldot, remove substrate patch

* Update client/relay-chain-rpc-interface/src/rpc_client.rs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Reduce duplicate code

* Update smoldot

* Update smoldot

* Fix build

* Update smoldot

* Make platform compile

* Clean up dependencies

* Use crates.io instead of github for smoldot

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Docs

* Improve docs

* Remove `RpcFrontend`

---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-08-22 15:53:32 +02:00
Sebastian Kunertandparity-processbot 3d1b31a775 Companion for substrate#14776 (#3038)
* Bump

* update lockfile for {"polkadot", "substrate"}

---------

Co-authored-by: parity-processbot <>
2023-08-21 15:44:50 +00:00
Sebastian Kunertandparity-processbot e6451b7232 Companion for substrate#14776 (#7648)
* Bump dalek

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-08-21 15:02:13 +00:00
5b5942ef60 Use trie-cache for validate_block (#2813)
* Simple cache

* Fix node insertion

* Switch to hashbrown hashmap

* Remove unused phantomdata

* Return error when fetch_node fails

* Remove cargo patches

* Move trie cache to extra module

* Add ReadOnceBackend

* Add readonlybackend

* Improve naming and get_or_insert

* Stylistic improvements

* Improve naming, add docs

* Revert unwanted changes

* Remove unused dependencies

* Improve docs

* Use RefCell

* lockfile

* Remove ReadOnceBackend

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Code review

* Do not use value cache when calculating storage roots

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Remove hash-db dep

* Update pallets/parachain-system/src/validate_block/trie_cache.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Anton <anton.kalyaev@gmail.com>
2023-07-19 11:46:58 +00:00
Sebastian KunertandBastian Köcher b6b1952fc6 Make trie-cache resettable from backend (#14516)
* Add ability to reset trie-cache

* comment

* Update client/db/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-07-18 17:05:40 +02:00
Sebastian Kunertandparity-processbot 3a10a29df5 Companion for substrate#14403 (#7435)
* Bump trie-db

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-06-28 14:27:05 +00:00
30a7e6788c Make triecache generic and work for no-std environments (#14403)
* Remove `AsLocalTrieCache` trait

* Introduce new trait AsTrieDbCache

* Use AsTrieDbCache trait

* Make it compile

* Docs

* Improve naming of associated type, implement cache usage for no-std

* Improve naming

* Improve docs

* Allow construction with optional cache

* FMT

* Remove unused variable

* Revert unwanted change

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* More comment adjustments

* Docs

* Trigger CI

* ".git/.scripts/commands/fmt/fmt.sh"

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Review comments

* Review comments

* Apply suggestions from code review

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* fmt

* Bump trie-db again

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
2023-06-28 12:57:41 +00:00
Sebastian Kunertandparity-processbot f653f4519b Companion for substrate #14464 (#7429)
* Remove unneeded feature flag

* More serde flags

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-06-27 14:06:57 +00:00
Sebastian Kunert ede49c7ae6 Remove unused dependencies (#14464) 2023-06-27 13:33:42 +00:00
c047ec900a Add benchmarks for for import/validation/production (#2579)
* Add benchmarks

* Remove endowed accounts

* Fix group

* Remove duplicate code

* Comments

* Use execute_block instead of block import

* Apply suggestions from code review

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* Add missing comments

* Clippy

* Reorganize utils

* Remove unused import

* Switch to compiled again

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Refactor local_testnet

* Do not write wasm blob to file

* Remove unnecessary block

* Add comment fixes

* fmt

* lock

---------

Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-06-05 16:41:20 +02:00
Sebastian Kunert 1d42bdf664 Remove manual implementation (#14096) 2023-05-08 20:32:01 +00:00
Sebastian Kunert f79d45da09 Remove manual implementation (#14096) 2023-05-08 20:31:39 +00:00
Sebastian Kunertandparity-processbot b72ae577a0 Make zombienet tests required (#2527)
Co-authored-by: parity-processbot <>
2023-05-06 07:58:25 +02:00
Sebastian KunertandBastian Köcher d3b3a46d27 Only calculate tree route during finalization when there are multiple leaves (#14067)
* Only calculate tree route when there are multiple leaves

* Update client/service/src/client/client.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-05-03 11:41:51 +00:00
Sebastian Kunert 2d9b3131d5 Skip av-store, make consensus task blocking (#2497) 2023-05-01 10:22:56 +02:00
Sebastian KunertandMichal Kucharczyk 0ae0393042 Add option to skip av-store requests in availability-recovery-subsystem (#7131)
* Allow to skip availability-store

* Update node/network/availability-recovery/src/lib.rs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
2023-04-28 10:13:04 +00:00
Sebastian Kunert 36b0459d6b Parachain node should not recover blocks while syncing (#2462) 2023-04-19 16:02:35 +02:00
Sebastian Kunert 6c896b1ee6 Cleanup: Remove polkadot-service dependency from minimal node (#2430)
* Remove polkadot-service dependency from minimal-node

* Clean up error handline

* Remove unwanted changes

* Unused deps
2023-04-04 15:05:38 +00:00
Sebastian Kunert 59fb7ff59c help text examples + clean up (#2418) 2023-04-01 16:44:32 +02:00
Sebastian Kunertandparity-processbot d98f3d1fec Remove HeaderBackend from RelayChainRPCClient (#2385)
* Remove HeaderBackend from RelayChainRPCClient

* update lockfile for {"substrate", "polkadot"}

---------

Co-authored-by: parity-processbot <>
2023-03-28 07:37:58 +00:00
Sebastian Kunert c41eadc090 Remove HeaderBackend requirement from AuthorityDiscovery and NetworkWorker (#13730)
* Remove `HeaderBackend` requirement from `NetworkWorker`

* Remove HeaderBackend from authority-discovery
2023-03-28 07:10:50 +00:00
Sebastian Kunert 7bd3f3d762 Remove unused light-client leftover (#13687)
* Remove unused light-client leftover

* Remove more unused code
2023-03-23 11:35:39 +01:00
Sebastian Kunert ccfe5fe08c Always pass port to jsonrpsee WebSocket client (#2339)
* Always pass port to jsonrpsee

* Remove useless host check

* Do not silently drop
2023-03-20 14:05:06 +00:00
Sebastian Kunert f4f467beff ReadMe followup for #2313 (#2317)
* Readme again

* relaychain -> relay chain
2023-03-15 09:33:42 +01:00
7aa2ea2690 README: Document usage of RPC node (#2313)
* Update readme

* Improve

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Quote urls

* Apply suggestions from code review

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
2023-03-14 14:56:32 +00:00
Sebastian Kunert 15ba3d666f Cleanup dependencies + dead code (#2302) 2023-03-10 17:09:38 +01:00
Sebastian KunertandBastian Köcher 588bdad7f6 Add retry mechanism for pov-recovery, fix full-node pov-recovery (#2164)
* Increase delay for pov-recovery

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Comment

* FMT

* Clear waiting_recovery when block is recovered or recovery failed

* Introduce recovery queue that preserved insertion order

* Better error logs

* Decrease slot duration

* Style improvements

* Add option to use unordered queue

* Maintain cache of finalized blocks

* Wait for one relay chain slot before recovery

* Make retries testable

* fmt

* Improve docs

* Improve docs

* Simplify RecoveryQueue

* Remove unwanted changes

* Adjust to comments

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Move recovery delay into the queue

* Check for finalized number

* Clean up

* Use timer

Co-authored-by: Bastian Köcher <git@kchr.de>

* Simplify implementation

* Revert "Use timer"

This reverts commit 3809eed840d3a09d54212f99486782ff80cdc1c9.

* Properly clear `to_recover` flag

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-02-09 14:18:55 +01:00
Sebastian Kunert 3af67716c3 Do not verify on state import for relay chain verifier (#2140)
* Skip verification on state import

* Actually use existing variables

* Merge checks
2023-01-26 12:27:24 +00:00
Sebastian Kunert ad453c556b Remove assumption that there is a local bootnode in RPC collator (#2091)
* Remove assumption that there is a local bootnode

* Bump zombienet to 1.3.29

* Separate "--"
2023-01-24 15:07:18 +01:00
Sebastian Kunertandparity-processbot 0f94229465 Companion for Substrate #13157 (#6582)
* Update getrandom and version_check

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2023-01-19 15:48:56 +00:00
9be655701a Notification-based block pinning (#13157)
* Worker

* Reorganize and unpin onnotification drop

* Pin in state-db, pass block number

* Pin blocks in blockchain db

* Switch to reference counted LRU

* Disable pinning when we keep all blocks

* Fix pinning hint for state-db

* Remove pinning from backend layer

* Improve readability

* Add justifications to test

* Fix justification behaviour

* Remove debug prints

* Convert channels to tracing_unbounded

* Add comments to the test

* Documentation and Cleanup

* Move task start to client

* Simplify cache

* Improve test, remove unwanted log

* Add tracing logs, remove expect for block number

* Cleanup

* Add conversion method for unpin handle to Finalitynotification

* Revert unwanted changes

* Improve naming

* Make clippy happy

* Fix docs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Use `NumberFor` instead of u64 in API

* Hand over weak reference to unpin worker task

* Unwanted

* &Hash -> Hash

* Remove number from interface, rename `_unpin_handle`, LOG_TARGET

* Move RwLock one layer up

* Apply code style suggestions

* Improve comments

* Replace lru crate by schnellru

* Only insert values for pinned items + better docs

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Improve comments, log target and test

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-01-19 15:13:16 +00:00
Sebastian Kunert b929e0282d Remove unused code in state-db (#13103) 2023-01-09 14:46:09 +01:00
Sebastian Kunert 123726f308 Remove unused finalization_displaced variable in db (#13090) 2023-01-06 21:09:57 +01:00
Sebastian Kunert cf00218f77 Remove integration tests that are covered by zombienet (#1998) 2022-12-15 17:06:02 +01:00
Sebastian Kunert 7cab12e9d2 Allow to specify multiple relay chain RPC urls for collator node (#1880)
* Allow specification of multiple urls for relay chain rpc nodes

* Add pooled RPC client basics

* Add list of clients to pooled client

* Improve

* Forward requests to dispatcher

* Switch clients on error

* Implement rotation logic

* Improve subscription handling

* Error handling cleanup

* Remove retry from rpc-client

* Improve naming

* Improve documentation

* Improve `ClientManager` abstraction

* Adjust zombienet test

* Add more comments

* fmt

* Apply reviewers comments

* Extract reconnection to extra method

* Add comment to reconnection method

* Clean up some dependencies

* Fix build

* fmt

* Provide alias for cli argument

* Apply review comments

* Rename P* to Relay*

* Improve zombienet test

* fmt

* Fix zombienet sleep

* Simplify zombienet test

* Reduce log clutter and fix starting position

* Do not distribute duplicated imported and finalized blocks

* fmt

* Apply code review suggestions

* Move building of relay chain interface to `cumulus-client-service`

* Refactoring to not push back into channel

* FMT
2022-12-15 11:42:07 +01:00
Sebastian Kunert edee3d557b Remove more protocols from minimal collator (#1927) 2022-12-01 10:35:47 +01:00
Sebastian Kunert 3f1db7bfb0 Remove av-store, av-distribution, chain-api subsystems from minimal node (#1903)
* Remove unused subsystems

* Consume requests we are not going to answer

* Respond to incoming chunk and pov requests.

* fmt

* Improve comment and logging

* Add reputation changes

* Do not register pov-chunk-request protocol

* Remove inbound chunk request channel

* Bump Polkadot
2022-11-27 14:35:51 +00:00
Sebastian Kunert 53d9a50c06 Add support for outbound only configs on request/response protocols (#6343)
* Add option ot add outbound_only configurations

* Improve comment
2022-11-26 10:22:09 +01:00
Sebastian KunertandBastian Köcher 9a3e518e5c Add example zombienet network file and instructions (#1839)
* Add simple example on how to spin up network.

* Update readme

* Remove unnecessary prefix

* Improve folder structure

* Add link to file

* Fix paths in readme

* Update README.md

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-11-08 18:26:20 +00:00
Sebastian Kunert b32fcdc96d Add RPC nodes to pov-recovery test (#1807) 2022-10-31 19:45:20 +01:00
Sebastian Kunert 9e690fe1b6 Bump substrate (#6164) 2022-10-18 17:13:28 +00:00
Sebastian Kunert 096553cb59 Use ArgAction::Set for enable-offchain-indexing flag (#12521)
* Use ArgAction::Set for enable-offchain-indexing

* Provide default value for `enable-offchain-indexing`
2022-10-18 15:41:19 +00:00
Sebastian Kunert 975403e802 Update clap to version 4 (#1745)
* Bump to clap4

* Adjust to clap 4 style

* Remove two more deprecated occurences of clap macro

* Remove "deprecated" feature from clap

* Update cargo lock

* Fix group name

* More skipped group names
2022-10-18 11:30:22 +00:00
Sebastian Kunertandparity-processbot 99f705537b Update clap to version 4 (#6128)
* Move clap to 4.0.9

* Remove "deprecated" feature flag

* Convert to default_value_t

* update lockfile for {"substrate"}

* Add group(skip) to avoid naming conflict

* More group(skip)

Co-authored-by: parity-processbot <>
2022-10-18 09:02:41 +00:00
Sebastian KunertandBastian Köcher f687db40f7 Bump clap to 4.0.x and adjust to best practices (#12381)
* Bump clap to 3.2.22

* Replace `from_os_str` with `value_parser`

* Replace `from_str` and `try_from_str` with `value_parser`

* Move possible_values to the new format

* Remove unwanted print

* Add missing match branch

* Update clap to 4.0.9 and make it compile

* Replace deprecated `clap` macro with `command` and `value`

* Move remaining `clap` attributes to `arg`

* Remove no-op value_parsers

* Adjust value_parser for state_version

* Remove "deprecated" feature flag and bump to 4.0.11

* Improve range

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions

* Trigger CI

* Fix unused error warning

* Fix doc errors

* Fix ArgGroup naming conflict

* Change default_value to default_value_t

* Use 1.. instead of 0..

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-10-18 06:52:46 +00:00
Sebastian Kunert cdcf5ddd9b Enable collation via RPC relay chain node (#1585)
* Add minimal overseer gen with dummy subsystems

* Fix dependencies

* no-compile: only client transaction pool missing

* Remove unused imports

* Continue to hack towards PoC

* Continue

* Make mini node compile

* Compiling version with blockchainevents trait

* Continue

* Check in lockfile

* Block with tokio

* update patches

* Update polkadot patches

* Use polkadot-primitives v2

* Fix build problems

* First working version

* Adjust cargo.lock

* Add integration test

* Make integration test work

* Allow startinc collator without relay-chain args

* Make OverseerRuntimeClient async

* Create separate integration test

* Remove unused ChainSelection code

* Remove unused parameters on new-mini

* Connect collator node in test to relay chain nodes

* Make BlockChainRPCClient obsolete

* Clean up

* Clean up

* Reimplement blockchain-rpc-events

* Revert "Allow startinc collator without relay-chain args"

This reverts commit f22c70e16521f375fe125df5616d48ceea926b1a.

* Add `strict_record_validation` to AuthorityDiscovery

* Move network to cumulus

* Remove BlockchainRPCEvents

* Remove `BlockIdTo` and `BlockchainEvents`

* Make AuthorityDiscovery async

* Use hash in OverseerRuntime

* Adjust naming of runtime client trait

* Implement more rpc-client methods

* Improve error handling for `ApiError`

* Extract authority-discovery creationand cleanup

* RPC -> Rpc

* Extract bitswap

* Adjust to changes on master

* Implement `hash` method

* Introduce DummyChainSync, remove ProofProvider and BlockBackend

* Remove `HeaderMetadata` from blockchain-rpc-client

* Make ChainSync work

* Implement NetworkHeaderBackend

* Cleanup

* Adjustments after master merge

* Remove ImportQueue from network parameters

* Remove cargo patches

* Eliminate warnings

* Revert to HeaderBackend

* Add zombienet test

* Implement `status()` method

* Add more comments, improve readability

* Remove patches from Cargo.toml

* Remove integration test in favor of zombienet

* Remove unused dependencies, rename minimal node crate

* Adjust to latest master changes

* fmt

* Execute zombienet test on gitlab ci

* Reuse network metrics

* Chainsync metrics

* fmt

* Feed RPC node as boot node to the relay chain minimal node

* fmt

* Add bootnodes to zombienet collators

* Allow specification of relay chain args

* Apply review suggestions

* Remove unnecessary casts

* Enable PoV recovery for rpc full nodes

* Revert unwanted changes

* Make overseerHandle non-optional

* Add availability-store subsystem

* Add AuxStore and ChainApiSubsystem

* Add availability distribution subsystem

* Improve pov-recovery logging and add RPC nodes to tests

* fmt

* Make availability config const

* lock

* Enable debug logs for pov-recovery in zombienet

* Add log filters to test binary

* Allow wss

* Address review comments

* Apply reviewer comments

* Adjust to master changes

* Apply reviewer suggestions

* Bump polkadot

* Add builder method for minimal node

* Bump substrate and polkadot

* Clean up overseer building

* Add bootnode to two in pov_recovery test

* Fix missing quote in pov recovery zombienet test

* Improve zombienet pov test

* More debug logs for pov-recovery

* Remove reserved nodes like on original test

* Revert zombienet test to master
2022-10-10 09:06:26 +02:00
Sebastian Kunert 9b669edf22 Change visibility of open_database for use in cumulus (#6034)
* Make some helper methods public

* Make availability config const.
2022-09-22 08:45:58 +00:00
Sebastian KunertandSebastian Kunert 800bc5cd8c Simplify trait bounds in network to prepare for collator-rpc (#12082)
* Hack towards PoC

* Abstract away runtime requirement

* blockchainevents

* Remove bitswap

* Remove unused sync more

* Remove unused features in network

* Re-enable bitswap change

* Remove `Chain` trait bound

* Reimplement blockchain-rpc-events

* Move network to cumulus

* Make AuthorityDiscovery async

* Remove `ProofProvider` requirement from network behaviour

* Extract bitswap

* Adjustments after merge

* Remove HeaderMetadata trait from network

* Introduce NetworkHeaderBackend

* Add comments

* Improve comments

* Move NetworkHeaderBackend to new module

* Improve naming, remove redundand send + sync

* Clean up generics

* Fix CI

* Improve comment and readability

* Remove NetworkHeaderBackend

* Fix Cargo.lock

Co-authored-by: Sebastian Kunert <skunert@Sebastians-MacBook-Pro.fritz.box>
2022-08-31 14:55:46 +02:00
Sebastian KunertandDavide Galassi 36e580a56a Introduce RPC Worker (#1462)
* Extract json-rpc-client and introduce worker

* Initial rpc worker

* Add error handling

* Use bounded channels for listeners

* Improve naming and clean up

* Use tracing channels

* Improve code readability

* Decrease channel size limit

* Remove unused dependency

* Fix docs

* RPC -> Rpc

* Start worker in initialization method

* Print error in case a distribution channel is full

* Fix docs

* Make `RpcStreamWorker` private

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Use tokio channels and add TODO item

* Remove `Option` from `to_worker_channel`

Co-authored-by: Davide Galassi <davxy@datawok.net>
2022-08-01 12:08:57 +02:00
Sebastian Kunert c831138c0a Remove retain_mut crate (#11926)
* Remove retain_mut crate

* Remove reain_mut crate from babe-consensus
2022-07-27 22:35:33 +00:00
Sebastian Kunertandalvicsam 9f409dc0b8 Prepare for rust 1.62.1 (#11903)
* Update UI test output for rust 1.62.1

* switch ci to staging image to check that everything works

* fix artifacts node-bench-regression-guard

* Imeplement `scale_info::TypeInfo` manually to silence aggressive rust warning

* Fix more clippy lints

* Make clippy happy by relying on auto-deref were possible

* Add tracking issue to the comments

* pin ci image

Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-07-26 14:37:05 +02:00
Sebastian Kunertandalvicsam 88d9aca202 Prepare for rust 1.62.1 (#5811)
* Update update-rust-stable script

* Update UI tests for rust-1.62.1

* Use staging CI image

* pin ci image

Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-07-26 14:37:02 +02:00
Sebastian Kunert 9afc93cbd1 Bump strum to 0.24.1 (#5816) 2022-07-26 14:02:43 +02:00
Sebastian Kunertandparity-processbot 7582c735df Companion for polkadot#5782 (#1447)
* Make HeadSupportsParachain async

* Update Cargo.lock

* update lockfile for {"substrate", "polkadot"}

Co-authored-by: parity-processbot <>
2022-07-20 11:03:02 +00:00
72bde2889f Introduce async runtime calling trait for runtime-api subsystem (#5782)
* Implement OverseerRuntimeClient

* blockchainevents

* Update patches

* Finish merging rntime-api subsystem

* First version that is able to produce blocks

* Make OverseerRuntimeClient async

* Move overseer notification stream forwarding to cumulus

* Remove unused imports

* Add more logging to collator-protocol

* Lockfile

* Use hashes in OverseerRuntimeClient

* Move OverseerRuntimeClient into extra module

* Fix old session info call and make HeadSupportsParachain async

* Improve naming of trait

* Cleanup

* Remove unused From trait implementation

* Remove unwanted debug print

* Move trait to polkadot-node-subsystem-types

* Add sections to runtime client

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Reorder methods

* Fix spelling

* Fix spacing in Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove unused babe methods

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-07-20 10:23:25 +00:00
Sebastian Kunert 6e13b44694 Rename help string parachain-collator -> parachain-template-node (#1392) 2022-06-23 11:36:33 +02:00
Sebastian KunertandBastian Köcher 10e966e643 Respect cargo offline env variable in wasm builder (#11735)
* Support offline env variable in wasm builder

* Clean up

* Improve checks

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update crate docs

* Add docs to `lib.rs` and introduce helper method `offline_build`

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-06-22 15:27:46 +00:00
Sebastian Kunert 15e7db81d7 Strip 'Cargo.toml' from workspace path for patches (#1391) 2022-06-21 10:26:03 +02:00
Sebastian KunertandBastian Köcher 6a6490a699 Disable beefy for inprocess relay chain node (#1382)
* Disable beefy for inprocess relay chain node

* Update client/relay-chain-inprocess-interface/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-06-17 14:27:01 +00:00
Sebastian Kunert 2248d19163 Remove without_storage_info from pallet transaction-storage (#11668)
* Introduce BoundedVec

* Fix typos

* Add comments to the bounds

* Remove migration

* Improve bound value access syntax
2022-06-15 21:12:26 +00:00
Sebastian Kunert b1cc31027f Remove without_storage_info for membership pallet (#11591) 2022-06-07 19:49:36 +02:00
b32764ff99 Create binary for cumulus testing node #1128 (#1158)
* Create binary for cumulus testing node

* Fix call to wrong method

* Fix iterator bug

* Add real networking to polkadot node

* Allow using of file chain specs

* Implement export of wasm blob and genesis state

* Add parachain id to test service export

* Remove debug logs

* Fix main.rs

* Do not use cli from polkadot-collator

* Add tracing dependency

* Improve cli structure

* Use struct instead of enum for export commands

* Add missing license header

* Fix benchmark

* add build-spec subcommand (#19)

* add build-spec subcommand

* reorder args

* Fix formatting

* Add zombienet test (#26)

* add migrated tests

* rename test files

* uncomment args and remove extra collator node

* Fix indentation (#27)

* add migrated tests

* rename test files

* uncomment args and remove extra collator node

* fix identation

* Review comments

* Remove unwanted version changes

Co-authored-by: Sebastian Kunert <skunert@Sebastians-MacBook-Pro.fritz.box>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
2022-05-18 16:41:48 +02:00
Sebastian Kunert 20a7ffbfcc Bump jsonrpsee-core to 11.0 2022-04-22 11:46:19 +00:00
Sebastian Kunert e495168051 Support running relay chain full node as non-collator (#1069)
* Run relay chain full node not as collator

* Clean up
2022-03-04 17:02:37 +01:00
Sebastian Kunert 36f735e18b Improve conflicts to properly detect helper flags (#1063) 2022-03-02 12:13:33 +01:00
Sebastian Kunert 6cf8177dac Bump jsonrpsee from 0.8.0 to 0.9.0 2022-03-02 09:57:35 +00:00
Sebastian KunertandKoute bc532724b0 Introduce rpc client for relay chain full node (#963)
* Initial network interface preparations

* Implement get_storage_by_key

* Implement `validators` and `session_index_for_child`

* Implement persisted_validation_data and candidate_pending_availability

* Fix method name for persisted_validation_data and add encoded params

* Implement `retrieve_dmq_contents` and `retrieve_all_inbound_hrmp_channel_contents`

* Implement `prove_read`

* Introduce separate RPC client, expose JsonRpSee errors

* Simplify closure in call_remote_runtime_function

* Implement import stream, upgrade JsonRpSee

* Implement finality stream

* Remove unused method from interface

* Implement `is_major_syncing`

* Implement `wait_on_block`

* Fix tests

* Unify error handling `ApiError`

* Replace WaitError with RelayChainError

* Wrap BlockChainError in RelayChainError

* Unify error handling in relay chain intefaces

* Fix return type of proof method

* Improve error handling of new methods

* Improve error handling and move logging outside of interface

* Clean up

* Remove unwanted changes, clean up

* Remove unused import

* Add format for StatemachineError and remove nused From trait

* Use 'thiserror' crate to simplify error handling

* Expose error for overseer, further simplify error handling

* Reintroduce network interface

* Implement cli option

* Adjust call_state method to use hashes

* Disable PoV recovery when RPC is used

* Add integration test for network full node

* Use Hash instead of BlockId to ensure compatibility with RPC interface

* Fix cargo check warnings

* Implement retries

* Remove `expect` statements from code

* Update jsonrpsee to 0.8.0 and make collator keys optional

* Make cli arguments conflicting

* Remove unused `block_status` method

* Add clippy fixes

* Cargo fmt

* Validate relay chain rpc url

* Clean up dependencies and add one more integration test

* Clean up

* Clean up dependencies of relay-chain-network

* Use hash instead of blockid for rpc methods

* Fix tests

* Update client/cli/src/lib.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Improve error message of cli validation

* Add rpc client constructor

* Do not use debug formatting for errors

* Improve logging for remote runtime methods

* Only retry on transport problems

* Use PHash by value, rename test

* Improve tracing, return error  on relay-chain-interface build

* Fix naming, use generics instead of deserializing manually

* Rename RelayChainLocal and RelayChainNetwork

* lock

* Format

* Use impl trait for encodable runtime payload

* Only instantiate full node in tests when we need it

* Upgrade scale-codec to 3.0.0

* Improve expect log

Co-authored-by: Koute <koute@users.noreply.github.com>
2022-03-01 12:37:51 +01:00
Sebastian Kunert dacd0aed5e Unify RelayChainInterface error handling and introduce async (#909) 2022-01-25 18:27:54 +01:00
Sebastian Kunert 57d04e0fc8 Add missing features in rococo Cargo.toml (#887) 2022-01-05 13:30:47 +00:00
Sebastian Kunert 876e594f59 Introduce interface for relay chain interaction (#835) 2021-12-22 19:02:11 +01:00
Sebastian Kunert 2691e1928e Port transaction pool benchmark (#738)
Adds a benchmark to measure transaction throughput.
2021-11-11 10:30:54 +01:00