Commit Graph
25 Commits
Author SHA1 Message Date
Sebastian Kunert ebc73fee8e Do not verify on state import for relay chain verifier (#2140)
* Skip verification on state import

* Actually use existing variables

* Merge checks
2023-01-26 12:27:24 +00:00
Sebastian Kunert fb829ab873 Remove assumption that there is a local bootnode in RPC collator (#2091)
* Remove assumption that there is a local bootnode

* Bump zombienet to 1.3.29

* Separate "--"
2023-01-24 15:07:18 +01:00
Sebastian Kunert b66421fa4a Remove integration tests that are covered by zombienet (#1998) 2022-12-15 17:06:02 +01:00
Sebastian Kunert babf73bbc6 Allow to specify multiple relay chain RPC urls for collator node (#1880)
* Allow specification of multiple urls for relay chain rpc nodes

* Add pooled RPC client basics

* Add list of clients to pooled client

* Improve

* Forward requests to dispatcher

* Switch clients on error

* Implement rotation logic

* Improve subscription handling

* Error handling cleanup

* Remove retry from rpc-client

* Improve naming

* Improve documentation

* Improve `ClientManager` abstraction

* Adjust zombienet test

* Add more comments

* fmt

* Apply reviewers comments

* Extract reconnection to extra method

* Add comment to reconnection method

* Clean up some dependencies

* Fix build

* fmt

* Provide alias for cli argument

* Apply review comments

* Rename P* to Relay*

* Improve zombienet test

* fmt

* Fix zombienet sleep

* Simplify zombienet test

* Reduce log clutter and fix starting position

* Do not distribute duplicated imported and finalized blocks

* fmt

* Apply code review suggestions

* Move building of relay chain interface to `cumulus-client-service`

* Refactoring to not push back into channel

* FMT
2022-12-15 11:42:07 +01:00
Sebastian Kunert b9a0e759ca Remove more protocols from minimal collator (#1927) 2022-12-01 10:35:47 +01:00
Sebastian Kunert 5d82dc7d72 Remove av-store, av-distribution, chain-api subsystems from minimal node (#1903)
* Remove unused subsystems

* Consume requests we are not going to answer

* Respond to incoming chunk and pov requests.

* fmt

* Improve comment and logging

* Add reputation changes

* Do not register pov-chunk-request protocol

* Remove inbound chunk request channel

* Bump Polkadot
2022-11-27 14:35:51 +00:00
Sebastian KunertandBastian Köcher f68e124a96 Add example zombienet network file and instructions (#1839)
* Add simple example on how to spin up network.

* Update readme

* Remove unnecessary prefix

* Improve folder structure

* Add link to file

* Fix paths in readme

* Update README.md

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-11-08 18:26:20 +00:00
Sebastian Kunert ab02131008 Add RPC nodes to pov-recovery test (#1807) 2022-10-31 19:45:20 +01:00
Sebastian Kunert e960589cf9 Update clap to version 4 (#1745)
* Bump to clap4

* Adjust to clap 4 style

* Remove two more deprecated occurences of clap macro

* Remove "deprecated" feature from clap

* Update cargo lock

* Fix group name

* More skipped group names
2022-10-18 11:30:22 +00:00
Sebastian Kunert 7612d616e0 Enable collation via RPC relay chain node (#1585)
* Add minimal overseer gen with dummy subsystems

* Fix dependencies

* no-compile: only client transaction pool missing

* Remove unused imports

* Continue to hack towards PoC

* Continue

* Make mini node compile

* Compiling version with blockchainevents trait

* Continue

* Check in lockfile

* Block with tokio

* update patches

* Update polkadot patches

* Use polkadot-primitives v2

* Fix build problems

* First working version

* Adjust cargo.lock

* Add integration test

* Make integration test work

* Allow startinc collator without relay-chain args

* Make OverseerRuntimeClient async

* Create separate integration test

* Remove unused ChainSelection code

* Remove unused parameters on new-mini

* Connect collator node in test to relay chain nodes

* Make BlockChainRPCClient obsolete

* Clean up

* Clean up

* Reimplement blockchain-rpc-events

* Revert "Allow startinc collator without relay-chain args"

This reverts commit f22c70e16521f375fe125df5616d48ceea926b1a.

* Add `strict_record_validation` to AuthorityDiscovery

* Move network to cumulus

* Remove BlockchainRPCEvents

* Remove `BlockIdTo` and `BlockchainEvents`

* Make AuthorityDiscovery async

* Use hash in OverseerRuntime

* Adjust naming of runtime client trait

* Implement more rpc-client methods

* Improve error handling for `ApiError`

* Extract authority-discovery creationand cleanup

* RPC -> Rpc

* Extract bitswap

* Adjust to changes on master

* Implement `hash` method

* Introduce DummyChainSync, remove ProofProvider and BlockBackend

* Remove `HeaderMetadata` from blockchain-rpc-client

* Make ChainSync work

* Implement NetworkHeaderBackend

* Cleanup

* Adjustments after master merge

* Remove ImportQueue from network parameters

* Remove cargo patches

* Eliminate warnings

* Revert to HeaderBackend

* Add zombienet test

* Implement `status()` method

* Add more comments, improve readability

* Remove patches from Cargo.toml

* Remove integration test in favor of zombienet

* Remove unused dependencies, rename minimal node crate

* Adjust to latest master changes

* fmt

* Execute zombienet test on gitlab ci

* Reuse network metrics

* Chainsync metrics

* fmt

* Feed RPC node as boot node to the relay chain minimal node

* fmt

* Add bootnodes to zombienet collators

* Allow specification of relay chain args

* Apply review suggestions

* Remove unnecessary casts

* Enable PoV recovery for rpc full nodes

* Revert unwanted changes

* Make overseerHandle non-optional

* Add availability-store subsystem

* Add AuxStore and ChainApiSubsystem

* Add availability distribution subsystem

* Improve pov-recovery logging and add RPC nodes to tests

* fmt

* Make availability config const

* lock

* Enable debug logs for pov-recovery in zombienet

* Add log filters to test binary

* Allow wss

* Address review comments

* Apply reviewer comments

* Adjust to master changes

* Apply reviewer suggestions

* Bump polkadot

* Add builder method for minimal node

* Bump substrate and polkadot

* Clean up overseer building

* Add bootnode to two in pov_recovery test

* Fix missing quote in pov recovery zombienet test

* Improve zombienet pov test

* More debug logs for pov-recovery

* Remove reserved nodes like on original test

* Revert zombienet test to master
2022-10-10 09:06:26 +02:00
Sebastian KunertandDavide Galassi b6a2a38c91 Introduce RPC Worker (#1462)
* Extract json-rpc-client and introduce worker

* Initial rpc worker

* Add error handling

* Use bounded channels for listeners

* Improve naming and clean up

* Use tracing channels

* Improve code readability

* Decrease channel size limit

* Remove unused dependency

* Fix docs

* RPC -> Rpc

* Start worker in initialization method

* Print error in case a distribution channel is full

* Fix docs

* Make `RpcStreamWorker` private

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Use tokio channels and add TODO item

* Remove `Option` from `to_worker_channel`

Co-authored-by: Davide Galassi <davxy@datawok.net>
2022-08-01 12:08:57 +02:00
Sebastian Kunertandparity-processbot 79d663b4e4 Companion for polkadot#5782 (#1447)
* Make HeadSupportsParachain async

* Update Cargo.lock

* update lockfile for {"substrate", "polkadot"}

Co-authored-by: parity-processbot <>
2022-07-20 11:03:02 +00:00
Sebastian Kunert 0a069ce7f3 Rename help string parachain-collator -> parachain-template-node (#1392) 2022-06-23 11:36:33 +02:00
Sebastian Kunert ebbf6b6e21 Strip 'Cargo.toml' from workspace path for patches (#1391) 2022-06-21 10:26:03 +02:00
Sebastian KunertandBastian Köcher da3ae61939 Disable beefy for inprocess relay chain node (#1382)
* Disable beefy for inprocess relay chain node

* Update client/relay-chain-inprocess-interface/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-06-17 14:27:01 +00:00
b1a5d70256 Create binary for cumulus testing node #1128 (#1158)
* Create binary for cumulus testing node

* Fix call to wrong method

* Fix iterator bug

* Add real networking to polkadot node

* Allow using of file chain specs

* Implement export of wasm blob and genesis state

* Add parachain id to test service export

* Remove debug logs

* Fix main.rs

* Do not use cli from polkadot-collator

* Add tracing dependency

* Improve cli structure

* Use struct instead of enum for export commands

* Add missing license header

* Fix benchmark

* add build-spec subcommand (#19)

* add build-spec subcommand

* reorder args

* Fix formatting

* Add zombienet test (#26)

* add migrated tests

* rename test files

* uncomment args and remove extra collator node

* Fix indentation (#27)

* add migrated tests

* rename test files

* uncomment args and remove extra collator node

* fix identation

* Review comments

* Remove unwanted version changes

Co-authored-by: Sebastian Kunert <skunert@Sebastians-MacBook-Pro.fritz.box>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
2022-05-18 16:41:48 +02:00
Sebastian Kunert 3f9a57dedb Bump jsonrpsee-core to 11.0 2022-04-22 11:46:19 +00:00
Sebastian Kunert 66bb76dc97 Support running relay chain full node as non-collator (#1069)
* Run relay chain full node not as collator

* Clean up
2022-03-04 17:02:37 +01:00
Sebastian Kunert e42a7c3fff Improve conflicts to properly detect helper flags (#1063) 2022-03-02 12:13:33 +01:00
Sebastian Kunert 0546af470c Bump jsonrpsee from 0.8.0 to 0.9.0 2022-03-02 09:57:35 +00:00
Sebastian KunertandKoute 234c42c2df Introduce rpc client for relay chain full node (#963)
* Initial network interface preparations

* Implement get_storage_by_key

* Implement `validators` and `session_index_for_child`

* Implement persisted_validation_data and candidate_pending_availability

* Fix method name for persisted_validation_data and add encoded params

* Implement `retrieve_dmq_contents` and `retrieve_all_inbound_hrmp_channel_contents`

* Implement `prove_read`

* Introduce separate RPC client, expose JsonRpSee errors

* Simplify closure in call_remote_runtime_function

* Implement import stream, upgrade JsonRpSee

* Implement finality stream

* Remove unused method from interface

* Implement `is_major_syncing`

* Implement `wait_on_block`

* Fix tests

* Unify error handling `ApiError`

* Replace WaitError with RelayChainError

* Wrap BlockChainError in RelayChainError

* Unify error handling in relay chain intefaces

* Fix return type of proof method

* Improve error handling of new methods

* Improve error handling and move logging outside of interface

* Clean up

* Remove unwanted changes, clean up

* Remove unused import

* Add format for StatemachineError and remove nused From trait

* Use 'thiserror' crate to simplify error handling

* Expose error for overseer, further simplify error handling

* Reintroduce network interface

* Implement cli option

* Adjust call_state method to use hashes

* Disable PoV recovery when RPC is used

* Add integration test for network full node

* Use Hash instead of BlockId to ensure compatibility with RPC interface

* Fix cargo check warnings

* Implement retries

* Remove `expect` statements from code

* Update jsonrpsee to 0.8.0 and make collator keys optional

* Make cli arguments conflicting

* Remove unused `block_status` method

* Add clippy fixes

* Cargo fmt

* Validate relay chain rpc url

* Clean up dependencies and add one more integration test

* Clean up

* Clean up dependencies of relay-chain-network

* Use hash instead of blockid for rpc methods

* Fix tests

* Update client/cli/src/lib.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Improve error message of cli validation

* Add rpc client constructor

* Do not use debug formatting for errors

* Improve logging for remote runtime methods

* Only retry on transport problems

* Use PHash by value, rename test

* Improve tracing, return error  on relay-chain-interface build

* Fix naming, use generics instead of deserializing manually

* Rename RelayChainLocal and RelayChainNetwork

* lock

* Format

* Use impl trait for encodable runtime payload

* Only instantiate full node in tests when we need it

* Upgrade scale-codec to 3.0.0

* Improve expect log

Co-authored-by: Koute <koute@users.noreply.github.com>
2022-03-01 12:37:51 +01:00
Sebastian Kunert a9630551c2 Unify RelayChainInterface error handling and introduce async (#909) 2022-01-25 18:27:54 +01:00
Sebastian Kunert e9e775f9e7 Add missing features in rococo Cargo.toml (#887) 2022-01-05 13:30:47 +00:00
Sebastian Kunert 7b42df10f3 Introduce interface for relay chain interaction (#835) 2021-12-22 19:02:11 +01:00
Sebastian Kunert d4045c85e7 Port transaction pool benchmark (#738)
Adds a benchmark to measure transaction throughput.
2021-11-11 10:30:54 +01:00