Commit Graph

21 Commits

Author SHA1 Message Date
Sebastian Kunert c93f5aba8a Cumulus test service cleanup (#2887)
closes #2567 

Followup for https://github.com/paritytech/polkadot-sdk/pull/2331

This PR contains multiple internal cleanups:

1. This gets rid of the functionality in `generate_genesis_block` which
was only used in one benchmark
2. Fixed `transaction_pool` and `transaction_throughput` benchmarks
failing since they require a tokio runtime now.
3. Removed `parachain_id` CLI option from the test parachain
4. Removed `expect` call from `RuntimeResolver`
2024-01-11 16:44:03 +00:00
Joshy Orndorff 8683bbeefb Rename ExportGenesisStateCommand to ExportGenesisHeadCommand and make it respect custom genesis block builders (#2331)
Closes #2326.

This PR both fixes a logic bug and replaces an incorrect name.

## Bug Fix: Respecting custom genesis builder

Prior to this PR the standard logic for creating a genesis block was
repeated inside of cumulus. This PR removes that duplicated logic, and
calls into the proper `BuildGenesisBlock` implementation.

One consequence is that if the genesis block has already been
initialized, it will not be re-created, but rather read from the
database like it is for other node invocations. So you need to watch out
for old unpurged data during the development process. Offchain tools may
need to be updated accordingly. I've already filed
https://github.com/paritytech/zombienet/issues/1519

## Rename: It doesn't export state. It exports head data.

The name export-genesis-state was always wrong, nad it's never too late
to right a wrong. I've changed the name of the struct to
`ExportGenesisHeadCommand`.

There is still the question of what to do with individual nodes' public
CLIs. I have updated the parachain template to a reasonable default that
preserves compatibility with tools that will expect
`export-genesis-state` to still work. And I've chosen not to modify the
public CLIs of any other nodes in the repo. I'll leave it up to their
individual owners/maintains to decide whether that is appropriate.

---------

Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Bastian Köcher <info@kchr.de>
2023-12-13 11:18:33 +02:00
Michal Kucharczyk 8ba7a6aba8 chain-spec: getting ready for native-runtime-free world (#1256)
This PR prepares chains specs for _native-runtime-free_  world.

This PR has following changes:
- `substrate`:
  - adds support for:
- JSON based `GenesisConfig` to `ChainSpec` allowing interaction with
runtime `GenesisBuilder` API.
- interacting with arbitrary runtime wasm blob to[
`chain-spec-builder`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/bin/utils/chain-spec-builder/src/lib.rs#L46)
command line util,
- removes
[`code`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/frame/system/src/lib.rs#L660)
from `system_pallet`
  - adds `code` to the `ChainSpec`
- deprecates
[`ChainSpec::from_genesis`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/client/chain-spec/src/chain_spec.rs#L263),
but also changes the signature of this method extending it with `code`
argument.
[`ChainSpec::builder()`](https://github.com/paritytech/substrate/blob/20bee680ed098be7239cf7a6b804cd4de267983e/client/chain-spec/src/chain_spec.rs#L507)
should be used instead.
- `polkadot`:
- all references to `RuntimeGenesisConfig` in `node/service` are
removed,
- all
`(kusama|polkadot|versi|rococo|wococo)_(staging|dev)_genesis_config`
functions now return the JSON patch for default runtime `GenesisConfig`,
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,

- `cumulus`:
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,
- _JSON_ patch configuration used instead of `RuntimeGenesisConfig
struct` in all chain specs.
  
---------

Co-authored-by: command-bot <>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Kevin Krone <kevin@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-11-05 15:19:23 +01:00
Sebastian Kunert 9485b0b492 Point documentation links to monorepo (#1741) 2023-09-29 22:33:26 +02:00
Przemek Rzad bfb241d7f3 Add missing licenses and tune the scanning workflow (#1288)
* Add missing Cumulus licenses

* Typo

* Add missing Substrate licenses

* Single job checking the sub-repos in steps

* Remove dates

* Remove dates

* Add missing (C)

* Update FRAME UI tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update more UI tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-08-30 15:45:49 +03:00
Adrian Catangiu d841bdca18 Revert "companion for 14754: cli: move no-beefy flag to sc-cli (#2996)" (#3007)
* Revert "companion for 14754: cli: move no-beefy flag to sc-cli (#2996)"

This reverts commit ab840db20d.

* bump substrate ref

* update lockfile for {"polkadot", "substrate"}

---------

Co-authored-by: parity-processbot <>
2023-08-15 16:40:54 +00:00
Adrian Catangiu ab840db20d companion for 14754: cli: move no-beefy flag to sc-cli (#2996)
* companion 14754: no-beefy flag moved to sc-cli

Signed-off-by: Adrian Catangiu <adrian@parity.io>

* bump substrate ref

Signed-off-by: Adrian Catangiu <adrian@parity.io>

* bump polkadot ref

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
2023-08-11 16:11:24 +00:00
Bastian Köcher fdd2e5221e Companion for Substrate#14511 (#2821)
* Companion for Substrate#14511

https://github.com/paritytech/substrate/pull/14511

* update lockfile for {"substrate", "polkadot"}

---------

Co-authored-by: parity-processbot <>
2023-07-05 08:39:48 +00:00
Arkadiy Paronyan 09a0538bc5 Companion for #13701 (#2423)
* Set data_path

* fmt

* Updated substrate and polkadot
2023-05-04 11:34:43 +00:00
Niklas Adolfsson cb0f623d6f companion for #13384 (#2417)
* companion for #13384

* update rpc cli

* fix missed stuff

* update parachain rpc commands

* update polkadot and substrate deps

* update substrate & polkadot

---------

Co-authored-by: parity-processbot <>
Co-authored-by: muharem <ismailov.m.h@gmail.com>
2023-05-04 10:40:11 +00:00
Sebastian Kunert 588bdad7f6 Add retry mechanism for pov-recovery, fix full-node pov-recovery (#2164)
* Increase delay for pov-recovery

* Update client/service/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Comment

* FMT

* Clear waiting_recovery when block is recovered or recovery failed

* Introduce recovery queue that preserved insertion order

* Better error logs

* Decrease slot duration

* Style improvements

* Add option to use unordered queue

* Maintain cache of finalized blocks

* Wait for one relay chain slot before recovery

* Make retries testable

* fmt

* Improve docs

* Improve docs

* Simplify RecoveryQueue

* Remove unwanted changes

* Adjust to comments

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Move recovery delay into the queue

* Check for finalized number

* Clean up

* Use timer

Co-authored-by: Bastian Köcher <git@kchr.de>

* Simplify implementation

* Revert "Use timer"

This reverts commit 3809eed840d3a09d54212f99486782ff80cdc1c9.

* Properly clear `to_recover` flag

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-02-09 14:18:55 +01:00
Sebastian Kunert 975403e802 Update clap to version 4 (#1745)
* Bump to clap4

* Adjust to clap 4 style

* Remove two more deprecated occurences of clap macro

* Remove "deprecated" feature from clap

* Update cargo lock

* Fix group name

* More skipped group names
2022-10-18 11:30:22 +00:00
Sebastian Kunert cdcf5ddd9b Enable collation via RPC relay chain node (#1585)
* Add minimal overseer gen with dummy subsystems

* Fix dependencies

* no-compile: only client transaction pool missing

* Remove unused imports

* Continue to hack towards PoC

* Continue

* Make mini node compile

* Compiling version with blockchainevents trait

* Continue

* Check in lockfile

* Block with tokio

* update patches

* Update polkadot patches

* Use polkadot-primitives v2

* Fix build problems

* First working version

* Adjust cargo.lock

* Add integration test

* Make integration test work

* Allow startinc collator without relay-chain args

* Make OverseerRuntimeClient async

* Create separate integration test

* Remove unused ChainSelection code

* Remove unused parameters on new-mini

* Connect collator node in test to relay chain nodes

* Make BlockChainRPCClient obsolete

* Clean up

* Clean up

* Reimplement blockchain-rpc-events

* Revert "Allow startinc collator without relay-chain args"

This reverts commit f22c70e16521f375fe125df5616d48ceea926b1a.

* Add `strict_record_validation` to AuthorityDiscovery

* Move network to cumulus

* Remove BlockchainRPCEvents

* Remove `BlockIdTo` and `BlockchainEvents`

* Make AuthorityDiscovery async

* Use hash in OverseerRuntime

* Adjust naming of runtime client trait

* Implement more rpc-client methods

* Improve error handling for `ApiError`

* Extract authority-discovery creationand cleanup

* RPC -> Rpc

* Extract bitswap

* Adjust to changes on master

* Implement `hash` method

* Introduce DummyChainSync, remove ProofProvider and BlockBackend

* Remove `HeaderMetadata` from blockchain-rpc-client

* Make ChainSync work

* Implement NetworkHeaderBackend

* Cleanup

* Adjustments after master merge

* Remove ImportQueue from network parameters

* Remove cargo patches

* Eliminate warnings

* Revert to HeaderBackend

* Add zombienet test

* Implement `status()` method

* Add more comments, improve readability

* Remove patches from Cargo.toml

* Remove integration test in favor of zombienet

* Remove unused dependencies, rename minimal node crate

* Adjust to latest master changes

* fmt

* Execute zombienet test on gitlab ci

* Reuse network metrics

* Chainsync metrics

* fmt

* Feed RPC node as boot node to the relay chain minimal node

* fmt

* Add bootnodes to zombienet collators

* Allow specification of relay chain args

* Apply review suggestions

* Remove unnecessary casts

* Enable PoV recovery for rpc full nodes

* Revert unwanted changes

* Make overseerHandle non-optional

* Add availability-store subsystem

* Add AuxStore and ChainApiSubsystem

* Add availability distribution subsystem

* Improve pov-recovery logging and add RPC nodes to tests

* fmt

* Make availability config const

* lock

* Enable debug logs for pov-recovery in zombienet

* Add log filters to test binary

* Allow wss

* Address review comments

* Apply reviewer comments

* Adjust to master changes

* Apply reviewer suggestions

* Bump polkadot

* Add builder method for minimal node

* Bump substrate and polkadot

* Clean up overseer building

* Add bootnode to two in pov_recovery test

* Fix missing quote in pov recovery zombienet test

* Improve zombienet pov test

* More debug logs for pov-recovery

* Remove reserved nodes like on original test

* Revert zombienet test to master
2022-10-10 09:06:26 +02:00
Bastian Köcher 4257576828 Companion for https://github.com/paritytech/substrate/pull/12147 (#1587)
* Companion for https://github.com/paritytech/substrate/pull/12147

* update lockfile for {"substrate", "polkadot"}

* Fix tests

Co-authored-by: parity-processbot <>
2022-09-02 17:33:17 +00:00
Bastian Köcher baa1c3600c Companion for new trie cache (#1361)
* Update branch

* Make it compile

* Update Substrate

* Revert "Update branch"

This reverts commit 7b0278160a09312fc6501e0a31923c5a8d8f8c56.

* FMT

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>
2022-08-18 22:02:08 +00:00
Javier Viola 807db6ff0d Zombienet add tests (#1321)
* changes to read json spec in test binary

* add zombienet tests

* fmt

* use {{COL_IMAGE}} and clean config

* add comment and use relay image from env

* use test-parachain image from pr

* fix warns

* fix warns

* fmt

* typo

* fix ci to use zombienet image

* fix spawn nodes for test

* reorg test

* add within to test

* remove check for full node collators is up

* add tests for pov, mirate solo to para, sync blocks

* bump zombienet image

* add job dep with artifacts

* add sleep for test

* fix after merge

* fmt

* bump zombienet version

* changes from clap

* use base/shared params

* fmt

* debug ci

* add upgrade test

* update js test for debug

* less debug in test

* print assertion

* fix upgrade test

* Collator key only needed if we run as collator

* [Fix] Benchmark build artifact folder creation (#1518)

* Trivial networking changes for Substrate PR #11940 (#1486)

* Trivial networking changes for Substrate PR https://github.com/paritytech/substrate/pull/11940

* Apply formatting rules

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>

* bump zombienet version

* update network def for test

* typo

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
2022-08-18 17:59:36 +02:00
Xiliang Chen 71df7f0148 Companion for substrate#11786 (#1436)
* bump deps

* companion substrate#11786

* cargo fmt

* update substrate

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2022-07-12 12:58:01 +00:00
Branislav Kontur c623784f13 Typo + some clippy stuff (unnecessary clone/to_string/refs) + fmt (#1437) 2022-07-11 11:05:40 +00:00
Qinxuan Chen ad6fe6d191 Move ExportGenesisStateCommand and ExportGenesisWasmCommand to cumulus-client-cli (#1325)
* Move ExportGenesisStateCommand and ExportGenesisWasmCommand to cumulus-client-cli

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Remove useless

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Use shared_params.chain

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2022-06-18 15:33:52 +02:00
dependabot[bot] 48b8b1734c Bump clap from 3.1.18 to 3.2.2 (#1370)
* Bump clap from 3.1.18 to 3.2.2

Bumps [clap](https://github.com/clap-rs/clap) from 3.1.18 to 3.2.2.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.18...v3.2.2)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump clap to 3.2.5 and fix warnings

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2022-06-17 15:05:44 +00:00
Sebastian Kunert b32764ff99 Create binary for cumulus testing node #1128 (#1158)
* Create binary for cumulus testing node

* Fix call to wrong method

* Fix iterator bug

* Add real networking to polkadot node

* Allow using of file chain specs

* Implement export of wasm blob and genesis state

* Add parachain id to test service export

* Remove debug logs

* Fix main.rs

* Do not use cli from polkadot-collator

* Add tracing dependency

* Improve cli structure

* Use struct instead of enum for export commands

* Add missing license header

* Fix benchmark

* add build-spec subcommand (#19)

* add build-spec subcommand

* reorder args

* Fix formatting

* Add zombienet test (#26)

* add migrated tests

* rename test files

* uncomment args and remove extra collator node

* Fix indentation (#27)

* add migrated tests

* rename test files

* uncomment args and remove extra collator node

* fix identation

* Review comments

* Remove unwanted version changes

Co-authored-by: Sebastian Kunert <skunert@Sebastians-MacBook-Pro.fritz.box>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
2022-05-18 16:41:48 +02:00