df818d2974
Move cumulus zombienet tests to aura & async backing ( #3568 )
...
Cumulus test-parachain node and test runtime were still using relay
chain consensus and 12s blocktimes. With async backing around the corner
on the major chains we should switch our tests too.
Also needed to nicely test the changes coming to collators in #3168 .
### Changes Overview
- Followed the [migration
guide](https://wiki.polkadot.network/docs/maintain-guides-async-backing )
for async backing for the cumulus-test-runtime
- Adjusted the cumulus-test-service to use the correct import-queue,
lookahead collator etc.
- The block validation function now uses the Aura Ext Executor so that
the seal of the block is validated
- Previous point requires that we seal block before calling into
`validate_block`, I introduced a helper function for that
- Test client adjusted to provide a slot to the relay chain proof and
the aura pre-digest
2024-04-09 16:53:30 +00:00
a035dc9be7
Remove AssignmentProviderConfig and use parameters from HostConfiguration instead ( #3181 )
...
This PR removes `AssignmentProviderConfig` and uses the corresponding
ondemand parameters from `HostConfiguration` instead. Additionally
`scheduling_lookahead` and all coretime/ondemand related parameters are
extracted in a separate struct - `SchedulerParams`.
The most relevant commit from the PR is [this
one](https://github.com/paritytech/polkadot-sdk/pull/3181/commits/830bc0f5e858944474171bbe33382ad96040b535 ).
Fixes https://github.com/paritytech/polkadot-sdk/issues/2268
---------
Co-authored-by: command-bot <>
2024-02-29 07:12:02 +00:00
f8b03d9564
Stabilize pov-recovery zombienet ( #2611 )
...
Smoldot sometimes stops reporting finalized blocks to us. Since we are
recovering from the relay chain with a huge delay on full nodes, we can
not rely on import notifications to set the best block. Because
sometimes we also do not receive finality notifications, the height
check in zombienet fails.
Proper solution is to update smoldot, there have been some changes since
the version we use. But upgrade is blocked by version conflict which
will be resolved with
https://github.com/paritytech/polkadot-sdk/pull/1631 .
2023-12-05 09:02:49 +01:00
ec3a61ed86
Remove pov-recovery race condition/Improve zombienet test ( #2526 )
...
The test was a bit flaky on CI.
There was a race condition in the pov-recovery system. If the timing is
bad, it can happen that a block waits for a parent that is already
queued for import. The check if a block has children waiting happens
when we insert into the import queue. So we need to do an additional
check once we receive the import notification for the parent block.
Second issue is that `alice` was missing `--in-peers 0` and `--out-peers
0`, so alice was sometimes still fetching block via sync and the
assertion on the logs in zombienet would fail.
There is another potential issue that I saw once locally. We have a
failing pov-recovery queue that fails from time to time to check that
the retry mechanism does what it should. We now make sure that the same
candidate is never failed twice, so the tests become more predictable.
2023-11-28 16:59:06 +01:00
471eafcb8d
Disable any peer connections for parachain nodes in pov-recovery zombienet test ( #2475 )
...
I noticed that this test broke at some point. The parachain nodes should
only acquire their blocks from the relay chain. But they were connecting
to their peers and started fetching blocks from there.
In this test I now take additional measures so we check that each nodes
really uses pov-recovery to get the blocks.
2023-11-24 13:52:55 +01:00
8ba7a6aba8
chain-spec: getting ready for native-runtime-free world (#1256 )
...
This PR prepares chains specs for _native-runtime-free_ world.
This PR has following changes:
- `substrate`:
- adds support for:
- JSON based `GenesisConfig` to `ChainSpec` allowing interaction with
runtime `GenesisBuilder` API.
- interacting with arbitrary runtime wasm blob to[
`chain-spec-builder`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/bin/utils/chain-spec-builder/src/lib.rs#L46 )
command line util,
- removes
[`code`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/frame/system/src/lib.rs#L660 )
from `system_pallet`
- adds `code` to the `ChainSpec`
- deprecates
[`ChainSpec::from_genesis`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/client/chain-spec/src/chain_spec.rs#L263 ),
but also changes the signature of this method extending it with `code`
argument.
[`ChainSpec::builder()`](https://github.com/paritytech/substrate/blob/20bee680ed098be7239cf7a6b804cd4de267983e/client/chain-spec/src/chain_spec.rs#L507 )
should be used instead.
- `polkadot`:
- all references to `RuntimeGenesisConfig` in `node/service` are
removed,
- all
`(kusama|polkadot|versi|rococo|wococo)_(staging|dev)_genesis_config`
functions now return the JSON patch for default runtime `GenesisConfig`,
- `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,
- `cumulus`:
- `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,
- _JSON_ patch configuration used instead of `RuntimeGenesisConfig
struct` in all chain specs.
---------
Co-authored-by: command-bot <>
Co-authored-by: Javier Viola <javier@parity.io >
Co-authored-by: Davide Galassi <davxy@datawok.net >
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com >
Co-authored-by: Kevin Krone <kevin@parity.io >
Co-authored-by: Bastian Köcher <git@kchr.de >
2023-11-05 15:19:23 +01:00
1c0b437330
cumulus test runtime: remove GenesisExt ( #2147 )
...
This PR removes the `GenesisExt` wrapper over the `GenesisRuntimeConfig`
in `cumulus-test-service`. Initialization of values that were performed
by `GenesisExt::BuildStorage` was moved into `test_pallet` genesis.
---------
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de >
2023-11-04 11:25:07 +02:00
a30092ab42
Markdown linter ( #1309 )
...
* Add markdown linting
- add linter default rules
- adapt rules to current code
- fix the code for linting to pass
- add CI check
fix #1243
* Fix markdown for Substrate
* Fix tooling install
* Fix workflow
* Add documentation
* Remove trailing spaces
* Update .github/.markdownlint.yaml
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix mangled markdown/lists
* Fix captalization issues on known words
2023-09-04 12:02:32 +03:00
2c3e869c38
substrate: chain-spec paths corrected in zombienet tests ( #1362 )
...
* substrate: chain-spec paths corrected in zombienet tests
* fix chain-spec path in cumulus test
* disable beefy on validator
---------
Co-authored-by: Javier Viola <javier@parity.io >
2023-09-02 21:25:43 +02:00
af432f0fc3
fix chain-spec path for substrate tests ( #1307 )
...
* fix chain-spec path for substrate tests
* update chain-spec path for cumulus test
2023-08-30 14:38:03 -03:00
817c4503db
Allow integrated relay chain light client ( #2270 )
...
* Add embedded light client to cli
* Prepare for light-client-worker
* First working version
* Clean up
* Remove unwanted logs
* Simplify subscription code
* Let jsonrpsee handle rpc management
* Simplify implementation
* Reorganize crate structure
* Use relay chain arg chainspec for light-client
* Clean up command line
* Add light client worker file
* Use smoldot master to avoid wasmtime conflict
* Remove sleep
* Improve naming of cli option
* Remove conflict with `validator`
* Improve docs
* Update smoldot, remove unwanted change
* Apply suggestions from code review
Co-authored-by: Dmitry Markin <dmitry@markin.tech >
* Disable collation
* Reviewer comments
* Update smoldot and tokio-platform
* Update smoldot
* Update smoldot
* Adjust to new version
* Patch substrate
* Use constants
* Add readme entry, improve zombienet tests
* Apply suggestions from code review
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com >
* Make execution mode an enum
* Update smoldot, remove substrate patch
* Update client/relay-chain-rpc-interface/src/rpc_client.rs
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com >
* Reduce duplicate code
* Update smoldot
* Update smoldot
* Fix build
* Update smoldot
* Make platform compile
* Clean up dependencies
* Use crates.io instead of github for smoldot
* Apply suggestions from code review
Co-authored-by: Davide Galassi <davxy@datawok.net >
* Docs
* Improve docs
* Remove `RpcFrontend`
---------
Co-authored-by: Dmitry Markin <dmitry@markin.tech >
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com >
Co-authored-by: Davide Galassi <davxy@datawok.net >
2023-08-22 15:53:32 +02:00
ee3383eb44
bump zombienet version ( #2525 )
...
* bump zombienet version
* fix para registration args
2023-05-05 22:53:16 +00:00
5c9cd167df
bump zombienet version to v1.3.35 ( #2226 )
...
* update zombienet version
* remove 'is up' assertions
2023-02-23 13:49:18 -03:00
1d9af70c73
Rename .feature extension to .zndsl ( #2215 )
...
* Rename .feature extension to .zndsl
* Rename .feature to .zndsl into scripts
2023-02-20 23:20:44 +01:00
f2dc402cec
add warp_sync_params ( #1909 )
...
* wait for relay chain to sync then get parachain header
* Spawn new thread to wait for the target block
* second round of comments from the PR on substrate
* third round of pr comments
* add zombienet tests
* rebase issues
* refactor tests based on pr comments
* rebase issues
* pr comments
* passing zombienet test
* cargo +nightly fmt
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* use cargo lock from master
* pr comments
* cargo fmt
* use finalised block instead of best block
* use import notification stream
* rebase changes
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/relay-chain-interface/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/relay-chain-interface/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* pr comments
* use new file names
* db snaphots moved to google cloud storage
* Update client/network/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update client/service/src/lib.rs
Co-authored-by: Sebastian Kunert <skunert49@gmail.com >
* pr comments
* Update zombienet/tests/0007-full_node_warp_sync.toml
Co-authored-by: Sebastian Kunert <skunert49@gmail.com >
* Update zombienet/tests/0007-full_node_warp_sync.toml
Co-authored-by: Sebastian Kunert <skunert49@gmail.com >
* Scenario 1
Parachain node and in-node relay chain both start with --sync warp. This ensures that the waiting logic works as expected.
Scenario 2
Parachain node starts with warp sync, relay chain points to a node already synced up
scenario 3
Parachain node starts with warp sync, relay chain points to a node that uses warp sync
* Use test-parachain
* use test-parachain chainspecs
* remove relay chain spec as it is no longer required
* add back relaychain spec file
* pr comments
* Upload snapshots to google cloud
* Update zombienet/tests/0007-prepare-warp-sync-db-snapshot.md
Co-authored-by: Sebastian Kunert <skunert49@gmail.com >
* update documentation
* Fix snapshot URLs
* use master lock file
* add finalized_block_hash
* Patch diener for CI
* Bump Zombienet
* Add 0007 zombienet test
* Bump zombienet
* Revert "Patch diener for CI"
This reverts commit 9ece6c9fc9b17058b61cd7e9dee29d3a9af87841.
* merge fixes
* use master lock file
* Update Substrate & Polkadot
---------
Co-authored-by: Bastian Köcher <git@kchr.de >
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com >
Co-authored-by: Sebastian Kunert <skunert49@gmail.com >
Co-authored-by: Bastian Köcher <info@kchr.de >
2023-02-14 19:15:43 +00:00
588bdad7f6
Add retry mechanism for pov-recovery, fix full-node pov-recovery ( #2164 )
...
* Increase delay for pov-recovery
* Update client/service/src/lib.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Comment
* FMT
* Clear waiting_recovery when block is recovered or recovery failed
* Introduce recovery queue that preserved insertion order
* Better error logs
* Decrease slot duration
* Style improvements
* Add option to use unordered queue
* Maintain cache of finalized blocks
* Wait for one relay chain slot before recovery
* Make retries testable
* fmt
* Improve docs
* Improve docs
* Simplify RecoveryQueue
* Remove unwanted changes
* Adjust to comments
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de >
* Move recovery delay into the queue
* Check for finalized number
* Clean up
* Use timer
Co-authored-by: Bastian Köcher <git@kchr.de >
* Simplify implementation
* Revert "Use timer"
This reverts commit 3809eed840d3a09d54212f99486782ff80cdc1c9.
* Properly clear `to_recover` flag
---------
Co-authored-by: Bastian Köcher <git@kchr.de >
2023-02-09 14:18:55 +01:00
ad453c556b
Remove assumption that there is a local bootnode in RPC collator ( #2091 )
...
* Remove assumption that there is a local bootnode
* Bump zombienet to 1.3.29
* Separate "--"
2023-01-24 15:07:18 +01:00
7cab12e9d2
Allow to specify multiple relay chain RPC urls for collator node ( #1880 )
...
* Allow specification of multiple urls for relay chain rpc nodes
* Add pooled RPC client basics
* Add list of clients to pooled client
* Improve
* Forward requests to dispatcher
* Switch clients on error
* Implement rotation logic
* Improve subscription handling
* Error handling cleanup
* Remove retry from rpc-client
* Improve naming
* Improve documentation
* Improve `ClientManager` abstraction
* Adjust zombienet test
* Add more comments
* fmt
* Apply reviewers comments
* Extract reconnection to extra method
* Add comment to reconnection method
* Clean up some dependencies
* Fix build
* fmt
* Provide alias for cli argument
* Apply review comments
* Rename P* to Relay*
* Improve zombienet test
* fmt
* Fix zombienet sleep
* Simplify zombienet test
* Reduce log clutter and fix starting position
* Do not distribute duplicated imported and finalized blocks
* fmt
* Apply code review suggestions
* Move building of relay chain interface to `cumulus-client-service`
* Refactoring to not push back into channel
* FMT
2022-12-15 11:42:07 +01:00
9a3e518e5c
Add example zombienet network file and instructions ( #1839 )
...
* Add simple example on how to spin up network.
* Update readme
* Remove unnecessary prefix
* Improve folder structure
* Add link to file
* Fix paths in readme
* Update README.md
Co-authored-by: Bastian Köcher <git@kchr.de >
Co-authored-by: Bastian Köcher <git@kchr.de >
2022-11-08 18:26:20 +00:00
Sebastian Kunert