* runtime: add BEEFY and MMR to Westend
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* runtime: add BEEFY and MMR to Kusama
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* node/service: enable BEEFY for Westend and Kusama
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* node/service: regenerate genesis keys for westend-native and kusama-native
Since these keys are only used for development/local chains, also publish
the secret seeds used to generate the public keys, so that developers can
recover/generate the private key pairs if needed.
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* runtime: add session keys migration to add BEEFY to Westend and Kusama
* runtime: fix migration
* fix try-runtime build
* cargo fmt
* fix parachains slashing benchmark
* address review comments
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <git@kchr.de>
* runtime: fix session keys migration
---------
Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
* remove try-runtime-cli
* fix ci pipeline
* fix link
* remove chain var
* build runtime with try-runtime feature
* use main branch
* pin to commit
* fix build
* update lockfile for {"substrate"}
---------
Co-authored-by: parity-processbot <>
* Remove superflous parameter `overseer_enable_anyways`
We don't need this flag, as we don't need the overseer enabled when the
node isn't a collator or validator.
* Rename `IsCollator` to `IsParachainNode`
`IsParachainNode` is more expressive and also encapsulates the state of
the parachain node being a full node. Some functionality like the
overseer needs to run always when the node runs alongside a parachain
node. The parachain node needs the overseer to e.g. recover PoVs. Other
things like candidate validation or pvf checking are only required for
when the node is running as validator.
* FMT
* Fix CI
* PVF: Fix external workers being required for non-collator full nodes
* Fix
* Make check more precise
> This might as well be if role.is_authority() as there aren't really any nodes
> that are both authorities and collators. But if there were a node that is an
> authority and a collator, it'd probably want to validate candidates (in its
> role as an authority).
* Just seeing if this fixes zombienet CI...
* [WIP] PVF: Split out worker binaries
* Address compilation problems and re-design a bit
* Reorganize once more, fix tests
* Reformat with new nightly to make `cargo fmt` test happy
* Address `clippy` warnings
* Add temporary trace to debug zombienet tests
* Fix zombienet node upgrade test
* Fix malus and its CI
* Fix building worker binaries with malus
* More fixes for malus
* Remove unneeded cli subcommands
* Support placing auxiliary binaries to `/usr/libexec`
* Fix spelling
* Spelling
Co-authored-by: Marcin S. <marcin@realemail.net>
* Implement review comments (mostly nits)
* Fix worker node version flag
* Rework getting the worker paths
* Address a couple of review comments
* Minor restructuring
* Fix CI error
* Add tests for worker binaries detection
* Improve tests; try to fix CI
* Move workers module into separate file
* Try to fix failing test and workers not printing latest version
- Tests were not finding the worker binaries
- Workers were not being rebuilt when the version changed
- Made some errors easier to read
* Make a bunch of fixes
* Rebuild nodes on version change
* Fix more issues
* Fix tests
* Pass node version from node into dependencies to avoid recompiles
- [X] get version in CLI
- [X] pass it in to service
- [X] pass version along to PVF
- [X] remove rerun from service
- [X] add rerun to CLI
- [X] don’t rerun pvf/worker’s (these should be built by nodes which have rerun enabled)
* Some more improvements for smoother tests
- [X] Fix tests
- [X] Make puppet workers pass None for version and remove rerun
- [X] Make test collators self-contained
* Add back rerun to PVF workers
* Move worker binaries into files in cli crate
As a final optimization I've separated out each worker binary from its own crate
into the CLI crate. Before, the worker bin shared a crate with the worker lib,
so when the binaries got recompiled so did the libs and everything transitively
depending on the libs. This commit fixes this regression that was causing
recompiles after every commit.
* Fix bug (was passing worker version for node version)
* Move workers out of cli into root src/bin/ dir
- [X] Pass in node version from top-level (polkadot)
- [X] Add build.rs with rerun-git-head to root dir
* Add some sanity checks for workers to dockerfiles
* Update malus
+ [X] Make it self-contained
+ [X] Undo multiple binary changes
* Try to fix clippy errors
* Address `cargo run` issue
- [X] Add default-run for polkadot
- [X] Add note about installation to error
* Update readme (installation instructions)
* Allow disabling external workers for local/testing setups
+ [X] cli flag to enable single-binary mode
+ [X] Add message to error
* Revert unnecessary Cargo.lock changes
* Remove unnecessary build scripts from collators
* Add back missing malus commands (should fix failing ZN job)
* Some minor fixes
* Update Cargo.lock
* Fix some build errors
* Undo self-contained binaries; cli flag to disable version check
+ [X] Remove --dont-run-external-workers
+ [X] Add --disable-worker-version-check
+ [X] Remove PVF subcommands
+ [X] Redo malus changes
* Try to fix failing job and add some docs for local tests
---------
Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: parity-processbot <>
* Companion for Substrate#14511
https://github.com/paritytech/substrate/pull/14511
* Make benchmarking build without any native runtime
* Fix warning
* update lockfile for {"substrate"}
---------
Co-authored-by: parity-processbot <>
* Implement runtime apis for fake runtime
These runtime api implementations are only used to make the compiler
think that we have implemented all required runtime apis. They will not
be called as we switch the executor to `WasmExecutor`. In the near
future we will not require these fake implementations anymore after
Substrate has shifted away from this compile time requirement.
This brings us the advantage that the `polkadot-service` doesn't need to
depend on the runtimes for getting the `RuntimeApi` type.
It also removes around 1min of build time on my machine ;)
* Fix warning
* FMT
* ".git/.scripts/commands/fmt/fmt.sh"
* Use more descriptive id
* Fix warnings
* Adapt path
* Fix 🙈
---------
Co-authored-by: command-bot <>
We consider BEEFY mature enough to run by default on all nodes
for test networks (Rococo/Wococo/Versi).
Right now, most nodes are not running it since it's opt-in using
--beefy flag. Switch to an opt-out model for test networks.
Replace --beefy flag from CLI with --no-beefy and have BEEFY
client start by default on test networks.
Signed-off-by: acatangiu <adrian@parity.io>
* PVF: Refactor workers into separate crates, remove host dependency
* Fix compile error
* Remove some leftover code
* Fix compile errors
* Update Cargo.lock
* Remove worker main.rs files
I accidentally copied these from the other PR. This PR isn't intended to
introduce standalone workers yet.
* Address review comments
* cargo fmt
* Update a couple of comments
* Update log targets
* Happy New Year!
* Remove year entierly
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Remove years from copyright notice in the entire repo
---------
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Check spawned worker version vs node version before PVF preparation
* Address discussions
* Propagate errors and shutdown preparation and execution pipelines properly
* Add logs; Fix execution worker checks
* Revert "Propagate errors and shutdown preparation and execution pipelines properly"
This reverts commit b96cc3160ff58db5ff001d8ca0bfea9bd4bdd0f2.
* Don't try to shut down; report the condition and exit worker
* Get rid of `VersionMismatch` preparation error
* Merge master
* Add docs; Fix tests
* Update Cargo.lock
* Kill again, but only the main node process
* Move unsafe code to a common safe function
* Fix libc dependency error on MacOS
* pvf spawning: Add some logging, add a small integration test
* Minor fixes
* Restart CI
---------
Co-authored-by: Marcin S <marcin@realemail.net>
* BlockId removal: refactor: HeaderBackend::header
It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`
methods from: `BlockId<Block>` to: `Block::Hash`
This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
* missed fixes
* BlockId removal: refactor: HeaderBackend::expect_header
It changes the arguments of `HeaderBackend::expect_header` method from: `BlockId<Block>` to: `Block::Hash`
* update lockfile for {"substrate"}
* misspell fixed
Co-authored-by: parity-processbot <>
* Add clippy config and remove .cargo from gitignore
* first fixes
* Clippyfied
* Add clippy CI job
* comment out rusty-cachier
* minor
* fix ci
* remove DAG from check-dependent-project
* add DAG to clippy
Co-authored-by: alvicsam <alvicsam@gmail.com>
* Enable correct features
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Feature gate function
The expose_db function is currently feature gated with runtime-benchmarks.
After the related Substrate MR, this feature gate will actually *work*
as intended instead of doing nothing.
BUT then we also have to pass through the correct feature, hence this MR.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* update lockfile for {"substrate"}
* Fix bench features
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* fix more features
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
* use deny-list instead of allow-list for BEEFY
Instead of allowing BEEFY to run on specific test nets,
inverse the condition to explicitly disallow BEEFY on
production chains that we don't want it to run on yet.
This allows other test chains (other than Rococo/Wococo)
that use the polkadot service file to enable and test BEEFY.
Signed-off-by: Adrian Catangiu <adrian@parity.io>
* address review comments
* throw error if BEEFY enabled on production networks
Signed-off-by: acatangiu <adrian@parity.io>
* Align to changes in Substrate
* Align to the newest changes in substrate
* Rename `--disable-hardware-benchmarks` to `--no-hardware-benchmarks`
* Fix `polkadot-test-service` compilation
* Fix compilation of test parachains
* Rename to BagError
* Additional parameter for 'revert' command
* Set aux revert param to None
* Align to changes in how the WASM executor is configured in `substrate`
* update lockfile for {"substrate"}
* update lockfile for {"substrate"}
* Update substrate
* Update substrate
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: parity-processbot <>
Adrian Catangiu