Commit Graph
100 Commits
Author SHA1 Message Date
f910a15c1c GenesisConfig presets for runtime (#2714)
The runtime now can provide a number of predefined presets of
`RuntimeGenesisConfig` struct. This presets are intended to be used in
different deployments, e.g.: `local`, `staging`, etc, and should be
included into the corresponding chain-specs.

Having `GenesisConfig` presets in runtime allows to fully decouple node
from runtime types (the problem is described in #1984).

**Summary of changes:**
- The `GenesisBuilder` API was adjusted to enable this functionality
(and provide better naming - #150):
   ```rust
    fn preset_names() -> Vec<PresetId>;
fn get_preset(id: Option<PresetId>) -> Option<serde_json::Value>;
//`None` means default
    fn build_state(value: serde_json::Value);
    pub struct PresetId(Vec<u8>);
   ```

- **Breaking change**: Old `create_default_config` method was removed,
`build_config` was renamed to `build_state`. As a consequence a node
won't be able to interact with genesis config for older runtimes. The
cleanup was made for sake of API simplicity. Also IMO maintaining
compatibility with old API is not so crucial.
- Reference implementation was provided for `substrate-test-runtime` and
`rococo` runtimes. For rococo new
[`genesis_configs_presets`](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b97c5ff0ec4a1989da5ffd8b9f3f588e3/polkadot/runtime/rococo/src/genesis_config_presets.rs#L530)
module was added and is used in `GenesisBuilder`
[_presets-related_](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b97c5ff0ec4a1989da5ffd8b9f3f588e3/polkadot/runtime/rococo/src/lib.rs#L2462-L2485)
methods.

- The `chain-spec-builder` util was also improved and allows to
([_doc_](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b97c5ff0ec4a1989da5ffd8b9f3f588e3/substrate/bin/utils/chain-spec-builder/src/lib.rs#L19)):
   - list presets provided by given runtime (`list-presets`),
- display preset or default config provided by the runtime
(`display-preset`),
   - build chain-spec using named preset (`create ... named-preset`),


- The `ChainSpecBuilder` is extended with
[`with_genesis_config_preset_name`](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b97c5ff0ec4a1989da5ffd8b9f3f588e3/substrate/client/chain-spec/src/chain_spec.rs#L447)
method which allows to build chain-spec using named preset provided by
the runtime. Sample usage on the node side
[here](https://github.com/paritytech/polkadot-sdk/blob/2caffaae803e08a3d5b46c860e8016da023ff4ce/polkadot/node/service/src/chain_spec.rs#L404).

Implementation of #1984.
fixes: #150
part of: #25

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2024-04-04 18:30:54 +00:00
Michal Kucharczyk 0becc45bd8 sp_runtime: TryFrom<RuntimeString> for &str (#3942)
Added `TryFrom<&'a RuntimeString> for &'a str`
2024-04-02 16:06:01 +00:00
a756baf3b2 Support for keyring in runtimes (#2044)
This functionality is required for #1984.

This PR enables
[`sp-keyring`](https://github.com/paritytech/polkadot-sdk/blob/21d36b7b4229c4d5225944f197918cde23fda4ea/substrate/primitives/keyring/src/sr25519.rs#L31-L40)
in `no-std` environments, allowing to generate the public key (e.g.
`AccountKeyring::Alice.public().to_ss58check()`), which can be later
used in the any of built-in [_runtime-genesis-config_
variant](https://github.com/paritytech/polkadot-sdk/blob/21d36b7b4229c4d5225944f197918cde23fda4ea/polkadot/node/service/src/chain_spec.rs#L1066-L1073).


The proposal is as follows:
- expose [`core::Pair`
trait](https://github.com/paritytech/polkadot-sdk/blob/d6f15306282e3de848a09c9aa9cba6f95a7811f0/substrate/primitives/core/src/crypto.rs#L832)
in `no-std`,
- `full_crypto` feature enables `sign` method,
- `std` feature enables `generate_with_phrase` and `generate` methods
(randomness is required),
- All other functionality, currently gated by `full_crypto` will be
available unconditionally (`no-std`):
-- `from_string`
-- `from_string_with_seed`
-- `from seed`
-- `from_seed_slice`
-- `from_phrase`
-- `derive`
-- `verify`

---

Depends on https://github.com/rust-bitcoin/rust-bip39/pull/57

---------

Co-authored-by: command-bot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
2024-03-12 11:43:31 +00:00
9f5d9fa96f core: replace secp256k with k256 in crypto::ecdsa (#3525)
This PR replaces the usage of
[secp256k](https://crates.io/crates/secp256k1) crate with
[k256](https://crates.io/crates/k256) in `core::crypto::ecdsa` for
`non-std` environments as outcome of discussion in #3448.

`secp256k1` is used in `std`, meaning that we should not affect host
performance with this PR.
`k256` is enabled in runtimes (`no-std`), and is required to proceed
with #2044.

If desirable, in future we can switch to `k256` also for `std`. That
would require some performance evaluation (e.g. for EVM chains as per
https://github.com/paritytech/polkadot-sdk/issues/3448#issuecomment-1976780391).

Closes https://github.com/paritytech/polkadot-sdk/issues/3448

---------

Co-authored-by: command-bot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
2024-03-09 10:14:06 +00:00
Michal Kucharczyk 73fd8cd717 test-runtime: json_patch dep removed (#3062)
We don't need dependency on `json_patch` there is implementation of
needed function in `sc_chain_spec` crate.
This PR makes this function publicly available and removes the dep.
2024-01-25 15:03:40 +00:00
Michal Kucharczyk 87927bbd7c GenesisBuilder: arbitrary_precision feature enabled for serde_json (#2987)
[`arbitrary_precision`](https://github.com/serde-rs/json/blob/6d44b9fac9269b4decf76acac5d68e8ec9d10c58/Cargo.toml#L69-L75)
feature allows to (de-)serialize big numbers w/o error.
For some details refer also to
https://github.com/paritytech/polkadot-sdk/pull/1256#discussion_r1455564450

fixes: #2963
2024-01-18 20:55:50 +00:00
Michal Kucharczyk c8112e2c6f frame-support: sp-runtime dependency updated (serde) (#2907)
`frame-support` crate compilation fails (reported by @koute):

```
$ cargo check --no-default-features --target=wasm32-unknown-unknown

error[E0277]: the trait bound `GC: Serialize` is not satisfied
    --> substrate/frame/support/src/genesis_builder_helper.rs:32:24
     |
32   |     serde_json::to_string(&GC::default())
     |     --------------------- ^^^^^^^^^^^^^^ the trait `Serialize` is not implemented for `GC`
     |     |
     |     required by a bound introduced by this call
     |
note: required by a bound in `serde_json::to_string`
    --> /home/kou/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde_json-1.0.111/src/ser.rs:2209:17
     |
2207 | pub fn to_string<T>(value: &T) -> Result<String>
     |        --------- required by a bound in this function
2208 | where
2209 |     T: ?Sized + Serialize,
     |                 ^^^^^^^^^ required by this bound in `to_string`
help: consider further restricting this bound
     |
30   |     GC: BuildGenesisConfig + Default + serde::Serialize,
     |                                      ++++++++++++++++++
```

This PR should fix this.

For all runtimes `sp-runtime/serde` feature was likely enabled by this
(and few other pallets):

https://github.com/paritytech/polkadot-sdk/blob/f2a750ee86e72c9ab677aaf588d0a33ee8446bef/substrate/frame/system/Cargo.toml#L27
2024-01-11 00:30:32 +00:00
f6548aee31 keyring: remove lazy_static public keys hash maps (#2387)
The `lazy_static` package does not work well in `no-std`: it requires
`spin_no_std` feature, which also will propagate into `std` if enabled.
This is not what we want.

This PR removes public/private key hash-maps and replaces them with
simple static byte arrays.

`&T` versions of `AsRef/Deref/From` traits implementation were removed.

Little const helper for converting hex strings into array during compile
time was also added. (somewhat similar to _hex_literal_).

---------

Co-authored-by: command-bot <>
Co-authored-by: Koute <koute@users.noreply.github.com>
2023-12-11 15:21:20 +01:00
Michal Kucharczyk 5789d6a7b4 sassafras: taplo happy (#2583)
fixes: `taplo format check`
2023-12-01 22:23:19 +02:00
Michal Kucharczyk c68ce6e147 state-db: log_target usage fixed (#2547)
Use `LOG_TARGET/LOG_TARGET_PIN` in logs.
2023-11-29 19:19:44 +00:00
Michal KucharczykandDavide Galassi 39d6c95c0d substrate-node: NativeElseWasmExecutor is no longer used (#2521)
This PR removes `NativeElseWasmExecutor` usage from substrate node.
Instead [`WasmExecutor<(sp_io::SubstrateHostFunctions,
sp_statement_store::runtime_api::HostFunctions)>`](https://github.com/paritytech/polkadot-sdk/blob/49a41ab3bb3f630c20e5b24cec8d92382404631c/substrate/bin/node/executor/src/lib.rs#L26)
is used.

Related to #2358.

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-11-29 10:30:09 +01:00
Michal Kucharczykandcommand-bot d5d15a1802 polkadot-parachain: one chain-spec for all (#2457)
This PR removes some `ChainSpec` types which are not necessary
(left-overs from #1256). Currently `ChainSpec` does not have to be
generic over the specific `RuntimeGenesisConfig`, it is enough to use
single type for all:

https://github.com/paritytech/polkadot-sdk/blob/9f787018857660440182142adc806954d7d07709/cumulus/polkadot-parachain/src/chain_spec/mod.rs#L53-L54


related to: https://github.com/paritytech/polkadot-sdk/issues/25

---------

Co-authored-by: command-bot <>
2023-11-25 09:56:17 +01:00
Michal Kucharczyk 891628a7f1 relay-chain-consensus: set a fork_choice (#2462)
After #2001 the `fork_choice` strategy may remain uninitialized in the
block import pipeline which uses relay-chain verifier.
Refer to
https://github.com/paritytech/polkadot-sdk/pull/2430#issuecomment-1823979813
for some further discussion.
2023-11-24 14:06:19 +01:00
Michal KucharczykandSebastian Kunert 49874882cf polkadot-node-subsystems: ChainApiBackend added + polkadot-debug image version fixed (#2411)
The out-dated version (bad tag) of [polkadot
image](https://github.com/paritytech/polkadot-sdk/blob/ede4a362622dfa69eb167eaa876246b1289f4b41/.gitlab/pipeline/zombienet/cumulus.yml#L31)
([docker
info](https://hub.docker.com/layers/paritypr/polkadot-debug/master/images/sha256:adb1658052cf671b50c90d5cece5c7a131efa1a95978249bd5cb85a5ad654f7a?context=explore))
was used. This PR fixes this.

See also:
https://github.com/paritytech/polkadot-sdk/pull/2411#issuecomment-1822632724

Also adds an abstraction that allows asynchronous backends to be passed to `ChainApiSubsystem`
---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-11-22 19:24:11 +01:00
Michal Kucharczyk 2183669d05 cumulus-test-service: block import fix (#2430)
This is follow-up for:
https://github.com/paritytech/polkadot-sdk/pull/2001

Block import queue for `test-parachain` (`cumulus-test-service`) shall
use delayed best block feature.

This should fixed broken zombienet tests.
2023-11-21 20:03:55 +01:00
Michal KucharczykandBastian Köcher b25d29a502 cumulus-consensus-common: block import: delayed_best_block flag added (#2001)
This PR adds the `delayed_best_block` flag to `ParachainBlockImport`. If
not set, the `params.fork_choice` is not updated (to
`ForkChoiceStrategy::Custom`) during the block import.

When `delayed_best_block` is set to `false` all parachain blocks on the
[longest
fork](https://github.com/paritytech/polkadot-sdk/blob/552be4800d9e4b480f79a300fc531783e04be099/substrate/client/service/src/client/client.rs#L708-L709)
will be notified as the best block, allowing transaction pool to be
updated with every imported block.

Otherwise imported blocks will not be notified as best blocks
(`fork_choice=ForkChoiceStrategy::Custom(false)`), and transaction pool
would be updated only with best block received from relay-chain.

Improvement for: #1202

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-11-21 16:22:42 +02:00
2fd8c51ebc chain-spec-builder: cleanup (#2174)
This PR removes:
-  `New`, `Generate`, `Edit` commands,
- `kitchensink` dependency
from the `chain-spec-builder` util.

New `convert-to-raw`, `update-code` commands were added.

Additionally renames the `runtime` command (which was added in #1256) to
`create`.

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-11-21 11:51:23 +01:00
5007e2dd5c crypto: lazy_static removed, light parser for address URI added (#2250)
The `lazy_static` package does not work well in `no-std`: it requires
`spin_no_std` feature, which also will propagate into `std` if enabled.
This is not what we want.

This PR provides simple address uri parser which allows to get rid of
_regex_ which was used to parse the address uri, what in turns allows to
remove lazy_static.

Three regular expressions
(`SS58_REGEX`,`SECRET_PHRASE_REGEX`,`JUNCTION_REGEX`) were replaced with
the parser which unifies all of them.

The new parser does not support Unicode, it is ASCII only.

Related to: #2044

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: command-bot <>
2023-11-17 14:43:37 +01:00
Michal Kucharczyk eabf9fb897 integrations-test: build_genesis_storage name fix (#2232)
Some legacy tests were mistakenly merged in #1256 for `emulated-integration-tests-common` crate.
This PR fixes the function name `build_genesis_storage` (no need to use `legacy` suffix, even though the genesis is built from `RuntimeGenesisConfig`).
2023-11-08 18:10:40 +01:00
b8acc57c50 sc-chain-spec: add support for custom host functions (#2190)
Genesis building in runtime may involve calling some custom host
functions. This PR allows to pass `HostFunctions` into the `ChainSpec`
struct, which in turn are passed to `WasmExecutor`. The `ChainSpec` now
has extended host functions type parameter:
```
pub struct ChainSpec<G, E = NoExtension, EHF = ()>
```
which will be combined with the default set
(`sp_io::SubstrateHostFunctions`) in an instance of `WasmExecutor` used
to build the genesis config.

Fix for #2188

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-11-08 18:31:37 +02:00
Michal Kucharczyk 305aefc43d serde_json: bumped to 1.0.108 (#2168)
This PR updates the version of `serde_json` to `1.0.108` throughout the
codebase.
2023-11-06 14:00:06 +02:00
8ba7a6aba8 chain-spec: getting ready for native-runtime-free world (#1256)
This PR prepares chains specs for _native-runtime-free_  world.

This PR has following changes:
- `substrate`:
  - adds support for:
- JSON based `GenesisConfig` to `ChainSpec` allowing interaction with
runtime `GenesisBuilder` API.
- interacting with arbitrary runtime wasm blob to[
`chain-spec-builder`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/bin/utils/chain-spec-builder/src/lib.rs#L46)
command line util,
- removes
[`code`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/frame/system/src/lib.rs#L660)
from `system_pallet`
  - adds `code` to the `ChainSpec`
- deprecates
[`ChainSpec::from_genesis`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/client/chain-spec/src/chain_spec.rs#L263),
but also changes the signature of this method extending it with `code`
argument.
[`ChainSpec::builder()`](https://github.com/paritytech/substrate/blob/20bee680ed098be7239cf7a6b804cd4de267983e/client/chain-spec/src/chain_spec.rs#L507)
should be used instead.
- `polkadot`:
- all references to `RuntimeGenesisConfig` in `node/service` are
removed,
- all
`(kusama|polkadot|versi|rococo|wococo)_(staging|dev)_genesis_config`
functions now return the JSON patch for default runtime `GenesisConfig`,
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,

- `cumulus`:
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,
- _JSON_ patch configuration used instead of `RuntimeGenesisConfig
struct` in all chain specs.
  
---------

Co-authored-by: command-bot <>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Kevin Krone <kevin@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-11-05 15:19:23 +01:00
1c0b437330 cumulus test runtime: remove GenesisExt (#2147)
This PR removes the `GenesisExt` wrapper over the `GenesisRuntimeConfig`
in `cumulus-test-service`. Initialization of values that were performed
by `GenesisExt::BuildStorage` was moved into `test_pallet` genesis.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-11-04 11:25:07 +02:00
Michal Kucharczyk a69da4a85f Switch from tiny-bip39 to bip39 crate (#2084)
Switch from: 
https://crates.io/crates/tiny-bip39
to:
https://crates.io/crates/bip39

Required for: https://github.com/paritytech/polkadot-sdk/pull/2044
2023-10-30 17:03:30 +01:00
Michal KucharczykandBastian Köcher 50242a61d7 rococo-runtime: RococoGenesisExt removed (#1490)
[`RococoGenesisExt`](https://github.com/paritytech/polkadot-sdk/blob/a414ea7515c9cdc81f1d12410e646afc148250e8/polkadot/node/service/src/chain_spec.rs#L152-L171)
is removed. It was the hack to allow overwriting
`EpochDurationInBlocks`. Removal of `RococGenesisExt` prevents from
manipulating the state to change the runtime constants.

Changes:
- Environment variable which controls the `time::EpochDurationInBlocks`
value was added: `ROCOCO_EPOCH_DURATION` (epoch duration will be set to
the value of env),
- `10,100,600` versions of rococo-runtime are built in CI and put into `polkadot-debug` docker image.

`rococo-runtime` building examples:
- to build runtime for `versi_staging_testnet` which had
EpochDurationInBlocks set to 100:
  ```
ROCOCO_EPOCH_DURATION=100 cargo build --features=fast-runtime -p
rococo-runtime
  ```
- to build runtime for `wococo_development`
  ```
ROCOCO_EPOCH_DURATION=10 cargo build --features=fast-runtime -p
rococo-runtime
  ```
- to build `versi-staging` chain spec:
  ```
ROCOCO_EPOCH_DURATION=100 cargo run -p polkadot --features=fast-runtime
-- build-spec --chain versi-staging --raw
  ```
- to build `wococo-dev` chain spec:
  ```
ROCOCO_EPOCH_DURATION=10 cargo run -p polkadot --features=fast-runtime
-- build-spec --chain wococo-dev --raw
  ```

It is also possible to change the epoch duration by replacing the `code` field in the chain spec with the hex dump of pre-built runtime wasm blob (because the epoch duration is hard-coded into wasm blob).

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-28 18:29:12 +02:00
Michal Kucharczykandordian 5a2833cceb genesis-builder: implemented for all runtimes (#1492)
This PR implements [`GenesisBuilder`
API](https://github.com/paritytech/polkadot-sdk/blob/a414ea7515c9cdc81f1d12410e646afc148250e8/substrate/primitives/genesis-builder/src/lib.rs#L38)
for all the runtimes in polkadot repo.

Step towards: paritytech/polkadot-sdk#25

---------

Co-authored-by: ordian <write@reusable.software>
2023-09-27 11:59:19 +02:00
ab3a3bc278 BlockId removal: tx-pool refactor (#1678)
It changes following APIs:
- trait `ChainApi`
-- `validate_transaction`

- trait `TransactionPool` 
--`submit_at`
--`submit_one`
--`submit_and_watch`

and some implementation details, in particular:
- impl `Pool` 
--`submit_at`
--`resubmit_at`
--`submit_one`
--`submit_and_watch`
--`prune_known`
--`prune`
--`prune_tags`
--`resolve_block_number`
--`verify`
--`verify_one`

- revalidation queue

All tests are also adjusted.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-27 11:58:39 +02:00
Michal Kucharczyk af3e56cd86 RuntimeCode: doc fixed (#1461) 2023-09-08 16:56:19 +02:00
Michal KucharczykandJavier Viola 2c3e869c38 substrate: chain-spec paths corrected in zombienet tests (#1362)
* substrate: chain-spec paths corrected in zombienet tests

* fix chain-spec path in cumulus test

* disable beefy on validator

---------

Co-authored-by: Javier Viola <javier@parity.io>
2023-09-02 21:25:43 +02:00
53cbda1de6 test-runtime: GenesisBuilder runtime API impl + tests (#14310)
* test-runtime: GenesisBuilder runtime API impl + tests

This PR provides implementation of `GenesisBuilder` API for `substrate-test-runtime`, can be considered as reference imiplementation for other runtimes.
The `GenesisBuilder` implementation is gated by `gensis-config` feature.

Tested scenarios:
- default `GenesisConfig` to JSON blob,
- deserialization of `GenesisConfig` from custom JSON, and storing its keys into the Storage (genesis storage creation).
- creation of genesis storage using partial JSON definition,
- checking if invalid/renamed JSON files causes the runtime to panic,

* missing file added

* client: GenesisBuilder helper added

* feature renamed: genesis-config -> genesis-builder

* Update Cargo.toml

* Update Cargo.toml

* Update Cargo.toml

* Update Cargo.toml

* redundant function removed

* genesis builder helper: introduced RuntimeGenesisBuild

* test-runtime: get rid of unused T

* redundant bound removed

* helper: use GenesisBuild again

* tests adjusted for on_genesis

* test-runtime: support for BuildGenesisConfig

* helper: BuildGenesisConfig support

* Update client/genesis-builder/src/lib.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/test_json/readme.md

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/test_json/readme.md

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* jsons are now human-friendly

* fix

* improvements

* jsons fixed

* helper: no_defaults added

* test-runtime: no_defaults added

* test-runtime: patching fn removed

* helper: patching fn removed

* helper: moved to frame_support

* test-runtime: fixes

* Cargo.lock updated

* fmt + naming

* test-runtime: WasmExecutor used

* helper / test-runtime: struct removed

* test-runtime: merge fixes

* Cargo.lock + test-utils/runtime/Cargo.toml updated

* doc fixed

* client/rpc: test fixed (new rt api)

* client/rpc-spec-v2: test fix

* doc fix

* test-runtime: disable-genesis-builder feature

* fix

* fix

* ".git/.scripts/commands/fmt/fmt.sh"

* test-runtime: rerun added to build script

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: parity-processbot <>
2023-07-24 16:12:52 +00:00
Michal Kucharczyk 14b086263e chain-spec-builder: redundant file removed src/main.rs (#14600)
* chain-spec-builder: redundant functions removed from main

* src/main.rs removed
2023-07-20 12:22:29 +00:00
Michal Kucharczyk b15a56972b chain-spec: polkadot_staging_testnet config removed (#7509)
* chain-spec: polkadot_staging_testnet config removed

The config has empty initial authority test. It is unusable.

```
$ polkadot export-blocks --chain polkadot-staging --from=0 --to=0
...
Thread 'main' panicked at 'Empty validator set for session 0 in genesis block!',
/home/miszka/.cargo/git/checkouts/substrate-7e08433d4c370a21/c446786/frame/session/src/lib.rs:463
```

* warning fix
2023-07-18 10:58:08 +00:00
Michal Kucharczykandcommand-bot 94d2e4bc28 GenesisBuild<T,I> deprecated. BuildGenesisConfig added (#2757)
* GenesisBuild<T,I> deprecated. BuildGenesisConfig added

* ".git/.scripts/commands/fmt/fmt.sh"

* integration-tests/emulated: ..Default::default added to genesis configs

* Cargo.lock updated

* Cargo.lock updated

* update lockfile for {"polkadot", "substrate"}

* clippy fixes

* clippy fixes

* clippy fixes again

---------

Co-authored-by: command-bot <>
2023-07-12 12:00:17 +00:00
Michal Kucharczykandparity-processbot 95ba849ee6 GenesisBuild<T,I> deprecated. BuildGenesisConfig added (#7397)
* GenesisBuild<T,I> deprecated. BuildGenesisConfig added

* fmt

* fixes

* more fixes

* more fixes

* fixes

* update lockfile for {"substrate"}

* fix

---------

Co-authored-by: parity-processbot <>
2023-07-12 11:04:15 +00:00
87d41d0a89 GenesisBuild<T,I> deprecated. BuildGenesisConfig added. (#14306)
* frame::support: GenesisConfig types for Runtime enabled

* frame::support: macro generating GenesisBuild::build for RuntimeGenesisConfig

* frame: ambiguity BuildStorage vs GenesisBuild fixed

* fix

* RuntimeGenesisBuild added

* Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed"

This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293.

* Revert "fix"

This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b.

* Revert "RuntimeGenesisBuild added"

This reverts commit 3c131b618138ced29c01ab8d15d8c6410c9e128b.

* Revert "Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed""

This reverts commit 2b1ecd467231eddec69f8d328039ba48a380da3d.

* Revert "Revert "fix""

This reverts commit fd7fa629adf579d83e30e6ae9fd162637fc45e30.

* Code review suggestions

* frame: BuildGenesisConfig added, BuildGenesis deprecated

* frame: some pallets updated with BuildGenesisConfig

* constuct_runtime: support for BuildGenesisConfig

* frame::support: genesis_build macro supports BuildGenesisConfig

* frame: BuildGenesisConfig added, BuildGenesis deprecated

* Cargo.lock update

* test-runtime: fixes

* Revert "fix"

This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b.

* Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed"

This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293.

* self review

* doc fixed

* ui tests fixed

* fmt

* tests fixed

* genesis_build macrto fixed for non-generic GenesisConfig

* BuildGenesisConfig constraints added

* warning fixed

* some duplication removed

* fmt

* fix

* doc tests fix

* doc fix

* cleanup: remove BuildModuleGenesisStorage

* self review comments

* fix

* Update frame/treasury/src/tests.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Update frame/support/src/traits/hooks.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* doc fix: GenesisBuild exposed

* ".git/.scripts/commands/fmt/fmt.sh"

* frame: more serde(skip) + cleanup

* Update frame/support/src/traits/hooks.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* frame: phantom fields moved to the end of structs

* chain-spec: Default::default cleanup

* test-runtime: phantom at the end

* merge master fixes

* fix

* fix

* fix

* fix

* fix (facepalm)

* Update frame/support/procedural/src/pallet/expand/genesis_build.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* fmt

* fix

* fix

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-07-12 10:22:12 +00:00
9a5d9a586e GenesisBuilder runtime API (#14131)
* GenesisConfigBuilder: preliminary API proposal

* fmt

* comment removed

* build_default_config removed

* Update client/genesis-builder/src/lib.rs

* config -> gensis_config

* GenesisConfigBuilder: helper added

* moved to primitives

* licesne changed to apache-2.0

* Cargo.toml: name/path to genesis-builder updated

* helper removed

* sp-sd version bumped

* sp-std bump

* naming + new function

* fix

* build_from_patch_json -> build_with_patch

* fix

* Cargo.lock updated

* readme: license updated

* Update primitives/genesis-builder/src/lib.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update primitives/genesis-builder/src/lib.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update primitives/genesis-builder/Cargo.toml

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Cargo.lock updated

* removed redundant function

* GenesisConfigBuilder API: no_defaults function added

* Cargo.lock updated

* GenesisConfigBuilder API: patching fn removed

* trigger CI job

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-06-26 11:56:15 +00:00
Michal KucharczykandBastian Köcher 52a0910c71 construct_runtime: support for expanding the macro code (#14379)
* construct_runtime: support for expanding the macro code

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/procedural/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* fmt added

* fmt removed

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-06-14 18:49:48 +00:00
Michal Kucharczyk 2493375159 doc fix (#2676)
fixing error:
```
warning: unresolved link to `well_known_keys::CODE`
   --> pallets/parachain-system/src/lib.rs:560:12
    |
560 |     #[pallet::storage]
    |               ^^^^^^^
    |
    = note: the link appears in this line:
            " As soon as the relay chain gives us the go-ahead signal, we will overwrite the [`:code`][well_known_keys::CODE]"
                                                                                                       ^^^^^^^^^^^^^^^^^^^^^
    = note: no item named `well_known_keys` in scope
    = note: `#[warn(rustdoc::broken_intra_doc_links)]` on by default
```
2023-06-02 00:23:50 +01:00
Michal Kucharczykandparity-processbot 26116ef702 Serialize/Deserialize trait implemented in no-std for numerous types (#2660)
* Serialize/Deserialize trait implemented in no-std for numerous types

* Cargo.lock updated

* update lockfile for {"substrate", "polkadot"}

---------

Co-authored-by: parity-processbot <>
2023-06-01 18:50:24 +02:00
Michal Kucharczykandparity-processbot a767358122 Serialize/Deserialize trait implemented in no-std for numerous types (#7312)
* ParaId: Serialize/Deserialize trait implemented in no-std

* Serialize/Deserialize trait implemented in no-std for more types

* serde in deps enabled

* fix

* missing types added

* update lockfile for {"substrate"}

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: parity-processbot <>
2023-06-01 16:17:49 +00:00
Michal KucharczykandBastian Köcher dc7161276d frame: support for serde added (#14261)
* frame: support for serde added

- enabled `serde` features in dependent crates, no gate feature introduced, linker should do the job and strip unused code.

- frame::staking: added impl of `serde::Serialize, serde::Deserialize` for `enum Forcing`

- primitives::runtime: impl_opaque_keys macro provides `Serialize/Deserialize` impl if `serde` is enabled

- primitives::staking: added impl of `serde::Serialize`, `serde::Deserialize` for `enum StakerStatus`

* frame::support: serde for pallets' GenesisConfig enabled in no-std

* Cargo.lock updated

* Update primitives/staking/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* fix

* Cargo.lock update + missed serde/std in beefy

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-06-01 15:35:34 +00:00
Michal Kucharczyk e7e13116f7 app_crypto: macro improvements (#14263)
* app_crypto: macro improvements

During `app_crypto`  macro expansion the `cfg` feature gate was injected
  into the macro client crate. This resulted in compilation error if
`serde` or `std` was not defined in client crate. This PR fixes this
problem.

For reference, the error was:

```
  error: cannot find macro `format` in this scope
    --> /home/miszka/parity/10-genesis-config/substrate-2/primitives/consensus/aura/src/lib.rs:32:3
     |
  32 |         app_crypto!(sr25519, AURA);
     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
...

  error[E0433]: failed to resolve: use of undeclared type `String`
    -->
/home/miszka/parity/10-genesis-config/substrate-2/primitives/consensus/aura/src/lib.rs:32:3
     |
  32 |         app_crypto!(sr25519, AURA);
     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^ use of undeclared type `String`
```

* app_crypto: cleanup
2023-05-30 12:15:13 +00:00
Michal Kucharczykandparity-processbot 4f019c4e5b bounded-collections bump to 0.1.7 (#7305)
* bounded-collections bump to 0.1.7

Companion for: paritytech/substrate#14225

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-05-29 17:24:14 +00:00
Michal Kucharczyk f602db2d96 bounded-collections bump to 0.1.7 (#14255) 2023-05-29 16:52:03 +00:00
Michal Kucharczyk a673b7ff7c primitives: added serde feature (#14256)
serde feature added for:
- sp-statement-store
- sp-authority-discovery
2023-05-29 14:32:56 +00:00
Michal Kucharczykandcommand-bot 4909cbb5b1 pallets: implement Default for GenesisConfig in no_std (#2624)
* pallets: implement Default for GenesisConfig in no_std

This change is follow-up of: https://github.com/paritytech/substrate/pull/14108

It is a step towards: https://github.com/paritytech/substrate/issues/13334

* ".git/.scripts/commands/fmt/fmt.sh"

* update lockfile for {"substrate", "polkadot"}

---------

Co-authored-by: command-bot <>
2023-05-25 23:03:13 +00:00
Michal Kucharczykandparity-processbot 3d008cd074 pallets: implement Default for GenesisConfig in no_std (#7271)
* pallets: implement Default for GenesisConfig in no_std

This change is follow-up of: https://github.com/paritytech/substrate/pull/14108

It is a step towards: https://github.com/paritytech/substrate/issues/13334

* Cargo.lock updated

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-05-25 22:26:02 +00:00
Michal Kucharczykandparity-processbot e31a214a85 frame: GenesisBuild::build allowed in no_std (#14107)
* frame: GenesisBuild::build allowed in no_std

i`GenesisBuild::build` function will be required for no_std in no native
runtime world.

`GenesisBuild::build` macro generated function allows to build the runtime
GenesisConfig assembled from all pallets' GenesisConfigs.

* fixes

* GenesisBuild::build avaiable in no-std

- #[cfg(feature = "std")] is not longer added to GenesisBuild implementation.

* system: hash69 available for no-std

* elections-phragmen: panic message fixed for no_std

* frame::suport: doc updated

* test-runtime: default for GenesisConfig

* frame::test-pallet: serde/std added to std feature deps

* Cargo.toml: deps sorted

* Cargo.lock update

cargo update -p frame-support-test-pallet -p frame-support-test

* frame ui tests: cleanup

---------

Co-authored-by: parity-processbot <>
2023-05-25 21:49:25 +00:00
Michal Kucharczyk 6efbc1a438 xcm-builder: remove clone for clippy (#7291)
fix for:
```
error: using `clone` on type `[u8; 32]` which implements the `Copy`
trait
  --> xcm/xcm-builder/src/routing.rs:44:28
   |
44 |             message.0.push(SetTopic(unique_id.clone()));
   |                                     ^^^^^^^^^^^^^^^^^ help: try
removing the `clone` call: `unique_id`
   |
   = help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
   = note: `-D clippy::clone-on-copy` implied by `-D clippy::complexity`
```
2023-05-25 16:14:59 +02:00
Michal Kucharczykandcommand-bot 613420a035 frame: Enable GenesisConfig in no_std (#14108)
* frame: Default for GenesisConfig in no_std

`Default` for `GenesisConfig` will be required for no_std in no native
runtime world. It must be possible to instantiate default GenesisConfig
for pallets and runtime.

* ".git/.scripts/commands/fmt/fmt.sh"

* hash69 in no_std reverted

* derive(DefaultNoBound) for GenesisConfig used when possible

* treasury: derive(Default)

* Cargo.lock update

* genesis_config: compiler error improved

When std feature is not enabled for pallet, the GenesisConfig will be
defined, but serde::{Serialize,Deserialize} traits will not be
implemented.

The compiler error indicates the reason of latter errors.

This is temporary and serde traits will be enabled with together with
`serde` support in frame.

---------

Co-authored-by: command-bot <>
2023-05-20 07:34:23 +00:00
Michal Kucharczyk 580efaec85 Added serde feature to sp-consensus-aura (#14166)
This is followup of #13027.

`Aura` need to enable `serde` feature in dependent crates, otherwise
test-substrate-runtime compilation fails with the following error if
`serde` is enabled:

```
  error: cannot find macro `format` in this scope
    -->
/home/miszka/parity/10-genesis-config/substrate-master/primitives/consensus/aura/src/lib.rs:50:3
     |
  50 |         app_crypto!(ed25519, AURA);
     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = help: consider importing one of these items:
             scale_info::prelude::format
             sp_application_crypto::format
     = note: this error originates in the macro
`$crate::app_crypto_public_common_if_serde` which comes from the
expansion of the macro `app_crypto` (in Nightly builds, run with -Z
macro-backtrace for more info)
```
2023-05-18 11:36:48 +00:00
6a295e7c28 substrate-test-runtime migrated to "pure" frame runtime (#13737)
* substrate-test-runtime migrated to pure-frame based

* test block builder: helpers added

* simple renaming

* basic_authorship test adjusted

* block_building storage_proof test adjusted

* babe: tests: should_panic expected added

* babe: tests adjusted

ConsensusLog::NextEpochData is now added by pallet_babe as
pallet_babe::SameAuthoritiesForever trigger is used in runtime config.

* beefy: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digests stored during block execution are
checked against header digests:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(MmrRoot / AuthoritiesChange) is used.

* grandpa: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digest logs stored during block execution are
checked against header digest logs:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(ScheduledChange / ForcedChange and DigestItem::Other) is used.

* network:bitswap: test adjusted

The size of unchecked extrinsic was increased. The pattern used in test will
be placed at the end of scale-encoded buffer.

* runtime apis versions adjusted

* storage keys used in runtime adjusted

* wasm vs native tests removed

* rpc tests: adjusted

Transfer transaction processing was slightly improved, test was
adjusted.

* tests: sizes adjusted

Runtime extrinsic size was increased. Size of data read during block
execution was also increased due to usage of new pallets in runtime.

Sizes were adjusted in tests.

* cargo.lock update

cargo update -p substrate-test-runtime -p substrate-test-runtime-client

* warnings fixed

* builders cleanup: includes / std

* extrinsic validation cleanup

* txpool: benches performance fixed

* fmt

* spelling

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Apply code review suggestions

* Apply code review suggestions

* get rid of 1063 const

* renaming: UncheckedExtrinsic -> Extrinsic

* test-utils-runtime: further step to pure-frame

* basic-authorship: tests OK

* CheckSubstrateCall added + tests fixes

* test::Transfer call removed

* priority / propagate / no sudo+root-testing

* fixing warnings + format

* cleanup: build2/nonce + format

* final tests fixes

all tests are passing

* logs/comments removal

* should_not_accept_old_signatures test removed

* make txpool benches work again

* Cargo.lock reset

* format

* sudo hack removed

* txpool benches fix+cleanup

* .gitignore reverted

* rebase fixing + unsigned cleanup

* Cargo.toml/Cargo.lock cleanup

* force-debug feature removed

* mmr tests fixed

* make cargo-clippy happy

* network sync test uses unsigned extrinsic

* cleanup

* ".git/.scripts/commands/fmt/fmt.sh"

* push_storage_change signed call remove

* GenesisConfig cleanup

* fix

* fix

* GenesisConfig simplified

* storage_keys_works: reworked

* storage_keys_works: expected keys in vec

* storage keys list moved to substrate-test-runtime

* substrate-test: some sanity tests + GenesisConfigBuilder rework

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

* Review suggestions

* fix

* fix

* beefy: generate_blocks_and_sync block_num sync with actaul value

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* cargo update -p sc-rpc -p sc-transaction-pool

* Review suggestions

* fix

* doc added

* slot_duration adjusted for Babe::slot_duration

* small doc fixes

* array_bytes::hex used instead of hex

* tiny -> medium name fix

* Apply suggestions from code review

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* TransferData::try_from_unchecked_extrinsic -> try_from

* Update Cargo.lock

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-05-04 16:20:22 +00:00
469726ee74 proc-macro: check for non-args runtime calls added (#13742)
* proc-macro: check for non-args runtime calls added

* Update primitives/api/proc-macro/src/impl_runtime_apis.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: parity-processbot <>
2023-03-29 14:54:02 +00:00
Michal Kucharczyk f82272c5cc zombienet: renamed deprecated transfer function (#13652)
* zombienet: renamed deprecated transfer function

* camelCase
2023-03-21 07:56:33 +00:00
Michal Kucharczykandparity-processbot d5650ba07b frame::executive: LOG_TARGET const added (#13650)
part of: #12873

Co-authored-by: parity-processbot <>
2023-03-20 20:58:21 +00:00
Michal Kucharczyk b5e7fbfb58 Service::Error: Storage variant removed (#13481) 2023-02-27 23:10:34 +00:00
Michal Kucharczykandparity-processbot 749b078bac BlockId removal: BlockBuilderProvider::new_block_at (#2219)
* `BlockId` removal: `BlockBuilderProvider::new_block_at`

It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`

* update lockfile for {"polkadot", "substrate"}

* fix

---------

Co-authored-by: parity-processbot <>
2023-02-21 20:28:52 +00:00
Michal Kucharczykandparity-processbot 0e59d866f3 BlockId removal: BlockBuilderProvider::new_block_at (#6734)
* `BlockId` removal: `BlockBuilderProvider::new_block_at`

It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`

* useless import removed

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-02-21 19:26:46 +00:00
Michal Kucharczyk 5ef88dd398 BlockId removal: BlockBuilderProvider::new_block_at (#13401)
* `BlockId` removal: `BlockBuilderProvider::new_block_at`

It changes the arguments of `BlockBuilderProvider::new_block_at` from:
`BlockId<Block>` to: `Block::Hash`

* fmt

* fix

* more fixes
2023-02-21 18:36:00 +00:00
Michal Kucharczykandparity-processbot 21e5a982ff BlockId removal: refactor of runtime API (#2190)
* BlockId removal: refactor of runtime API

It changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`

* fmt

* cargo update -p polkadot-primitives

* Revert "cargo update -p polkadot-primitives"

This reverts commit 58506ecd31e1e5e42b78c2db3982acbc4d8edc70.

* update lockfile for {"substrate", "polkadot"}

---------

Co-authored-by: parity-processbot <>
2023-02-21 00:23:46 +00:00
Michal Kucharczykandparity-processbot d091a84a07 BlockId removal: refactor of runtime API (#6721)
* BlockId removal: refactor of runtime API

It changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`

* fmt

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>
2023-02-20 23:44:20 +00:00
7a10154188 BlockId removal: runtime-api refactor (#13255)
* BlockId removal: refactor of runtime API

It changes the arguments of:
- `ApiExt` methods:  `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`

It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* BlockId removal: refactor of runtime API - tests

- tests adjusted to new runtime API,
- some tests migrated from block number to block hash

* benchmarking-cli: BlockId(0) migrated to info().genesis_hash

`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.

* trivial: api.call(BlockId) -> api.call(Hash)

- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,

* CallApiAtParams::at changed to Block::Hash

* missed doc updated

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

* BlockId removal: Benchmark::consumed_weight

Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.

* at_hash renamed

* wrong merge fixed

* beefy worker: merged with master

* beefy: tests: missing block problem fixed

* Apply review suggestion

* fix

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-02-20 22:47:21 +00:00
Michal KucharczykAntonparity-processbot
36480b158d BlockId removal: Client::runtime_version_at (#13393)
* BlockId removal: Client::runtime_version_at

It changes the arguments of `Client::runtime_version_at` from: `BlockId<Block>` to: `Block::Hash`

* Apply suggestions from code review

Co-authored-by: Anton <anton.kalyaev@gmail.com>

---------

Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: parity-processbot <>
2023-02-16 22:34:00 +00:00
2e4ea01311 BlockId removal: CallApiAt::state_at (#13394)
* BlockId removal: CallApiAt::state_at

It changes the arguments of:
- `CallApiAt::state_at`,
- `Client::code_at`,
- `Client::state_at`

from: `BlockId<Block>` to: `Block::Hash`

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: parity-processbot <>
2023-02-16 22:21:15 +00:00
Michal Kucharczyk 61ef6baa1e zombienet: is up timtout increased to 30s (#13386) 2023-02-14 14:17:58 +00:00
Michal Kucharczykandparity-processbot cad4982afd zombienet: warp-sync tests for validators (#13078)
* zombienet validators warp sync draft

Sketch of warp-sync test for validators.
Not tested.

Follow-up of: #12769

* yet another warp-sync scenario added

- all validators are synced from DB,
- some full nodes are synced from DB,
- full-node is warp-synced

* fixes

* fixes

* missing files

* path fixed

---------

Co-authored-by: parity-processbot <>
2023-02-07 10:28:18 +01:00
c251912652 Storage monitor added to polkadot node (#6619)
* Storage monitor added to polkadot node

Related to: https://github.com/paritytech/substrate/issues/12399

* Cargo.lock updated

* Cargo.lock update

* Revert "Cargo.lock update"

This reverts commit 04dfe5fe65ea7d1476d4632ad647f2c1c669ab1a.

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <info@kchr.de>

* missing_docs fixes

* Cargo.lock updated

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <info@kchr.de>
2023-01-27 22:46:36 +01:00
Michal Kucharczyk 6d0ecc1967 nix upgrade to 0.26.1 (#2135) 2023-01-26 12:47:17 +00:00
Michal Kucharczyk 64b55cb170 upgrade nix to 0.26.1 (#13230) 2023-01-26 07:59:18 +00:00
Michal Kucharczyk 8102eebe9b nix upgrade to 0.26.1 (#6622) 2023-01-25 05:58:33 +00:00
Michal Kucharczyk d99e2e4bea storage-monitor: statvfs arithmetic bug fixed (#13234) 2023-01-24 19:22:47 +00:00
14a4eed2aa service: storage monitor added (#13082)
* service: storage monitor added

Storage monitor added. It uses `notify` create to get notifications
about any changes to monitored path (which is database path).
Notifications are consumed in essential task which terminates when
available storage space drops below given threshold.

Closes: #12399

* Cargo.lock updated

* misspell

* fs events throttling added

* minor updates

* filter out non mutating events

* misspell

* ".git/.scripts/commands/fmt/fmt.sh"

* Update client/service/src/storage_monitor.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* storage-monitor crate added

* cleanup: configuration + service builder

* storage_monitor in custom service (wip)

* copy-paste bad desc fixed

* notify removed

* storage_monitor added to node

* fix for clippy

* publish = false

* Update bin/node/cli/src/command.rs

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* crate name: storage-monitor -> sc-storage-monitor

* error handling improved

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* publish=false removed

Co-authored-by: command-bot <>
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-01-24 13:26:04 +00:00
Michal Kucharczyk d2ec213ce5 libc crate update (#6616)
cargo update -p libc
2023-01-24 12:56:47 +00:00
025f9d9ba3 BlockId removal: refactor: CallExecutor trait (#13173)
* BlockId removal: refactor: CallExecutor trait

It changes the arguments of CallExecutor methods:
-  `call`, 'contextual_call', 'runtime_version', 'prove_execution'

from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-01-24 12:52:01 +00:00
57b1de3f47 txpool: LOG_TARGET const added (#13180)
* txpool: LOG_TARGET const added

part of: #12873

* LOG_TARGET added to tests mod

* txpool::api for api

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-01-19 22:21:41 +00:00
Michal KucharczykandAlexander Samusev d85fd6b527 zobmienet tests are not supposed to fail (#13015)
* zobmienet tests are not supposed to fail

* Update scripts/ci/gitlab/pipeline/zombienet.yml

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
2023-01-18 16:26:12 +01:00
5134dabb4a txpool: don't maintain the pool during major sync (#13004)
* txpool: don't maintain the pool during major sync

Fix shall prevent from wasting the CPU during the major sync. No actions
are actually required in transaction pool during the major sync.

Fixes: #12903

* passing sync_oracle to maintain method

* fixed: builder, txpool tests

* do not maintain tx-pool if node gone out of sync

* EnactmentAction: all logic moved to EnactmentState

Tests to be done.

* maintain guard logic moved directly to MaintainedTransactionPool

* minor fixes

* EnactmentAction: all logic moved to EnactmentState (again)

* SyncOracle fixes here and there

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* sync_oracle removed

* spelling + fmt + doc

* Review suggestions applied

* log::info -> debug

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-01-16 23:15:22 +01:00
Michal Kucharczykandparity-processbot 394abf5e58 BlockId removal: refactor: BlockBackend::block|block_status (#2041)
* BlockId removal: refactor: BlockBackend::block|block_status

It changes the arguments of:
-  `BlockBackend::block`
-  `BlockBackend::block_status`

method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>
2023-01-02 12:32:51 +00:00
Michal Kucharczykandparity-processbot 01fcc27465 BlockId removal: refactor: BlockBackend::block|block_status (#6477)
* BlockId removal: refactor: BlockBackend::block|block_status

It changes the arguments of:
-  `BlockBackend::block`
-  `BlockBackend::block_status`

method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"substrate"}

* ".git/.scripts/fmt.sh"

Co-authored-by: parity-processbot <>
2023-01-02 11:56:26 +00:00
9c69fc1b32 BlockId removal: refactor: BlockBackend::block|block_status (#13014)
* BlockId removal: refactor: BlockBackend::block|block_status

It changes the arguments of:
-  `BlockBackend::block`
-  `BlockBackend::block_status`

method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* non-obvious reworks

* doc fix

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: parity-processbot <>
2023-01-02 09:42:05 +00:00
Michal KucharczykandBastian Köcher 8b57471221 txpool: don't validate block transactions if the pool is empty (#12973)
* txpool: don't validate block transactions if the pool is empty

Fix shall prevent from wasting the CPU during the major sync. Block
transaction don't need to be re-validated when the txpool is empty.

Fixes: #12903

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-12-22 22:33:01 +00:00
Michal Kucharczyk a1a9dff12c txpool: TestApi best_block fix (#12992)
This is a minor clean up.

There was a minor bug in TestApi::add_block: the best block flag for
given number was not cleared when the new best block with the same
number was added.

The usage of is_best_block argument in add_block method in txpool tests
were aligned with the BestBlock event.
2022-12-21 16:36:08 +00:00
Michal Kucharczykandparity-processbot 525b0cc032 BlockId removal: refactor: HeaderBackend::status (#2007)
* BlockId removal: refactor: HeaderBackend::status

It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>
2022-12-21 11:52:40 +00:00
ade78c2cf2 BlockId removal: refactor: HeaderBackend::status (#6459)
* BlockId removal: refactor: HeaderBackend::status

It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Update node/core/chain-api/src/tests.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* unused import removed

* update lockfile for {"substrate"}

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: parity-processbot <>
2022-12-21 11:15:59 +00:00
Michal Kucharczykandparity-processbot c237b82690 BlockId removal: refactor: HeaderBackend::status (#12981)
It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

Co-authored-by: parity-processbot <>
2022-12-21 10:25:16 +00:00
Michal Kucharczykandparity-processbot 19ad8c8b38 BlockId removal: refactor: HeaderBackend::header (#1977)
* BlockId removal: refactor: HeaderBackend::header

It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`

methods from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>
2022-12-20 12:46:44 +00:00
Michal Kucharczykandparity-processbot fcc26d42e4 BlockId removal: refactor: HeaderBackend::header (#6418)
* BlockId removal: refactor: HeaderBackend::header

It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`

methods from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* missed fixes

* BlockId removal: refactor: HeaderBackend::expect_header

It changes the arguments of `HeaderBackend::expect_header` method from: `BlockId<Block>` to: `Block::Hash`

* update lockfile for {"substrate"}

* misspell fixed

Co-authored-by: parity-processbot <>
2022-12-20 11:00:13 +00:00
Michal Kucharczykandparity-processbot 548955a73f BlockId removal: refactor: HeaderBackend::header (#12874)
* BlockId removal: refactor: HeaderBackend::header

It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`,
- `PeersClient::header`
- `ChainApi::block_header`

methods from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* non-trivial usages of haeder(block_id) refactored

This may required introduction of dedicated function:
header_for_block_num

* fmt

* fix

* doc fixed

* ".git/.scripts/fmt.sh"

* BlockId removal: refactor: HeaderBackend::expect_header

It changes the arguments of `HeaderBackend::expect_header` method from: `BlockId<Block>` to: `Block::Hash`

* ".git/.scripts/fmt.sh"

* readme updated

* ".git/.scripts/fmt.sh"

* fix

Co-authored-by: parity-processbot <>
2022-12-20 09:43:31 +00:00
Michal KucharczykandBastian Köcher 1be51ccb5c sc-network-test::Peer: block push methods return hashes vec (#12944)
* sc-network-test::Peer: block push methods return hashes vec

This commit reworks the block generation/push methods in
sc-network-test::Peer.

Now methods are providing the vector of hashes that were built.

This allows to get rid of redundant `block_hash_from_id` call, as all
hashes are known just after being built.

Similar approach was taken in BeefyTestNet::generate_blocks_and_sync
method.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* fix

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>
2022-12-19 10:18:32 +00:00
Michal Kucharczyk 225c260e07 zombienet timings adjusted (#12890)
* zombinet tests: add some timeout to allow net spin-up

Sometimes tests are failing at first try, as the pods were not up yet.
Adding timeout should allow the network to spin up properly.

* initial timeout increased to 30s
2022-12-09 12:47:04 +00:00
39cb3b06cd zombienet: warp-sync integration test added (#12675)
* zombienet: warp-sync integration test added

* spelling

* Readme corrected

* dir name updated

* Check second phase of warp sync

* zombienet pipeline enable + naive test network

* zombienet stage added

* paritypr/substrate-debug image added for zombienet testing

* debugs added

* debugs added

* buildah problem fixed

* rollback

* runner tag

* test name corrected

* dir renamed (regex problem)

* common code clean up

* common code clean up

* fix

* warp sync test improvements

* full sha used

as short is too short (https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2051228#L38)

* disable tracing for nodes

* COMMON_USER -> DOCKERIO_USER

* refs reworked

* paritypr/substrate image used

* DOCKERIO -> DOCKER

* generate-ws-db toml cleanup

* improvements

* fix

* raw chain spec used

* zombienet v1.3.18 used

* zombienet: warp sync test enabled

* chain-spec path corrected

* log parsing improved

Checking if log does not container error or verification failed messages

* warp sync test: removed validators

* fix

* review remarks applied

* dir test name changed: 0000_block_building -> 0000-block-building

* transaction finalized test added

* transaction finalized test: error handling improved

* trigger CI job

* trigger CI job

* trigger CI job

* trigger CI job

* Explicitly touch `version.rs` to invalidate the related cache

* zombienet add logs as artifacts

* Revert "Explicitly touch `version.rs` to invalidate the related cache"

This reverts commit 9d00ccfe897a280581156c281961a32665dba6d5.

* file naming changed

Co-authored-by: parity-processbot <>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
2022-12-07 22:44:40 +00:00
Michal Kucharczykandparity-processbot eb4aabf86a BlockId removal: &Hash to Hash (#1818)
* BlockId removal: &Hash to Hash

It changes &Block::Hash argument to Block::Hash.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>
2022-11-07 22:48:44 +00:00
Michal Kucharczykandparity-processbot 8057d45a67 BlockId removal: &Hash to Hash (#6246)
* BlockId removal: &Hash to Hash

It changes &Block::Hash argument to Block::Hash.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* missing file corrected

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-11-07 22:21:06 +00:00
Michal Kucharczyk 1ed70004e7 BlockId removal: &Hash to Hash (#12626)
It changes &Block::Hash argument to Block::Hash.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)
2022-11-07 21:42:16 +00:00
Michal Kucharczyk cf5c78dbe8 txpool: enactment state forced update (#12632)
* txpool: enactment state forced update

When `tree_route` computation fails, we still need to update the
`enactment_state` to be aligned with last known finalized/best block.

We do not execute enactment phase of maintain procedure, but we do
update the state.

* error -> debug

* test added
2022-11-07 21:12:26 +01:00
Michal Kucharczyk 0f17de6cc5 Cargo.lock updated (#1824) 2022-11-07 19:07:55 +00:00
Michal Kucharczykandparity-processbot a87b2e5241 BlockId removal: refactor: Backend::block_indexed_body (#6233)
* BlockId removal: refactor: Backend::block_indexed_body

It changes the arguments of `Backend::block_indexed_body` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-11-03 19:32:32 +00:00
Michal Kucharczyk 11b6409c12 BlockId removal: refactor: Backend::block_indexed_body (#12609)
* BlockId removal: refactor: Backend::block_indexed_body

It changes the arguments of `Backend::block_indexed_body` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* trigger CI job
2022-11-03 16:57:10 +00:00
Michal Kucharczykandparity-processbot a4a5fa8523 BlockId removal: refactor: Backend::justifications (#6229)
* BlockId removal: refactor: Backend::justifications

It changes the arguments of `Backend::justifications` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* formatting

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
2022-11-02 22:59:27 +00:00
Michal KucharczykandAdrian Catangiu 84167bd7d4 BlockId removal: refactor: Backend::justifications (#12602)
* BlockId removal: refactor: Backend::justifications

It changes the arguments of `Backend::justifications` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* trigger CI job

* trigger CI job

* bug fix

* match -> if

Co-authored-by: Adrian Catangiu <adrian@parity.io>

Co-authored-by: Adrian Catangiu <adrian@parity.io>
2022-11-02 22:15:33 +00:00