Commit Graph

162 Commits

Author SHA1 Message Date
Michal Kucharczyk 8ba7a6aba8 chain-spec: getting ready for native-runtime-free world (#1256)
This PR prepares chains specs for _native-runtime-free_  world.

This PR has following changes:
- `substrate`:
  - adds support for:
- JSON based `GenesisConfig` to `ChainSpec` allowing interaction with
runtime `GenesisBuilder` API.
- interacting with arbitrary runtime wasm blob to[
`chain-spec-builder`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/bin/utils/chain-spec-builder/src/lib.rs#L46)
command line util,
- removes
[`code`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/frame/system/src/lib.rs#L660)
from `system_pallet`
  - adds `code` to the `ChainSpec`
- deprecates
[`ChainSpec::from_genesis`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/client/chain-spec/src/chain_spec.rs#L263),
but also changes the signature of this method extending it with `code`
argument.
[`ChainSpec::builder()`](https://github.com/paritytech/substrate/blob/20bee680ed098be7239cf7a6b804cd4de267983e/client/chain-spec/src/chain_spec.rs#L507)
should be used instead.
- `polkadot`:
- all references to `RuntimeGenesisConfig` in `node/service` are
removed,
- all
`(kusama|polkadot|versi|rococo|wococo)_(staging|dev)_genesis_config`
functions now return the JSON patch for default runtime `GenesisConfig`,
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,

- `cumulus`:
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,
- _JSON_ patch configuration used instead of `RuntimeGenesisConfig
struct` in all chain specs.
  
---------

Co-authored-by: command-bot <>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Kevin Krone <kevin@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-11-05 15:19:23 +01:00
Bastian Köcher ca5f10567a sc-block-builder: Remove BlockBuilderProvider (#2099)
The `BlockBuilderProvider` was a trait that was defined in
`sc-block-builder`. The trait was implemented for `Client`. This
basically meant that you needed to import `sc-block-builder` any way to
have access to the block builder. So, this trait was not providing any
real value. This pull request is removing the said trait. Instead of the
trait it introduces a builder for creating a `BlockBuilder`. The builder
currently has the quite fabulous name `BlockBuilderBuilder` (I'm open to
any better name 😅). The rest of the pull request is about
replacing the old trait with the new builder.

# Downstream code changes

If you used `new_block` or `new_block_at` before you now need to switch
it over to the new `BlockBuilderBuilder` pattern:

```rust
// `new` requires a type that implements `CallApiAt`. 
let mut block_builder = BlockBuilderBuilder::new(client)
                // Then you need to specify the hash of the parent block the block will be build on top of
		.on_parent_block(at)
                // The block builder also needs the block number of the parent block. 
                // Here it is fetched from the given `client` using the `HeaderBackend`
                // However, there also exists `with_parent_block_number` for directly passing the number
		.fetch_parent_block_number(client)
		.unwrap()
                // Enable proof recording if required. This call is optional.
		.enable_proof_recording()
                // Pass the digests. This call is optional.
                .with_inherent_digests(digests)
		.build()
		.expect("Creates new block builder");
```

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>
2023-11-03 19:06:31 +01:00
Kian Paimani 35eb133baa Ensure correct variant count in Runtime[Hold/Freeze]Reason (#1900)
closes https://github.com/paritytech/polkadot-sdk/issues/1882

## Breaking Changes

This PR introduces a new item to `pallet_balances::Config`:

```diff
trait Config {
++    type RuntimeFreezeReasons;
}
```

This value is only used to check it against `type MaxFreeze`. A similar
check has been added for `MaxHolds` against `RuntimeHoldReasons`, which
is already given to `pallet_balances`.

In all contexts, you should pass the real `RuntimeFreezeReasons`
generated by `construct_runtime` to `type RuntimeFreezeReasons`. Passing
`()` would also work, but it would imply that the runtime uses no
freezes at all.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-24 12:01:04 +02:00
Michal Kucharczyk 5a2833cceb genesis-builder: implemented for all runtimes (#1492)
This PR implements [`GenesisBuilder`
API](https://github.com/paritytech/polkadot-sdk/blob/a414ea7515c9cdc81f1d12410e646afc148250e8/substrate/primitives/genesis-builder/src/lib.rs#L38)
for all the runtimes in polkadot repo.

Step towards: paritytech/polkadot-sdk#25

---------

Co-authored-by: ordian <write@reusable.software>
2023-09-27 11:59:19 +02:00
Bastian Köcher ecf8035da6 Set StateBackend::Transaction to PrefixedMemoryDB (#14612)
* Yep

* Try to get it working everywhere

* Make `from_raw_storage` start with an empty db

* More fixes!

* Make everything compile

* Fix `child_storage_root`

* Fix after merge

* Cleanups

* Update primitives/state-machine/src/overlayed_changes/mod.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Review comments

* Fix issues

* Silence warning

* FMT

* Clippy

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-08-17 10:49:38 +00:00
Michal Kucharczyk 53cbda1de6 test-runtime: GenesisBuilder runtime API impl + tests (#14310)
* test-runtime: GenesisBuilder runtime API impl + tests

This PR provides implementation of `GenesisBuilder` API for `substrate-test-runtime`, can be considered as reference imiplementation for other runtimes.
The `GenesisBuilder` implementation is gated by `gensis-config` feature.

Tested scenarios:
- default `GenesisConfig` to JSON blob,
- deserialization of `GenesisConfig` from custom JSON, and storing its keys into the Storage (genesis storage creation).
- creation of genesis storage using partial JSON definition,
- checking if invalid/renamed JSON files causes the runtime to panic,

* missing file added

* client: GenesisBuilder helper added

* feature renamed: genesis-config -> genesis-builder

* Update Cargo.toml

* Update Cargo.toml

* Update Cargo.toml

* Update Cargo.toml

* redundant function removed

* genesis builder helper: introduced RuntimeGenesisBuild

* test-runtime: get rid of unused T

* redundant bound removed

* helper: use GenesisBuild again

* tests adjusted for on_genesis

* test-runtime: support for BuildGenesisConfig

* helper: BuildGenesisConfig support

* Update client/genesis-builder/src/lib.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/test_json/readme.md

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/test_json/readme.md

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* jsons are now human-friendly

* fix

* improvements

* jsons fixed

* helper: no_defaults added

* test-runtime: no_defaults added

* test-runtime: patching fn removed

* helper: patching fn removed

* helper: moved to frame_support

* test-runtime: fixes

* Cargo.lock updated

* fmt + naming

* test-runtime: WasmExecutor used

* helper / test-runtime: struct removed

* test-runtime: merge fixes

* Cargo.lock + test-utils/runtime/Cargo.toml updated

* doc fixed

* client/rpc: test fixed (new rt api)

* client/rpc-spec-v2: test fix

* doc fix

* test-runtime: disable-genesis-builder feature

* fix

* fix

* ".git/.scripts/commands/fmt/fmt.sh"

* test-runtime: rerun added to build script

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: parity-processbot <>
2023-07-24 16:12:52 +00:00
André Silva 50a63c30d9 Don't use fixed nominator count for report_equivocation weight calculation (#14471)
* babe: fix report_equivocation weight calculation

* grandpa: fix report_equivocation weight calculation

* beefy: fix report_equivocation weight calculation

* runtime: add missing MaxNominators constant
2023-07-18 14:11:00 +00:00
Juan 6a29a70a92 Replace system config Index for Nonce (#14290)
* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* wip

* remove index in lieu of nonce

* wip

* remove accountnonce in lieu of nonce

* add minor improvement

* rebase and merge conflicts
2023-07-14 06:56:48 +00:00
gupnik 5e7b27e98c Moves Block to frame_system instead of construct_runtime and removes Header and BlockNumber (#14437)
* Initial setup

* Adds node block

* Uses UncheckedExtrinsic and removes Where section

* Updates frame_system to use Block

* Adds deprecation warning

* Fixes pallet-timestamp

* Removes Header and BlockNumber

* Addresses review comments

* Addresses review comments

* Adds comment about compiler bug

* Removes where clause

* Refactors code

* Fixes errors in cargo check

* Fixes errors in cargo check

* Fixes warnings in cargo check

* Formatting

* Fixes construct_runtime tests

* Uses import instead of full path for BlockNumber

* Uses import instead of full path for Header

* Formatting

* Fixes construct_runtime tests

* Fixes imports in benchmarks

* Formatting

* Fixes construct_runtime tests

* Formatting

* Minor updates

* Fixes construct_runtime ui tests

* Fixes construct_runtime ui tests with 1.70

* Fixes docs

* Fixes docs

* Adds u128 mock block type

* Fixes split example

* fixes for cumulus

* ".git/.scripts/commands/fmt/fmt.sh"

* Updates new tests

* Fixes fully-qualified path in few places

* Formatting

* Update frame/examples/default-config/src/lib.rs

Co-authored-by: Juan <juangirini@gmail.com>

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Juan <juangirini@gmail.com>

* ".git/.scripts/commands/fmt/fmt.sh"

* Addresses some review comments

* Fixes build

* ".git/.scripts/commands/fmt/fmt.sh"

* Update frame/democracy/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/democracy/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Addresses review comments

* Updates trait bounds

* Minor fix

* ".git/.scripts/commands/fmt/fmt.sh"

* Removes unnecessary bound

* ".git/.scripts/commands/fmt/fmt.sh"

* Updates test

* Fixes build

* Adds a bound for header

* ".git/.scripts/commands/fmt/fmt.sh"

* Removes where block

* Minor fix

* Minor fix

* Fixes tests

* ".git/.scripts/commands/update-ui/update-ui.sh" 1.70

* Updates test

* Update primitives/runtime/src/traits.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update primitives/runtime/src/traits.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Updates doc

* Updates doc

---------

Co-authored-by: command-bot <>
Co-authored-by: Juan <juangirini@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-07-13 12:01:34 +00:00
Michal Kucharczyk 87d41d0a89 GenesisBuild<T,I> deprecated. BuildGenesisConfig added. (#14306)
* frame::support: GenesisConfig types for Runtime enabled

* frame::support: macro generating GenesisBuild::build for RuntimeGenesisConfig

* frame: ambiguity BuildStorage vs GenesisBuild fixed

* fix

* RuntimeGenesisBuild added

* Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed"

This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293.

* Revert "fix"

This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b.

* Revert "RuntimeGenesisBuild added"

This reverts commit 3c131b618138ced29c01ab8d15d8c6410c9e128b.

* Revert "Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed""

This reverts commit 2b1ecd467231eddec69f8d328039ba48a380da3d.

* Revert "Revert "fix""

This reverts commit fd7fa629adf579d83e30e6ae9fd162637fc45e30.

* Code review suggestions

* frame: BuildGenesisConfig added, BuildGenesis deprecated

* frame: some pallets updated with BuildGenesisConfig

* constuct_runtime: support for BuildGenesisConfig

* frame::support: genesis_build macro supports BuildGenesisConfig

* frame: BuildGenesisConfig added, BuildGenesis deprecated

* Cargo.lock update

* test-runtime: fixes

* Revert "fix"

This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b.

* Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed"

This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293.

* self review

* doc fixed

* ui tests fixed

* fmt

* tests fixed

* genesis_build macrto fixed for non-generic GenesisConfig

* BuildGenesisConfig constraints added

* warning fixed

* some duplication removed

* fmt

* fix

* doc tests fix

* doc fix

* cleanup: remove BuildModuleGenesisStorage

* self review comments

* fix

* Update frame/treasury/src/tests.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Update frame/support/src/traits/hooks.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* doc fix: GenesisBuild exposed

* ".git/.scripts/commands/fmt/fmt.sh"

* frame: more serde(skip) + cleanup

* Update frame/support/src/traits/hooks.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* frame: phantom fields moved to the end of structs

* chain-spec: Default::default cleanup

* test-runtime: phantom at the end

* merge master fixes

* fix

* fix

* fix

* fix

* fix (facepalm)

* Update frame/support/procedural/src/pallet/expand/genesis_build.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* fmt

* fix

* fix

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-07-12 10:22:12 +00:00
Bastian Köcher 5eb816d7a6 Removal of execution strategies (#14387)
* Start

* More work!

* Moar

* More changes

* More fixes

* More worrk

* More fixes

* More fixes to make it compile

* Adds `NoOffchainStorage`

* Pass the extensions

* Small basti making small progress

* Fix merge errors and remove `ExecutionContext`

* Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`

Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
`ExecutionExtension` which provides the default extensions.

* Fix compilation

* Register the global extensions inside runtime api instance

* Fixes

* Fix `generate_initial_session_keys` by passing the keystore extension

* Fix the grandpa tests

* Fix more tests

* Fix more tests

* Don't set any heap pages if there isn't an override

* Fix small fallout

* FMT

* Fix tests

* More tests

* Offchain worker custom extensions

* More fixes

* Make offchain tx pool creation reusable

Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
registered in the runtime externalities context. This factory will be required for a later pr to
make the creation of offchain transaction pools easier.

* Fixes

* Fixes

* Set offchain transaction pool in BABE before using it in the runtime

* Add the `offchain_tx_pool` to Grandpa as well

* Fix the nodes

* Print some error when using the old warnings

* Fix merge issues

* Fix compilation

* Rename `babe_link`

* Rename to `offchain_tx_pool_factory`

* Cleanup

* FMT

* Fix benchmark name

* Fix `try-runtime`

* Remove `--execution` CLI args

* Make clippy happy

* Forward bls functions

* Fix docs

* Update UI tests

* Update client/api/src/execution_extensions.rs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/cli/src/params/import_params.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Pass the offchain storage to the MMR RPC

* Update client/api/src/execution_extensions.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Review comments

* Fixes

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-07-11 14:21:38 +00:00
Bastian Köcher 448989f5ec sp-api: Support nested transactions (#14447)
* sp-api: Support nested transactions

Adds support for nested transactions in `sp-api` by using `execute_in_transaction`. This was working
until a recent refactor, but this was actually not intended. However, supporting nested transactions
is a worthwhile feature to have. So, this pr "brings it back" and adds a test to ensure it will not break.

* Make clippy happy

* Assert that the runtime api type is not unwind safe

* Count number of transactions
2023-06-29 18:01:45 +02:00
Abhijit Roy 4ba78088d3 Update Pallet placeholder struct as per FRAME v2 (#14348)
* Update Pallet placeholder struct as per FRAME v3

* ".git/.scripts/commands/fmt/fmt.sh"

* Update lib.rs

Removed the unused import

---------

Co-authored-by: command-bot <>
2023-06-12 15:22:05 +00:00
Juan 34b2a56af1 Soft deprecate GenesisConfig (#14210)
* soft deprecate genesisconfig

* temporarily add a deprecation attr

* update tests

* update mocks

* update genesis config

* update genesis config

* update genesis config

* update genesis config

* remove deprecation warning

* update deprecation comment

---------

Co-authored-by: parity-processbot <>
2023-05-26 09:42:47 +00:00
Michal Kucharczyk e31a214a85 frame: GenesisBuild::build allowed in no_std (#14107)
* frame: GenesisBuild::build allowed in no_std

i`GenesisBuild::build` function will be required for no_std in no native
runtime world.

`GenesisBuild::build` macro generated function allows to build the runtime
GenesisConfig assembled from all pallets' GenesisConfigs.

* fixes

* GenesisBuild::build avaiable in no-std

- #[cfg(feature = "std")] is not longer added to GenesisBuild implementation.

* system: hash69 available for no-std

* elections-phragmen: panic message fixed for no_std

* frame::suport: doc updated

* test-runtime: default for GenesisConfig

* frame::test-pallet: serde/std added to std feature deps

* Cargo.toml: deps sorted

* Cargo.lock update

cargo update -p frame-support-test-pallet -p frame-support-test

* frame ui tests: cleanup

---------

Co-authored-by: parity-processbot <>
2023-05-25 21:49:25 +00:00
Bastian Köcher 05da6d8e84 HoldReason: Improve usage (#13869)
* HoldReason: Improve usage

`HoldReason` was switched recently to use the `composite_enum` attribute that will merge the enums
from all pallets in the runtime to `RuntimeHoldReason`. `pallet-nis` was still requiring that the
variant was passed as constant to call `hold`. The proper implementation is to use the `HoldReason`
from inside the pallet directly when calling `hold`. This is done by adding a `RuntimeHoldReason` as
type to the `Config` trait and requiring that `Currency` is using the same reason. Besides that the
pr changes the name `HoldIdentifier` in `pallet_balances::Config` to `RuntimeHoldReason`.

* Update frame/nis/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review comment

* Fixes

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-05-24 21:59:34 +00:00
Bastian Köcher 86870ad0f0 test-runtime: Return hashed call as provides in unsigned validation (#14180)
This is required to make different unsigned extrinsics resolve to different transactions in the tx
pool by having `provides` set to theh hash of the call.
2023-05-22 10:50:58 +02:00
Davide Galassi 0b0bef5cb1 Postpone public key creation in the test GenesisStorageBuilder (#14142)
* Postpone public key creation from account id in the test genesis storage builder

* Store raw sr25519 in substrate test pallet

* Nitpick
2023-05-15 13:56:52 +02:00
Michal Kucharczyk 6a295e7c28 substrate-test-runtime migrated to "pure" frame runtime (#13737)
* substrate-test-runtime migrated to pure-frame based

* test block builder: helpers added

* simple renaming

* basic_authorship test adjusted

* block_building storage_proof test adjusted

* babe: tests: should_panic expected added

* babe: tests adjusted

ConsensusLog::NextEpochData is now added by pallet_babe as
pallet_babe::SameAuthoritiesForever trigger is used in runtime config.

* beefy: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digests stored during block execution are
checked against header digests:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(MmrRoot / AuthoritiesChange) is used.

* grandpa: tests adjusted

test-substrate-runtime is now using frame::executive to finalize the
block. during finalization the digest logs stored during block execution are
checked against header digest logs:
https://github.com/paritytech/substrate/blob/91bb2d29ca905599098a5b35eaf24867c4fbd60a/frame/executive/src/lib.rs#L585-L591

It makes impossible to directly manipulate header's digets, w/o
depositing logs into system pallet storage `Digest<T: Config>`.

Instead of this dedicated extrinsic allowing to store logs items
(ScheduledChange / ForcedChange and DigestItem::Other) is used.

* network:bitswap: test adjusted

The size of unchecked extrinsic was increased. The pattern used in test will
be placed at the end of scale-encoded buffer.

* runtime apis versions adjusted

* storage keys used in runtime adjusted

* wasm vs native tests removed

* rpc tests: adjusted

Transfer transaction processing was slightly improved, test was
adjusted.

* tests: sizes adjusted

Runtime extrinsic size was increased. Size of data read during block
execution was also increased due to usage of new pallets in runtime.

Sizes were adjusted in tests.

* cargo.lock update

cargo update -p substrate-test-runtime -p substrate-test-runtime-client

* warnings fixed

* builders cleanup: includes / std

* extrinsic validation cleanup

* txpool: benches performance fixed

* fmt

* spelling

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Apply code review suggestions

* Apply code review suggestions

* get rid of 1063 const

* renaming: UncheckedExtrinsic -> Extrinsic

* test-utils-runtime: further step to pure-frame

* basic-authorship: tests OK

* CheckSubstrateCall added + tests fixes

* test::Transfer call removed

* priority / propagate / no sudo+root-testing

* fixing warnings + format

* cleanup: build2/nonce + format

* final tests fixes

all tests are passing

* logs/comments removal

* should_not_accept_old_signatures test removed

* make txpool benches work again

* Cargo.lock reset

* format

* sudo hack removed

* txpool benches fix+cleanup

* .gitignore reverted

* rebase fixing + unsigned cleanup

* Cargo.toml/Cargo.lock cleanup

* force-debug feature removed

* mmr tests fixed

* make cargo-clippy happy

* network sync test uses unsigned extrinsic

* cleanup

* ".git/.scripts/commands/fmt/fmt.sh"

* push_storage_change signed call remove

* GenesisConfig cleanup

* fix

* fix

* GenesisConfig simplified

* storage_keys_works: reworked

* storage_keys_works: expected keys in vec

* storage keys list moved to substrate-test-runtime

* substrate-test: some sanity tests + GenesisConfigBuilder rework

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

* Review suggestions

* fix

* fix

* beefy: generate_blocks_and_sync block_num sync with actaul value

* Apply suggestions from code review

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* cargo update -p sc-rpc -p sc-transaction-pool

* Review suggestions

* fix

* doc added

* slot_duration adjusted for Babe::slot_duration

* small doc fixes

* array_bytes::hex used instead of hex

* tiny -> medium name fix

* Apply suggestions from code review

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* TransferData::try_from_unchecked_extrinsic -> try_from

* Update Cargo.lock

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-05-04 16:20:22 +00:00
yjh d5e460b3bf refactor(sc-executor): use wasm executor builder instead of old apis (#13740)
* refactor: use builder api for all executors

* improve a lot

* remove unused args

* cleanup deps

* fix inconsistency about heap alloc

* add `heap_pages` back to try-runtime

* fix

* chore: reduce duplicated code for sc-service-test

* cleanup code

* fmt

* improve test executor

* improve

* use #[deprecated]

* set runtime_cache_size: 4

* fix and improve

* refactor builder

* fix

* fix bench

* fix tests

* fix warnings

* fix warnings

* fix

* fix

* update by suggestions

* update name
2023-04-09 23:48:40 +00:00
Bastian Köcher fa8e323231 ProofRecorder: Implement transactional support (#13769)
* TrieRecorder: Start adding support for transactions

* Adds `transactions` functions and some test

* More tests

* Docs

* Ensure that we rollback failed transactions in the storage proof

* FMT

* Update primitives/trie/src/recorder.rs

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

* Review comments

* Update primitives/trie/src/recorder.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* ".git/.scripts/commands/fmt/fmt.sh"

* For the holy clippy!

* Update primitives/trie/src/recorder.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Anton <anton.kalyaev@gmail.com>
2023-04-05 14:43:17 +00:00
Alexandru Vasile 1a88833d73 Metadata V15: Expose API to fetch metadata for version (#13287)
* impl_runtime_apis: Generate getters for `metadata_at` functions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime: Implement new `Metadata` runtime trait

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* runtime: Move `metadata_at` functions to construct_runtime macro

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* contruct_runtime: Use `OpaqueMetadata` from hidden imports

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Add tests for the new API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Adjust metdata naming

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose `metadata-v14` feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose metadata only under feature flags

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose v14 metadata by default

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Expose metadata feature for testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Test metadata under different feature flags

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* client/tests: Adjust testing to reflect trait Metadata change

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Add intermediate representation types for metadata

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Convert metadata to V14

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Add API to convert metadata to multiple versions

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata-ir: Expose V14 under feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Adjust to metadata IR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: More adjustments

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Guard v14 details under feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* CI: Ensure `quick-benchmarks` uses `metadata-v14`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Use `metadata-v14` for benchmarks

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Adjust cargo fmt

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* kitchensink-runtime: Add feature flag for `metadata-v14`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support/test: Adjust testing

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support/test: Check crates locally

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Activate metadata-v14 for pallets

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Remove metadata-v14 feature flag

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/metadata_ir: Move `api.rs` to `mod.rs`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Handle latest metadata conversion via IR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Add constant for metadata version 14

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support/test: Fix merge conflict

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update frame/support/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/src/metadata_ir/mod.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/test/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* frame/metadata: Collect pallet documentation for MetadataIR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/tests: Check pallet documentation is propagated to MetadataIR

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/support: Improve documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-03-15 17:49:28 +00:00
Vivek Pandya 2009821cde Remove use of trait Store from all pallets and deprecate it. (#13535)
* Remove use of trait Store from staking pallet

* Remove use of trait Store from bounties pallet

* Remove use of trait Store from collective pallet

* Remove use of trait Store from babe pallet

* Remove use of trait Store from assets pallet

* Remove use of trait Store from grandpa pallet

* Remove use of trait Store from balances pallet

* Remove use of trait Store from authorship pallet

* Remove use of trait Store from authority-discovery pallet

* Remove use of trait Store from atomic-swap pallet

* Remove use of trait Store from sudo pallet

* Remove use of trait Store from scheduler pallet

* Remove use of trait Store from scored-pool pallet

* Remove use of trait Store from society pallet

* Remove use of trait Store from lottery pallet

* Remove use of trait Store from executive pallet

* Remove use of trait Store from democracy pallet

* Remove use of trait Store from elections-phragmen pallet

* Remove use of trait Store from indices pallet

* Remove use of trait Store from identity pallet

* Remove use of trait Store from multisig pallet

* Remove use of trait Store from merkle-mountain-range pallet

* Remove use of trait Store from im-online pallet

* Remove use of trait Store from membership pallet

* Remove use of trait Store from nicks pallet

* Remove use of trait Store from session pallet

* Remove use of trait Store from transaction-payment pallet

* Remove use of trait Store from utility pallet

* Remove use of trait Store from child-bounties pallet

* Remove use of trait Store from nis pallet

* Remove use of trait Store from nfts pallet

* Remove use of trait Store from conviction-voting pallet

* Remove use of trait Store from treasury pallet

* Remove use of trait Store from vesting pallet

* Remove use of trait Store from preimage pallet

* Remove use of trait Store from uniques pallet

* Remove use of trait Store from ranked-collective pallet

* Remove use of trait Store from beefy-mmr pallet

* Remove use of trait Store from referenda pallet

* Remove use of trait Store from whitelist pallet

* Remove use of trait Store from alliance pallet

* Remove use of trait Store from nomination-pools pallet

* Remove use of trait Store from state-trie-migration pallet

* Remove use of trait Store from message-queue pallet

* Remove use of trait Store from root-offences pallet

* Remove use of trait Store from root-testing pallet

* Remove use of trait Store from timestamps pallet

* Remove use of trait Store from system pallet

* Remove use of trait Store from offences pallet

* Remove use of trait Store from recovery pallet

* Remove use of trait Store from node-authorization pallet

* Remove use of trait Store from proxy pallet

* Remove use of trait Store from benchmarking pallet

* Remove use of trait Store from bags-list pallet

* Add deprecated warning in store_trait

* Change warning message

* Run cargo fmt

* Fix warning and update tests

* Remove unnecessary allow deprecated

* Remove use of trait Store

* Fix mismatch in expected output

* Minor update to warning message for deprecation of generate_store with Store trait attribute

* Fixes as per review comments

* Fixes as per review suggestions

* Remove use of Store trait from core-fellowship pallet

* Fix type in store_trait.rs

* Fixes as pre review comment
2023-03-13 15:30:05 +00:00
Davide Galassi 9dd10d131d Offences report system rework (#13425)
* Experiments with common equivocation trait

* Improved equivocation trait

* Fix grandpa equivocation implementation

* Remove some cruft

* Remove some more cruft

* More generic naming

* Simplification of offences manipilation

* More refactory

* Some prograss with the encapsulation of offence report system

* Finally unit type works as a universal null report system

* Align substrate node code

* Further simplification

* Fix test utils

* Remove not required associated type

* Fix benches

* Rollback to prev field name

* Box big params

* Fix typo

* Remove new tag computation

* Remove default implementations

* Better docs

* Return 'Result' instead of bool

* Change offence report system return types

* Some renaming and documentation

* Improve documentation

* More abstract offence report system

* Rename 'consume_evidence' to 'process_evidence'

* Further docs refinements

* Doc for dummy offence report

* Fix rustdoc

* Fix after master merge

* Apply code review suggestions

* Improve docs
2023-03-07 20:25:55 +00:00
Oliver Tale-Yazdi 9e56e1acdd Deprecate Weight::from_{ref_time, proof_size} (#13475)
* Deprecate Weight::from_{ref_time, proof_size}

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update templates

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use from_parts

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use from_parts

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Dont revert comment 🤦

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_balances

* Update weight files

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Adapt to Master changes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
2023-03-02 21:28:17 +00:00
yjh 20bf3c938e chore: move genesis block builder to chain-spec crate. (#13427)
* chore: move genesis block builder to block builder crate.

* add missing file

* chore: move genesis block builder to sc-chain-spec

* Update client/chain-spec/src/genesis.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update test-utils/runtime/src/genesismap.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update test-utils/runtime/client/src/lib.rs

* fix warnings

* fix warnings

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-02-28 20:50:57 +01:00
Davide Galassi 1ef9c473e7 Move BEEFY code to consensus (#13484)
* Move beefy primitives to consensus dir
* Move beefy gadget to client consensus folder
* Rename beefy crates
2023-02-28 15:56:22 +01:00
Davide Galassi 40c36c0c8a Move grandpa crates to consensus folder (#13458)
* Move grandpa under consensus dir
* Rename grandpa folder
* Finish grandpa renaming
* Minor tweaks
* Cargo fmt
* Adjust path to chain spec
2023-02-27 17:15:08 +01:00
Bastian Köcher 941288c6d0 wasm-executor: Support growing the memory (#12520)
* As always, start with something :P

* Add support for max_heap_pages

* Add support for wasmtime

* Make it compile

* Fix compilation

* Copy wrongly merged code

* Fix compilation

* Some fixes

* Fix

* Get stuff working

* More work

* More fixes

* ...

* More

* FIXEs

* Switch wasmi to use `RuntimeBlob` like wasmtime

* Removed unused stuff

* Cleanup

* More cleanups

* Introduce `CallContext`

* Fixes

* More fixes

* Add builder for creating the `WasmExecutor`

* Adds some docs

* FMT

* First round of feedback.

* Review feedback round 2

* More fixes

* Fix try-runtime

* Update client/executor/wasmtime/src/instance_wrapper.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/executor/common/src/wasm_runtime.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/executor/common/src/runtime_blob/runtime_blob.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/executor/common/src/wasm_runtime.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/allocator/src/freeing_bump.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/allocator/src/freeing_bump.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Feedback round 3

* FMT

* Review comments

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-02-24 11:43:01 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Michal Kucharczyk 7a10154188 BlockId removal: runtime-api refactor (#13255)
* BlockId removal: refactor of runtime API

It changes the arguments of:
- `ApiExt` methods:  `has_api`, `has_api_with`, `api_version`
- `CallApiAt` method: `runtime_version_at`
from: `BlockId<Block>` to: `Block::Hash`

It also changes the first argument of all generated runtime API calls from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* BlockId removal: refactor of runtime API - tests

- tests adjusted to new runtime API,
- some tests migrated from block number to block hash

* benchmarking-cli: BlockId(0) migrated to info().genesis_hash

`runtime_api.call()` now requires the block hash instead of BlockId::Number.
To access the genesis hash widely used in benchmarking engine the Client
was constrained to satisfy `sp_blockchain::HeaderBackend<Block>` trait
which provides `info().genesis_hash`.

* trivial: api.call(BlockId) -> api.call(Hash)

- Migrated all `runtime_api.calls` to use Hash
- Noteworthy (?):
-- `validate_transaction_blocking` in transaction pool,

* CallApiAtParams::at changed to Block::Hash

* missed doc updated

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

* BlockId removal: Benchmark::consumed_weight

Little refactor around `Benchmark::consumed_weight`: `BlockId` removed.

* at_hash renamed

* wrong merge fixed

* beefy worker: merged with master

* beefy: tests: missing block problem fixed

* Apply review suggestion

* fix

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
2023-02-20 22:47:21 +00:00
Adrian Catangiu c21f292a02 BEEFY: implement equivocations detection, reporting and slashing (#13121)
* client/beefy: simplify self_vote logic

* client/beefy: migrate to new state version

* client/beefy: detect equivocated votes

* fix typos

* sp-beefy: add equivocation primitives

* client/beefy: refactor vote processing

* fix version migration for new rounds struct

* client/beefy: track equivocations and create proofs

* client/beefy: adjust tests for new voting logic

* sp-beefy: fix commitment ordering and equality

* client/beefy: simplify handle_vote() a bit

* client/beefy: add simple equivocation test

* client/beefy: submit equivocation proof - WIP

* frame/beefy: add equivocation report runtime api - part 1

* frame/beefy: report equivocation logic - part 2

* frame/beefy: add pluggable Equivocation handler - part 3

* frame/beefy: impl ValidateUnsigned for equivocations reporting

* client/beefy: submit report equivocation unsigned extrinsic

* primitives/beefy: fix tests

* frame/beefy: add default weights

* frame/beefy: fix tests

* client/beefy: fix tests

* frame/beefy-mmr: fix tests

* frame/beefy: cross-check session index with equivocation report

* sp-beefy: make test Keyring useable in pallet

* frame/beefy: add basic equivocation test

* frame/beefy: test verify equivocation results in slashing

* frame/beefy: test report_equivocation_old_set

* frame/beefy: add more equivocation tests

* sp-beefy: fix docs

* beefy: simplify equivocations and fix tests

* client/beefy: address review comments

* frame/beefy: add ValidateUnsigned to test/mock runtime

* client/beefy: fixes after merge master

* fix missed merge damage

* client/beefy: add test for reporting equivocations

Also validated there's no unexpected equivocations reported in the
other tests.

Signed-off-by: acatangiu <adrian@parity.io>

* sp-beefy: move test utils to their own file

* client/beefy: add negative test for equivocation reports

* sp-beefy: move back MmrRootProvider - used in polkadot-service

* impl review suggestions

* client/beefy: add equivocation metrics

---------

Signed-off-by: acatangiu <adrian@parity.io>
Co-authored-by: parity-processbot <>
2023-02-17 09:45:00 +00:00
Adrian Catangiu 981ffb29f5 BEEFY: define on-chain beefy-genesis and use it to coordinate voter initialization (#13215)
* beefy: add support to configure BEEFY genesis

* client/beefy: more flexible test runtime api

* client/beefy: add tests for custom BEEFY genesis

* client/beefy: ignore old state that didn't account for pallet genesis

* client/beefy: fix clippy

* frame/beefy: default BEEFY-genesis is block One::one()

* frame/beefy: add extra doc comments

---------

Co-authored-by: parity-processbot <>
2023-02-03 10:16:16 +00:00
Alexandru Vasile 4b2cc67fc6 Metadata V15: Derive TypeInfo for describing runtime types (#13272)
* scale_info: Derive TypeInfo for types present in runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cargo: Update Cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-02-02 19:14:40 +02:00
Davirain 25388b267e Move beefy-merkle-tree to utils/binary-merkle-tree and make it generic (#13076)
* move BeefyMmrApi to pallet-beefy-mmr

* fix test_utils use pallet-beefy-mmr BeefyMmrApi

* Move beefy-merkle-tree to utils and Rename to Merkle-tree

* fix fmt and test

* Update merkle-tree to binary-merkle-tree and Remove Keccak256 mod from merkle-tree

* change merkle-tree name to binary-merkle-tree

* mirr fix
2023-02-02 10:07:48 +00:00
Marcin S aa21e56744 Remove mem_info and references to parity-util-mem (#12795)
* Remove mem_info and some references to parity-util-mem

* [Draft] Finish removing references to `parity-util-mem`

* Upgrade dependencies

* Update scripts/ci/deny.toml

Co-authored-by: ordian <write@reusable.software>

* Fix Cargo.lock (remove unwanted dependency changes)

* Removed unused argument

* Run cargo fmt (didn't have pre-commit set up)

* Fix some CI errors

* Fix another CI error

* Remove unused dependency

Co-authored-by: ordian <write@reusable.software>
2022-12-06 14:55:10 +00:00
Serban Iorga ff439ee335 pallet-mmr: move offchain logic to client-side gadget (#12753)
* Move MMR utils methods from pallet to primitives

Signed-off-by: Serban Iorga <serban@parity.io>

* Add method to MmrApi

* Move forks expanding logic from babe to primitives

* Implement MMR gadget

* Remove prunning logic from the MMR pallet

* Code review changes: 1st iteration

* Replace MaybeCanonEngine with CanonEngineBuilder

* fix mmr_leaves_count() for kitchen sink demo

* Update client/merkle-mountain-range/src/canon_engine.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Code review changes: 2nd iteration

* fix INDEXING_PREFIX

* impl review comments

* add documentation and minor rename

Signed-off-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
2022-11-29 14:39:52 +00:00
Bastian Köcher 0c934a9352 ed25519_verify: Support using dalek for historical blocks (#12661)
* ed25519_verify: Support using dalek for historical blocks

The switch from `ed25519-dalek` to `ed25519-zebra` was actually a breaking change. `ed25519-zebra`
is more permissive. To support historical blocks when syncing a chain this pull request introduces
an externalities extension `UseDalekExt`. This extension is just used as a signaling mechanism to
`ed25519_verify` to use `ed25519-dalek` when it is present. Together with `ExtensionBeforeBlock` it
can be used to setup a node in way to sync historical blocks that require `ed25519-dalek`, because
they included a transaction that verified differently as when using `ed25519-zebra`.

This feature can be enabled in the following way. In the chain service file, directly after the
client is created, the following code should be added:

```
use sc_client_api::ExecutorProvider;
client.execution_extensions().set_extensions_factory(
	sc_client_api::execution_extensions::ExtensionBeforeBlock::<Block, sp_io::UseDalekExt>::new(BLOCK_NUMBER_UNTIL_DALEK_SHOULD_BE_USED)
);
```

* Fix doc

* More fixes

* Update client/api/src/execution_extensions.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix merge and warning

* Fix docs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2022-11-27 15:34:07 +00:00
Michal Kucharczyk 09748f1b28 BlockId::Number refactor: trivial changes to BlockId::Hash (#12471)
* Trivial BlockId::Number => Hash

* missed BlockId::Hash added
2022-10-12 16:10:31 +00:00
Gavin Wood 93e8ffed55 Bound uses of Call (#11649)
* Introduce preimages module in traits

* Multisize Preimages

* Len not actually necessary

* Tweaks to the preimage API

* Fixes

* Get Scheduler building with new API

* Scheduler tests pass

* Bounded Scheduler 🎉

* Use Agenda holes and introduce IncompleteSince to avoid need to reschedule

* Tests pass with new weight system

* New benchmarks

* Add missing file

* Drop preimage when permenantly overeight

* Drop preimage when permenantly overeight

* Referenda uses latest preimage API

* Testing ok

* Adding tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add preimage migration

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

* Remove dbg

* Refactor Democracy

* Refactor Democracy

* Add final MEL

* Remove silly maps

* Fixes

* Minor refactor

* Formatting

* Fixes

* Fixes

* Fixes

* Update frame/preimage/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Add migrations to Democracy

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Resolve conflicts

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Resolve conflicts"

This reverts commit 734d66d69e54553471ffa54fa52e3e304dc8f106.

* Undo wrong resolves...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* massage clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy annoyance

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy annoyance

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add missing file

* Test <Preimage as QueryPreimage>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy harassment

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fixup tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove old stuff

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test <Scheduler as Anon> trait functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update pallet-ui tests

Why is this needed? Should not be the case unless master is broken...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More scheduler trait test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Apply review suggestion

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Beauty fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add Scheduler test migration_v3_to_v4_works

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Merge fixup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Keep referenda benchmarks instantiatable

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new scheduler weight functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new democracy weight functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use weight compare functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update pallet-ui tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More renaming…

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More renaming…

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add comment

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Implement OnRuntimeUpgrade for scheduler::v3_to_v4 migration

Put the migration into a proper `MigrateToV4` struct and implement
the OnRuntimeUpgrade hooks for it. Also move the test to use that
instead.

This should make it easier for adding it to Polkadot.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Handle undecodable Agendas

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove trash

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new OnRuntimeUpgrade functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix BoundedSlice::truncate_from

Co-authored-by: jakoblell

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix pre_upgrade hook return values

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more error logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Find too large preimages in the pre_upgrade hook

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test that too large Calls in agendas are ignored

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use new OnRuntimeUpgrade hooks

Why did the CI not catch this?!

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* works fine - just more logs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix staking migration

Causing issues on Kusama...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix UI tests

No idea why this is needed. This is actually undoing an earlier change.
Maybe the CI has different rustc versions!?

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove multisig's Calls (#12072)

* Remove multisig's Calls

* Multisig: Fix tests and re-introduce reserve logic (#12241)

* Fix tests and re-introduce reserve logic

* fix benches

* add todo

* remove irrelevant bench

* [Feature] Add a migration that drains and refunds stored calls (#12313)

* [Feature] Add a migration that drains and refunds stored calls

* migration fixes

* fixes

* address review comments

* consume the whole block weight

* fix assertions

* license header

* fix interface

Co-authored-by: parity-processbot <>

Co-authored-by: parity-processbot <>
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix multisig benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_democracy

* ".git/.scripts/bench-bot.sh" pallet dev pallet_scheduler

* ".git/.scripts/bench-bot.sh" pallet dev pallet_preimage

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: parity-processbot <>
Co-authored-by: Roman Useinov <roman.useinov@gmail.com>
2022-10-05 19:21:37 +01:00
Qinxuan Chen a395fec070 Migrate remaining old decl_* macros to the new pallet attribute macros (#12271)
* Migrate remaining old decl_* macros to the new pallet attribute macros

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Apply review suggestions

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Apply review suggestions

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* use pallet::storage

* Fix dev rpc test

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Fix service tests

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2022-09-22 08:53:51 +00:00
Sergej Sakac e4b6f4a66d BREAKING: Rename Origin (#12258)
* BREAKING: Rename Origin

* more renaming

* a bit more renaming

* fix

* more fixing

* fix in frame_support

* even more fixes

* fix

* small fix

* ...

* update .stderr

* docs

* update docs

* update docs

* docs
2022-09-20 22:13:09 +00:00
Sergej Sakac 6e8795afe6 BREAKING: Rename Call & Event (#11981)
* rename Event to RuntimeEvent

* rename Call

* rename in runtimes

* small fix

* rename Event

* small fix & rename RuntimeCall back to Call for now

* small fixes

* more renaming

* a bit more renaming

* fmt

* small fix

* commit

* prep for renaming associated types

* fix

* rename associated Event type

* rename to RuntimeEvent

* commit

* merge conflict fixes & fmt

* additional renaming

* fix.

* fix decl_event

* rename in tests

* remove warnings

* remove accidental rename

* .

* commit

* update .stderr

* fix in test

* update .stderr

* TRYBUILD=overwrite

* docs

* fmt

* small change in docs

* rename PalletEvent to Event

* rename Call to RuntimeCall

* renamed at wrong places :P

* rename Call

* rename

* rename associated type

* fix

* fix & fmt

* commit

* frame-support-test

* passing tests

* update docs

* rustdoc fix

* update .stderr

* wrong code in docs

* merge fix

* fix in error message

* update .stderr

* docs & error message

* .

* merge fix

* merge fix

* fmt

* fmt

* merge fix

* more fixing

* fmt

* remove unused

* fmt

* fix

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-09-12 22:03:31 +00:00
Bastian Köcher b356a5589a Remove native call (#12201)
* Remove native call

With the recent introduction of staging runtime apis the native call wasn't supported anymore. This
removes the entire support for this as it is not used anymore.

* FMT

* Fix benchmarks

* FIX ui tests
2022-09-12 12:25:56 +02:00
Davide Galassi 83aec6f93a Fetch Babe configuration from runtime state (#11760)
* Fetch babe config data from runtime state

* Some renaming

* More renaming

* Final nits

* Fix tests and benches

* Rename  to  in BabeConfiguration

* Remove duplicate babe parameter description

Already specified over the 'PRIMARY_PROBABILITY' constant value

* trigger pipeline

* trigger pipeline
2022-09-05 17:41:32 +00:00
Shawn Tabrizi 30951822ba Weight v1.5: Opaque Struct (#12138)
* initial idea

* update frame_support

* update a bunch more

* add ord

* adjust RuntimeDbWeight

* frame_system builds

* re-export

* frame_support tests pass

* frame_executive compile

* frame_executive builds

* frame_system tests passing

* pallet-utility tests pass

* fix a bunch of pallets

* more

* phragmen

* state-trie-migration

* scheduler and referenda

* pallet-election-provider-multi-phase

* aura

* staking

* more

* babe

* balances

* bunch more

* sudo

* transaction-payment

* asset-tx-payment

* last pallets

* fix alliance merge

* fix node template runtime

* fix pallet-contracts cc @athei

* fix node runtime

* fix compile on runtime-benchmarks feature

* comment

* fix frame-support-test

* fix more tests

* weight regex

* frame system works

* fix a bunch

* more

* more

* more

* more

* more

* more fixes

* update templates

* fix contracts benchmarks

* Update lib.rs

* Update lib.rs

* fix ui

* make scalar saturating mul const

* more const functions

* scalar div

* refactor using constant functions

* move impl

* fix overhead template

* use compactas

* Update lib.rs
2022-08-31 11:26:13 +00:00
Bastian Köcher 73d9ae3284 Introduce trie level cache and remove state cache (#11407)
* trie state cache

* Also cache missing access on read.

* fix comp

* bis

* fix

* use has_lru

* remove local storage cache on size 0.

* No cache.

* local cache only

* trie cache and local cache

* storage cache (with local)

* trie cache no local cache

* Add state access benchmark

* Remove warnings etc

* Add trie cache benchmark

* No extra "clone" required

* Change benchmark to use multiple blocks

* Use patches

* Integrate shitty implementation

* More stuff

* Revert "Merge branch 'master' into trie_state_cache"

This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff036463.

* Improve benchmark

* Adapt to latest changes

* Adapt to changes in trie

* Add a test that uses iterator

* Start fixing it

* Remove obsolete file

* Make it compile

* Start rewriting the trie node cache

* More work on the cache

* More docs and code etc

* Make data cache an optional

* Tests

* Remove debug stuff

* Recorder

* Some docs and a simple test for the recorder

* Compile fixes

* Make it compile

* More fixes

* More fixes

* Fix fix fix

* Make sure cache and recorder work together for basic stuff

* Test that data caching and recording works

* Test `TrieDBMut` with caching

* Try something

* Fixes, fixes, fixes

* Forward the recorder

* Make it compile

* Use recorder in more places

* Switch to new `with_optional_recorder` fn

* Refactor and cleanups

* Move `ProvingBackend` tests

* Simplify

* Move over all functionality to the essence

* Fix compilation

* Implement estimate encoded size for StorageProof

* Start using the `cache` everywhere

* Use the cache everywhere

* Fix compilation

* Fix tests

* Adds `TrieBackendBuilder` and enhances the tests

* Ensure that recorder drain checks that values are found as expected

* Switch over to `TrieBackendBuilder`

* Start fixing the problem with child tries and recording

* Fix recording of child tries

* Make it compile

* Overwrite `storage_hash` in `TrieBackend`

* Add `storage_cache` to  the benchmarks

* Fix `no_std` build

* Speed up cache lookup

* Extend the state access benchmark to also hash a runtime

* Fix build

* Fix compilation

* Rewrite value cache

* Add lru cache

* Ensure that the cache lru works

* Value cache should not be optional

* Add support for keeping the shared node cache in its bounds

* Make the cache configurable

* Check that the cache respects the bounds

* Adds a new test

* Fixes

* Docs and some renamings

* More docs

* Start using the new recorder

* Fix more code

* Take `self` argument

* Remove warnings

* Fix benchmark

* Fix accounting

* Rip off the state cache

* Start fixing fallout after removing the state cache

* Make it compile after trie changes

* Fix test

* Add some logging

* Some docs

* Some fixups and clean ups

* Fix benchmark

* Remove unneeded file

* Use git for patching

* Make CI happy

* Update primitives/trie/Cargo.toml

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update primitives/state-machine/src/trie_backend.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Introduce new `AsTrieBackend` trait

* Make the LocalTrieCache not clonable

* Make it work in no_std and add docs

* Remove duplicate dependency

* Switch to ahash for better performance

* Speedup value cache merge

* Output errors on underflow

* Ensure the internal LRU map doesn't grow too much

* Use const fn to calculate the value cache element size

* Remove cache configuration

* Fix

* Clear the cache in between for more testing

* Try to come up with a failing test case

* Make the test fail

* Fix the child trie recording

* Make everything compile after the changes to trie

* Adapt to latest trie-db changes

* Fix on stable

* Update primitives/trie/src/cache.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Fix wrong merge

* Docs

* Fix warnings

* Cargo.lock

* Bump pin-project

* Fix warnings

* Switch to released crate version

* More fixes

* Make clippy and rustdocs happy

* More clippy

* Print error when using deprecated `--state-cache-size`

* 🤦

* Fixes

* Fix storage_hash linkings

* Update client/rpc/src/dev/mod.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Review feedback

* encode bound

* Rework the shared value cache

Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.

* Improve of the cache by merging the old and new solution

* FMT

* Please stop coming back all the time :crying:

* Update primitives/trie/src/cache/shared_cache.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Fixes

* Make clippy happy

* Ensure we don't deadlock

* Only use one lock to simplify the code

* Do not depend on `Hasher`

* Fix tests

* FMT

* Clippy 🤦

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
2022-08-18 18:59:22 +00:00
Sacha Lansky a37226d09e Fix typos (#11914) 2022-07-26 00:09:50 +02:00
Adrian Catangiu 3018051cdd pallet-beefy-mmr: add API for BEEFY Authority Sets (#11406)
* pallet-beefy: add Config::OnNewValidatorSet type

Add a hook to pallet-beefy for doing specific work when
BEEFY validator set changes.

For example, this can be used by pallet-beefy-mmr to cache
a lightweight MMR root over validators and make it available
to light clients.

* pallet-beefy-mmr: implement OnNewValidatorSet

Implement pallet-beefy::OnNewValidatorSet to be notified of BEEFY
validator set changes. Use the notifications to compute and cache
a light weight 'BEEFY authority set' which is an MMR root over
BEEFY validator set plus some extra info.

Previously, pallet-beefy-mmr was interogating pallet-beefy about
validator set id on every block to find out when it needs to recompute
the authority set.
By using the event-driven approach in this commit, we also save one
extra state interogation per block.

* pallet-beefy-mmr: add new authority_set() API

Expose current and next BEEFY authority sets through runtime API.
These can be directly used by light clients to avoid having them
compute them themselves based on BEEFY validator sets.

Signed-off-by: acatangiu <adrian@parity.io>

* rename BeefyMmr exposed runtime api
2022-06-22 18:39:24 +00:00
Gavin Wood c47431118b MEL: Origin, Referenda, ConvictionVoting (#11631)
* Referenda & CV pallets ready

* Fix build

* Add mel_bound for Voting and Casting types

* Add mel_bound on Tally

* Add mel_bound on another Tally

* Add mel_bound for pallet_collective::RawOrigin

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2022-06-16 16:13:17 +00:00