Commit Graph

147 Commits

Author SHA1 Message Date
PG Herveou d38f6e6728 Update benchmarking macros (#3934)
Current benchmarking macro returns a closure with the captured
benchmarked code.
This can cause issues when the benchmarked code has complex lifetime
requirements.

This PR updates the existing macro by injecting the recording parameter
and invoking the start / stop method around the benchmarked block
instead of returning a closure

One other added benefit is that you can write this kind of code now as
well:

```rust
let v;
#[block]
{ v = func.call(); }
dbg!(v); // or assert something on v
```


[Weights compare
link](https://weights.tasty.limo/compare?unit=weight&ignore_errors=true&threshold=10&method=asymptotic&repo=polkadot-sdk&old=pg/fix-weights&new=pg/bench_update&path_pattern=substrate/frame/**/src/weights.rs,polkadot/runtime/*/src/weights/**/*.rs,polkadot/bridges/modules/*/src/weights.rs,cumulus/**/weights/*.rs,cumulus/**/weights/xcm/*.rs,cumulus/**/src/weights.rs)

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
2024-04-10 06:44:46 +00:00
Alexandru Gheorghe 8d0cd4ffc8 Fix kusama validators getting 0 backing rewards the first session they enter the active set (#3722)
There is a problem in the way we update `authorithy-discovery` next keys
and because of that nodes that enter the active set would be noticed at
the start of the session they become active, instead of the start of the
previous session as it was intended. This is problematic because:

1. The node itself advertises its addresses on the DHT only when it
notices it should become active on around ~10m loop, so in this case it
would notice after it becomes active.
2. The other nodes won't be able to detect the new nodes addresses at
the beginning of the session, so it won't added them to the reserved
set.

With 1 + 2, we end-up in a situation where the the new node won't be
able to properly connect to its peers because it won't be in its peers
reserved set. Now, the nodes accept by default`MIN_GOSSIP_PEERS: usize =
25` connections to nodes that are not in the reserved set, but given
Kusama size(> 1000 nodes) you could easily have more than`25` new nodes
entering the active set or simply the nodes don't have slots anymore
because, they already have connections to peers not in the active set.

In the end what the node would notice is 0 backing rewards because it
wasn't directly connected to the peers in its backing group.

## Root-cause

The flow is like this:
1. At BAD_SESSION - 1, in `rotate_session` new nodes are added to
QueuedKeys
https://github.com/paritytech/polkadot-sdk/blob/02e1a7f476d7d7c67153e975ab9a1bdc02ffea12/substrate/frame/session/src/lib.rs#L609
```
 <QueuedKeys<T>>::put(queued_amalgamated.clone());
<QueuedChanged<T>>::put(next_changed);
```
2. AuthorityDiscovery::on_new_session is called with `changed` being the
value of `<QueuedChanged<T>>:` at BAD_SESSION - **2** because it was
saved before being updated
https://github.com/paritytech/polkadot-sdk/blob/02e1a7f476d7d7c67153e975ab9a1bdc02ffea12/substrate/frame/session/src/lib.rs#L613
3. At BAD_SESSION - 1, `AuthorityDiscovery::on_new_session` doesn't
updated its next_keys because `changed` was false.
4. For the entire durations of `BAD_SESSION - 1` everyone calling
runtime api `authorities`(should return past, present and future
authorities) won't discover the nodes that should become active .
5. At the beginning of BAD_SESSION, all nodes discover the new nodes are
authorities, but it is already too late because reserved_nodes are
updated only at the beginning of the session by the `gossip-support`.
See above why this bad.

## Fix
Update next keys with the queued_validators at every session, not matter
the value of `changed` this is the same way babe pallet correctly does
it.
https://github.com/paritytech/polkadot-sdk/blob/02e1a7f476d7d7c67153e975ab9a1bdc02ffea12/substrate/frame/babe/src/lib.rs#L655

## Notes

- The issue doesn't reproduce with proof-authorities changes like
`versi` because `changed` would always be true and `AuthorityDiscovery`
correctly updates its next_keys every time.
- Confirmed at session `37651` on kusama that this is exactly what it
happens by looking at blocks with polkadot.js.

## TODO
- [ ] Move versi on proof of stake and properly test before and after
fix to confirm there is no other issue.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2024-03-18 11:45:39 +00:00
gupnik 7099f6e1b1 Removes as [disambiguation_path] from derive_impl usage (#3652)
Step in https://github.com/paritytech/polkadot-sdk/issues/171

This PR removes `as [disambiguation_path]` syntax from `derive_impl`
usage across the polkadot-sdk as introduced in
https://github.com/paritytech/polkadot-sdk/pull/3505
2024-03-15 07:46:09 +00:00
georgepisaltu bbd51ce867 Revert "FRAME: Create TransactionExtension as a replacement for SignedExtension (#2280)" (#3665)
This PR reverts #2280 which introduced `TransactionExtension` to replace
`SignedExtension`.

As a result of the discussion
[here](https://github.com/paritytech/polkadot-sdk/pull/3623#issuecomment-1986789700),
the changes will be reverted for now with plans to reintroduce the
concept in the future.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
2024-03-13 14:10:59 +00:00
Gavin Wood fd5f9292f5 FRAME: Create TransactionExtension as a replacement for SignedExtension (#2280)
Closes #2160

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature).
  - General transactions (without a hardcoded signature).

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now
permissible:
- 0b000000100: Bare (used to be called "Unsigned"): contains Signature
or Extra (extension data). After bare transactions are no longer
supported, this will strictly identify an Inherents only.
- 0b100000100: Old-school "Signed" Transaction: contains Signature and
Extra (extension data).
- 0b010000100: New-school "General" Transaction: contains Extra
(extension data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes two type parameters: `Call` and `Context`. `Call` is
the runtime call type which used to be an associated type; you can just
move it to become a type parameter for your trait impl. `Context` is not
currently used and you can safely implement over it as an unbounded
type.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

## TODO

- [x] Introduce `CheckSignature` impl of `TransactionExtension` to
ensure it's possible to have crypto be done wholly in a
`TransactionExtension`.
- [x] Deprecate `SignedExtension` and move all uses in codebase to
`TransactionExtension`.
  - [x] `ChargeTransactionPayment`
  - [x] `DummyExtension`
  - [x] `ChargeAssetTxPayment` (asset-tx-payment)
  - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
  - [x] `CheckWeight`
  - [x] `CheckTxVersion`
  - [x] `CheckSpecVersion`
  - [x] `CheckNonce`
  - [x] `CheckNonZeroSender`
  - [x] `CheckMortality`
  - [x] `CheckGenesis`
  - [x] `CheckOnlySudoAccount`
  - [x] `WatchDummy`
  - [x] `PrevalidateAttests`
  - [x] `GenericSignedExtension`
  - [x] `SignedExtension` (chain-polkadot-bulletin)
  - [x] `RefundSignedExtensionAdapter`
- [x] Implement `fn weight` across the board.
- [ ] Go through all pre-existing extensions which assume an account
signer and explicitly handle the possibility of another kind of origin.
- [x] `CheckNonce` should probably succeed in the case of a non-account
origin.
- [x] `CheckNonZeroSender` should succeed in the case of a non-account
origin.
- [x] `ChargeTransactionPayment` and family should fail in the case of a
non-account origin.
  - [ ] 
- [x] Fix any broken tests.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Marcin S. <marcin@realemail.net>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Javier Bullrich <javier@bullrich.dev>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Julian Eager <eagr@tutanota.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: yjh <yjh465402634@gmail.com>
Co-authored-by: Tom Mi <tommi@niemi.lol>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
Co-authored-by: PG Herveou <pgherveou@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: bader y <ibnbassem@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Parth <desaiparth08@gmail.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Jonathan Udd <jonathan@dwellir.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Evgeny Snitko <evgeny@parity.io>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro>
Co-authored-by: zhiqiangxu <652732310@qq.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Anwesh <anweshknayak@gmail.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com>
Co-authored-by: Muharem <ismailov.m.h@gmail.com>
Co-authored-by: joepetrowski <joe@parity.io>
Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com>
Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: command-bot <>
2024-03-04 19:12:43 +00:00
Liam Aharon 12ce4f7d04 Runtime Upgrade ref docs and Single Block Migration example pallet (#1554)
Closes https://github.com/paritytech/polkadot-sdk-docs/issues/55

- Changes 'current storage version' terminology to less ambiguous
'in-code storage version' (suggestion by @ggwpez)
- Adds a new example pallet `pallet-example-single-block-migrations`
- Adds a new reference doc to replace
https://docs.substrate.io/maintain/runtime-upgrades/ (temporarily living
in the pallet while we wait for developer hub PR to merge)
- Adds documentation for the `storage_alias` macro
- Improves `trait Hooks` docs 
- Improves `trait GetStorageVersion` docs
- Update the suggested patterns for using `VersionedMigration`, so that
version unchecked migrations are never exported
- Prevents accidental usage of version unchecked migrations in runtimes

https://github.com/paritytech/substrate/pull/14421#discussion_r1255467895
- Unversioned migration code is kept inside `mod version_unchecked`,
versioned code is kept in `pub mod versioned`
- It is necessary to use modules to limit visibility because the inner
migration must be `pub`. See
https://github.com/rust-lang/rust/issues/30905 and

https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504/40
for more.

### todo

- [x] move to reference docs to proper place within sdk-docs (now that
https://github.com/paritytech/polkadot-sdk/pull/2102 is merged)
- [x] prdoc

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Juan <juangirini@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
2024-02-28 07:32:02 +00:00
Oliver Tale-Yazdi e76b244853 [FRAME] Test for sane genesis default (#3412)
Closes https://github.com/paritytech/polkadot-sdk/issues/2713

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2024-02-22 00:35:01 +00:00
Gilt0 b78c72cf9c [FRAME Core] remove unnecessary overrides while using derive_impl for frame_system (#3317)
# Description

This PR removes redundant type definition from test definition config
implementations like
```
#[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::Config for Test {
    type A = A;
    ...
}
```

This changes avoid redundancies in the code as the macro `derive_impl`
defines the relevant types. To implement the changes, it was a simple
fact of running tests and making sure that the tests would still run
while the definition would be removed.

Closes #3237

As a note, here is a brief account of things done from the Issue's
description statement
```
alliance migrate alliance, fast-unstake and bags list to use derive-impl #1636
asset-conversion                                                                                            DONE
asset-rate                                                                                                  DONE
assets                                                                                                      DONE
atomic-swap                                                                                                 DONE
aura                                                                                                        DONE
authority-discovery                                                                                         DONE                                                                     
authorship  migrate babe and authorship to use derive-impl #1790
babe  migrate babe and authorship to use derive-impl #1790
bags-list migrate alliance, fast-unstake and bags list to use derive-impl #1636
balances                                                                                                    DONE
beefy                                                                                                       NOTHING TO DO --- also noted this error without failing tests Feb 13 13:49:08.941 ERROR runtime::timestamp: `pallet_timestamp::UnixTime::now` is called at genesis, invalid value returned: 0
beefy-mmr                                                                                                   NOTHING TO DO
bounties                                                                                                    DONE
child-bounties                                                                                              DONE
collective                                                                                                  DONE
contracts                                                                                                   DONE
conviction-voting                                                                                           DONE
core-fellowship                                                                                             NOTHING TO DO
democracy                                                                                                   DONE
election-provider-multi-phase                                                                               NOTHING TO DO
elections-phragmen                                                                                          DONE
executive                                                                                                   NOTHING TO DO
fast-unstake migrate alliance, fast-unstake and bags list to use derive-impl #1636
glutton                                                                                                     DONE
grandpa                                                                                                     DONE
identity                                                                                                    DONE
im-online                                                                                                   NOTHING TO DO
indices Refactor indices pallet #1789
insecure-randomness-collective-flip                                                                         DONE
lottery                                                                                                     DONE
membership                                                                                                  DONE
merkle-mountain-range                                                                                       NOTHING TO DO
message-queue                                                                                               DONE
multisig add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453
nft-fractionalization                                                                                       DONE
nfts                                                                                                        DONE
nicks Refactor pallet-state-trie-migration to fungible::* traits #1801                                      NOT IN REPO
nis                                                                                                         DONE
node-authorization                                                                                          DONE
nomination-pools                                                                                            NOTHING TO DO -- ONLY impl for Runtime
offences                                                                                                    DELETED EVERYTHING -- IS THAT CORRECT??
preimage                                                                                                    DONE
proxy add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453
ranked-collective                                                                                           NOTHING TO DO
recovery                                                                                                    DONE
referenda                                                                                                   DONE
remark                                                                                                      DONE
root-offences                                                                                               DONE
root-testing                                                                                                NOTHING TO DO
salary                                                                                                      NOTHING TO DO
scheduler                                                                                                   DONE
scored-pool                                                                                                 DONE
session                                                                                                     DONE -- substrate/frame/session/benchmarking/src/mock.rs untouched
society                                                                                                     NOTHING TO DO
staking                                                                                                     DONE
staking-bags-benchmarks                                                                                     NOT IN REPO
state-trie-migration                                                                                        NOTHING TO DO
statement                                                                                                   DONE
sudo                                                                                                        DONE
system                                                                                                      DONE
timestamp                                                                                                   DONE
tips                                                                                                        DONE
transaction-payment                                                                                         NOTHING TO DO
transaction-storage                                                                                         NOTHING TO DO
treasury                                                                                                    DONE
try-runtime                                                                                                 NOTHING TO DO -- no specific mention of 'for Test'
uniques                                                                                                     DONE
utility                                                                                                     DONE
vesting                                                                                                     DONE
whitelist                                                                                                   DONE
```

---------

Co-authored-by: command-bot <>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
2024-02-19 12:07:10 +00:00
Squirrel bc2e5e1fe2 sp-std -> core (#3199)
First in a series of PRs that reduces our use of sp-std with a view to
deprecating it.

This is just looking at /substrate and moving some of the references
from `sp-std` to `core`.
These particular changes should be uncontroversial.

Where macros are used `::core` should be used to remove any ambiguity.

part of https://github.com/paritytech/polkadot-sdk/issues/2101
2024-02-06 13:01:29 +00:00
joe petrowski 757ae372f7 Switch All construct_runtimes to New Syntax (#2979)
Clean up all the old syntax.

---------

Co-authored-by: command-bot <>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com>
Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>
2024-01-22 07:15:53 +00:00
Tsvetomir Dimitrov f8954093b4 Filter votes from disabled validators in BackedCandidates in process_inherent_data (#2889)
Backport of https://github.com/paritytech/polkadot-sdk/pull/1863 to
master

Extend candidate sanitation in paras_inherent by removing backing votes
from disabled validators. Check
https://github.com/paritytech/polkadot-sdk/issues/1592 for more details.

This change is related to the disabling strategy implementation
(https://github.com/paritytech/polkadot-sdk/pull/2226).

---------

Co-authored-by: ordian <noreply@reusable.software>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>
2024-01-18 07:33:58 +00:00
Kian Paimani eaf1bc5633 Introduce Polkadot-Sdk developer_hub (#2102)
This PR introduces the new crate `developer_hub` into the polkadot-sdk
repo. The vision for the developer-hub crate is detailed in [this
document](https://docs.google.com/document/d/1XLLkFNE8v8HLvZpI2rzsa8N2IN1FcKntc8q-Sc4xBAk/edit?usp=sharing).

<img width="1128" alt="Screenshot 2023-11-02 at 10 45 48"
src="https://github.com/paritytech/polkadot-sdk/assets/5588131/1e12b60f-fef5-42c4-8503-a3ba234077c3">


Other than adding the new crate, it also does the following: 

* Remove the `substrate` crate, as there is now a unique umbrella crate
for multiple things in `developer_hub::polkadot_sdk`.
* (backport candidate) A minor change to `frame-support` macros that
allows `T::RuntimeOrigin` to also be acceptable as the origin type.
* (backport candidate) A minor change to `frame-system` that allows us
to deposit events at genesis because now the real genesis config is
generated via wasm, and we can safely assume `cfg!(feature = "std")`
means only testing. related to #62.
* (backport candidate) Introduces a small `read_events_for_pallet` to
`frame_system` for easier event reading in tests.
* From https://github.com/paritytech/polkadot-sdk-docs/issues/31, it
takes action on improving the `pallet::call` docs.
* From https://github.com/paritytech/polkadot-sdk-docs/issues/31, it
takes action on improving the `UncheckedExtrinsic` docs.

## Way Forward

First, a version of this is deployed temporarily
[here](https://blog.kianenigma.nl/polkadot-sdk/developer_hub/index.html).
I will keep it up to date on a daily basis.

### This Pull Request

I see two ways forward: 

1. We acknowledge that everything in `developer-hub` is going to be WIP,
and merge this asap. We should not yet use links to this crate anywhere.
2. We make this be the feature branch, make PRs against this, and either
gradually backport it, or only merge to master once it is done.

I am personally in favor of option 1. If we stick to option 2, we need a
better way to deploy a staging version of this to gh-pages.

### Issue Tracking

The main issues related to the future of `developer_hub` are: 

- https://github.com/paritytech/polkadot-sdk-docs/issues/31
- https://github.com/paritytech/polkadot-sdk-docs/issues/4
- https://github.com/paritytech/polkadot-sdk-docs/issues/26 
- https://github.com/paritytech/polkadot-sdk-docs/issues/32
- https://github.com/paritytech/polkadot-sdk-docs/issues/36


### After This Pull Request

- [ ] create a redirect for
https://paritytech.github.io/polkadot-sdk/master/substrate/
- [x] analytics 
- [ ] link checker
- [ ] the matter of publishing, and how all of these relative links for
when we do, that is still an open question. There is section on this in
the landing page.
- [ ] updated https://paritytech.github.io/

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: bader y <ibnbassem@gmail.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
2023-11-30 12:15:46 +01:00
gupnik cd8741c8b5 Moves all test runtimes to use derive_impl (#2409)
Step in https://github.com/paritytech/polkadot-sdk/issues/171

This PR adds `derive_impl` on all `frame_system` config impls for mock
runtimes. The overridden configs are maintained as of now to ensure
minimal changes.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-11-28 14:13:57 +01:00
Juan bdbe982970 Restructure dispatch macro related exports (#1162)
* restructure dispatch macro related exports

* moved Dispatchable to lib.rs

* fix .gitignore final newline

* ".git/.scripts/commands/fmt/fmt.sh"

* fix rustdocs

* wip

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: command-bot <>
Co-authored-by: ordian <write@reusable.software>
2023-08-31 14:08:44 +03:00
Juan 878c562cd4 Restructure frame_support macro related exports (#14745)
* make reexports private

* make reexports private 2

* make reexports private for runtime-benchmarking

* make reexports private for try-runtime

* fix for try-runtime

* make reexports private for tests

* fmt

* make reexports private for tests

* make reexports private for experimental

* fix beefy

* fix ui test

* fix ui test

* fix benches

* ".git/.scripts/commands/fmt/fmt.sh"

* fix contracts use

* wip

* wip

* do not reexport sp_api::metadata_ir

* fix CI checks

* fix support tests

* ".git/.scripts/commands/fmt/fmt.sh"

* Update frame/support/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* import codec directly

* fmt

* fix node-cli tests

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-08-23 12:30:12 +00:00
Juan 6a29a70a92 Replace system config Index for Nonce (#14290)
* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* replace Index by Nonce

* wip

* remove index in lieu of nonce

* wip

* remove accountnonce in lieu of nonce

* add minor improvement

* rebase and merge conflicts
2023-07-14 06:56:48 +00:00
gupnik 5e7b27e98c Moves Block to frame_system instead of construct_runtime and removes Header and BlockNumber (#14437)
* Initial setup

* Adds node block

* Uses UncheckedExtrinsic and removes Where section

* Updates frame_system to use Block

* Adds deprecation warning

* Fixes pallet-timestamp

* Removes Header and BlockNumber

* Addresses review comments

* Addresses review comments

* Adds comment about compiler bug

* Removes where clause

* Refactors code

* Fixes errors in cargo check

* Fixes errors in cargo check

* Fixes warnings in cargo check

* Formatting

* Fixes construct_runtime tests

* Uses import instead of full path for BlockNumber

* Uses import instead of full path for Header

* Formatting

* Fixes construct_runtime tests

* Fixes imports in benchmarks

* Formatting

* Fixes construct_runtime tests

* Formatting

* Minor updates

* Fixes construct_runtime ui tests

* Fixes construct_runtime ui tests with 1.70

* Fixes docs

* Fixes docs

* Adds u128 mock block type

* Fixes split example

* fixes for cumulus

* ".git/.scripts/commands/fmt/fmt.sh"

* Updates new tests

* Fixes fully-qualified path in few places

* Formatting

* Update frame/examples/default-config/src/lib.rs

Co-authored-by: Juan <juangirini@gmail.com>

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Juan <juangirini@gmail.com>

* ".git/.scripts/commands/fmt/fmt.sh"

* Addresses some review comments

* Fixes build

* ".git/.scripts/commands/fmt/fmt.sh"

* Update frame/democracy/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/democracy/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Addresses review comments

* Updates trait bounds

* Minor fix

* ".git/.scripts/commands/fmt/fmt.sh"

* Removes unnecessary bound

* ".git/.scripts/commands/fmt/fmt.sh"

* Updates test

* Fixes build

* Adds a bound for header

* ".git/.scripts/commands/fmt/fmt.sh"

* Removes where block

* Minor fix

* Minor fix

* Fixes tests

* ".git/.scripts/commands/update-ui/update-ui.sh" 1.70

* Updates test

* Update primitives/runtime/src/traits.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update primitives/runtime/src/traits.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Updates doc

* Updates doc

---------

Co-authored-by: command-bot <>
Co-authored-by: Juan <juangirini@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-07-13 12:01:34 +00:00
Michal Kucharczyk 87d41d0a89 GenesisBuild<T,I> deprecated. BuildGenesisConfig added. (#14306)
* frame::support: GenesisConfig types for Runtime enabled

* frame::support: macro generating GenesisBuild::build for RuntimeGenesisConfig

* frame: ambiguity BuildStorage vs GenesisBuild fixed

* fix

* RuntimeGenesisBuild added

* Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed"

This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293.

* Revert "fix"

This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b.

* Revert "RuntimeGenesisBuild added"

This reverts commit 3c131b618138ced29c01ab8d15d8c6410c9e128b.

* Revert "Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed""

This reverts commit 2b1ecd467231eddec69f8d328039ba48a380da3d.

* Revert "Revert "fix""

This reverts commit fd7fa629adf579d83e30e6ae9fd162637fc45e30.

* Code review suggestions

* frame: BuildGenesisConfig added, BuildGenesis deprecated

* frame: some pallets updated with BuildGenesisConfig

* constuct_runtime: support for BuildGenesisConfig

* frame::support: genesis_build macro supports BuildGenesisConfig

* frame: BuildGenesisConfig added, BuildGenesis deprecated

* Cargo.lock update

* test-runtime: fixes

* Revert "fix"

This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b.

* Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed"

This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293.

* self review

* doc fixed

* ui tests fixed

* fmt

* tests fixed

* genesis_build macrto fixed for non-generic GenesisConfig

* BuildGenesisConfig constraints added

* warning fixed

* some duplication removed

* fmt

* fix

* doc tests fix

* doc fix

* cleanup: remove BuildModuleGenesisStorage

* self review comments

* fix

* Update frame/treasury/src/tests.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Update frame/support/src/traits/hooks.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* doc fix: GenesisBuild exposed

* ".git/.scripts/commands/fmt/fmt.sh"

* frame: more serde(skip) + cleanup

* Update frame/support/src/traits/hooks.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* frame: phantom fields moved to the end of structs

* chain-spec: Default::default cleanup

* test-runtime: phantom at the end

* merge master fixes

* fix

* fix

* fix

* fix

* fix (facepalm)

* Update frame/support/procedural/src/pallet/expand/genesis_build.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* fmt

* fix

* fix

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-07-12 10:22:12 +00:00
Oleg Plakida 0aa79f87ca ".git/.scripts/commands/bench-vm/bench-vm.sh" all (#13933)
Co-authored-by: command-bot <>
2023-06-19 14:31:20 +01:00
Michal Kucharczyk 613420a035 frame: Enable GenesisConfig in no_std (#14108)
* frame: Default for GenesisConfig in no_std

`Default` for `GenesisConfig` will be required for no_std in no native
runtime world. It must be possible to instantiate default GenesisConfig
for pallets and runtime.

* ".git/.scripts/commands/fmt/fmt.sh"

* hash69 in no_std reverted

* derive(DefaultNoBound) for GenesisConfig used when possible

* treasury: derive(Default)

* Cargo.lock update

* genesis_config: compiler error improved

When std feature is not enabled for pallet, the GenesisConfig will be
defined, but serde::{Serialize,Deserialize} traits will not be
implemented.

The compiler error indicates the reason of latter errors.

This is temporary and serde traits will be enabled with together with
`serde` support in frame.

---------

Co-authored-by: command-bot <>
2023-05-20 07:34:23 +00:00
Oliver Tale-Yazdi ebfe00d590 Fixes PoV over-estimation (#13766)
* Align log

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use max instead of sum

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make comment ordering deterministic

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Dont add Pov overhead when all is ignored

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update test pallet weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Re-run weights on bm2

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Actually use new weights

Fucked up the merge for this file...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update contract weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-04-13 13:11:58 +00:00
Mira Ressel 17765733f0 simplify some pattern matches to appease 1.68 clippy (#13833) 2023-04-11 10:57:14 +00:00
Oliver Tale-Yazdi 3a3d93afee Update all weights (#13614)
* Empty commit

* ".git/.scripts/commands/bench/bench.sh" all

---------

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: command-bot <>
2023-03-16 17:57:45 +01:00
Vivek Pandya 2009821cde Remove use of trait Store from all pallets and deprecate it. (#13535)
* Remove use of trait Store from staking pallet

* Remove use of trait Store from bounties pallet

* Remove use of trait Store from collective pallet

* Remove use of trait Store from babe pallet

* Remove use of trait Store from assets pallet

* Remove use of trait Store from grandpa pallet

* Remove use of trait Store from balances pallet

* Remove use of trait Store from authorship pallet

* Remove use of trait Store from authority-discovery pallet

* Remove use of trait Store from atomic-swap pallet

* Remove use of trait Store from sudo pallet

* Remove use of trait Store from scheduler pallet

* Remove use of trait Store from scored-pool pallet

* Remove use of trait Store from society pallet

* Remove use of trait Store from lottery pallet

* Remove use of trait Store from executive pallet

* Remove use of trait Store from democracy pallet

* Remove use of trait Store from elections-phragmen pallet

* Remove use of trait Store from indices pallet

* Remove use of trait Store from identity pallet

* Remove use of trait Store from multisig pallet

* Remove use of trait Store from merkle-mountain-range pallet

* Remove use of trait Store from im-online pallet

* Remove use of trait Store from membership pallet

* Remove use of trait Store from nicks pallet

* Remove use of trait Store from session pallet

* Remove use of trait Store from transaction-payment pallet

* Remove use of trait Store from utility pallet

* Remove use of trait Store from child-bounties pallet

* Remove use of trait Store from nis pallet

* Remove use of trait Store from nfts pallet

* Remove use of trait Store from conviction-voting pallet

* Remove use of trait Store from treasury pallet

* Remove use of trait Store from vesting pallet

* Remove use of trait Store from preimage pallet

* Remove use of trait Store from uniques pallet

* Remove use of trait Store from ranked-collective pallet

* Remove use of trait Store from beefy-mmr pallet

* Remove use of trait Store from referenda pallet

* Remove use of trait Store from whitelist pallet

* Remove use of trait Store from alliance pallet

* Remove use of trait Store from nomination-pools pallet

* Remove use of trait Store from state-trie-migration pallet

* Remove use of trait Store from message-queue pallet

* Remove use of trait Store from root-offences pallet

* Remove use of trait Store from root-testing pallet

* Remove use of trait Store from timestamps pallet

* Remove use of trait Store from system pallet

* Remove use of trait Store from offences pallet

* Remove use of trait Store from recovery pallet

* Remove use of trait Store from node-authorization pallet

* Remove use of trait Store from proxy pallet

* Remove use of trait Store from benchmarking pallet

* Remove use of trait Store from bags-list pallet

* Add deprecated warning in store_trait

* Change warning message

* Run cargo fmt

* Fix warning and update tests

* Remove unnecessary allow deprecated

* Remove use of trait Store

* Fix mismatch in expected output

* Minor update to warning message for deprecation of generate_store with Store trait attribute

* Fixes as per review comments

* Fixes as per review suggestions

* Remove use of Store trait from core-fellowship pallet

* Fix type in store_trait.rs

* Fixes as pre review comment
2023-03-13 15:30:05 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Vivek Pandya 35e8e137c5 cleanup <weight></weight> from docs comments (#13350)
* cleanup <weight></weight> from docs comments

* Changes to address review commnets

* Fix CI cargo test --docs

---------

Co-authored-by: parity-processbot <>
2023-02-14 14:09:36 +00:00
Shawn Tabrizi 1b27ae9549 Add Proof Size to Weight Output (#11637)
* initial impl

* add template test

* linear fit proof size

* always record proof when tracking storage

* calculate worst case pov

* remove duplicate worst case

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* more comment output

* add cli for worst case map size

* update name

* clap does not support underscores

* rename

* expose worst case map values

* improve some comments

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* update template

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix fmt

* more fmt

* more fmt

* Dont panic when there is no proof

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Whitelist :extrinsic_index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use whitelist when recording proof

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add logs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add PoV testing pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Deploy PoV testing pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Storage benches reside in the PoV pallet

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Linear regress PoV per component

Splits the PoV calculation into "measured" and "estimated".
The measured part is reported by the Proof recorder and linear
regressed over all components at once.
The estimated part is calculated as worst-case by using the max
PoV size per storage access and calculating one linear regress per
component. This gives each component a (possibly) independent PoV.
For now the measured size will always be lower than the PoV on
Polkadot since it is measured on an empty snapshot. The measured
part is therefor only used as diagnostic for debugging.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Put PoV into the weight templates

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Extra alanysis choise for PoV

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add+Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make benches faster

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use same template comments

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_balances

* ".git/.scripts/bench-bot.sh" pallet dev pallet_democracy

* Update referenda mock BlockWeights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Take measured value size into account

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_scheduler

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* proof_size: None

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ugly, but works

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* wup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add pov_mode attribute to the benchmarks! macro

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use pov_mode attribute in PoV benchmarking

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Scheduler, Whitelist: Add pov_mode attr

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update PoV weights

* Add CLI arg: default-pov-mode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Update PoV weights"

This reverts commit 2f3ac2387396470b118122a6ff8fa4ee12216f4b.

* Revert "WIP"

This reverts commit c34b538cd2bc45da4544e887180184e30957904a.

* Revert first approach

This reverts commit range 8ddaa2fffe5930f225a30bee314d0b7c94c344dd^..4c84f8748e5395852a9e0e25b0404953fee1a59e

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add extra benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_alliance

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_whitelist

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_scheduler

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy 🤦

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add reference benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix doc comments

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Undo logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add 'Ignored' pov_mode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Allow multiple attributes per benchmark

Turns out that the current benchmarking syntax does not support
multiple attributes per bench 🤦. Changing it to support that
since otherwise the `pov_mode` would conflict with the others.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Validate pov_mode syntax

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Ignore PoV for all contract benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Bump macro recursion limit

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update contract weights

They dont have a PoV component anymore.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix test ffs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* pov_mode is unsupported in V2 syntax

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix pallet ui tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* update pallet ui

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix pallet ui tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Your Name <you@example.com>
2023-01-26 22:35:39 +00:00
Anthony Alaribe c04e68fd01 introduce log-target constant to more frame pallets (#13116)
* introduce log-target constant to more frame pallets

* cargo fmt

* make LOG_TARGET in session public

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/elections-phragmen/src/migrations/v3.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/elections-phragmen/src/migrations/v3.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/elections-phragmen/src/migrations/v3.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* move LOG_TARGET=runtime::session_historical to migrations module, where it's actually used

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-01-10 19:18:12 +00:00
Oliver Tale-Yazdi 2bc6078738 Remove unused types (#13091)
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-01-07 13:42:44 +00:00
Oliver Tale-Yazdi 31f5119ecd Use explicit call indices (#12891)
* frame-system: explicit call index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use explicit call indices

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* pallet-template: explicit call index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* DNM: Temporarily require call_index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "DNM: Temporarily require call_index"

This reverts commit c4934e312e12af72ca05a8029d7da753a9c99346.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2022-12-12 14:32:41 +00:00
Shawn Tabrizi 78cddd7631 New Weights for All Pallets (#12325)
* new weights for everything

* fix

* fmt

* new batch

* fmt

* new batch

* Update run_all_benchmarks.sh

* add headers

* update weights

* Update lib.rs

* block and extrinsic weight
2022-11-08 18:50:28 +00:00
Sergej Sakac e4b6f4a66d BREAKING: Rename Origin (#12258)
* BREAKING: Rename Origin

* more renaming

* a bit more renaming

* fix

* more fixing

* fix in frame_support

* even more fixes

* fix

* small fix

* ...

* update .stderr

* docs

* update docs

* update docs

* docs
2022-09-20 22:13:09 +00:00
Sergej Sakac 6e8795afe6 BREAKING: Rename Call & Event (#11981)
* rename Event to RuntimeEvent

* rename Call

* rename in runtimes

* small fix

* rename Event

* small fix & rename RuntimeCall back to Call for now

* small fixes

* more renaming

* a bit more renaming

* fmt

* small fix

* commit

* prep for renaming associated types

* fix

* rename associated Event type

* rename to RuntimeEvent

* commit

* merge conflict fixes & fmt

* additional renaming

* fix.

* fix decl_event

* rename in tests

* remove warnings

* remove accidental rename

* .

* commit

* update .stderr

* fix in test

* update .stderr

* TRYBUILD=overwrite

* docs

* fmt

* small change in docs

* rename PalletEvent to Event

* rename Call to RuntimeCall

* renamed at wrong places :P

* rename Call

* rename

* rename associated type

* fix

* fix & fmt

* commit

* frame-support-test

* passing tests

* update docs

* rustdoc fix

* update .stderr

* wrong code in docs

* merge fix

* fix in error message

* update .stderr

* docs & error message

* .

* merge fix

* merge fix

* fmt

* fmt

* merge fix

* more fixing

* fmt

* remove unused

* fmt

* fix

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2022-09-12 22:03:31 +00:00
Boluwatife Bakre 3ec4d13e9f Use parameter_types instead of thread_local for test-setup (#12036)
* Edit to Assets. parameter_types

* fixes

* Test Fixes. WIP

* Edits to pallet-aura

* Camel Case

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Implementation of mutate fn

* update to pallet-aura

* Update to frame-system. Fixes

* Update to frame-support-test. CamelCases

* Updates to frame- contracts, offences, staking, bounties, child bounties

* Edit to mutate fn. Changes to frame-contracts. CamelCase pallet-aura

* Edits to frame-contracts & executive

* cargo +nightly fmt

* unused import removed

* unused import removed

* cargo +nightly fmt

* minor adjustment

* updates

* updates

* cargo +nightly fmt

* cargo +nightly fmt

* take fn implemented

* update

* update

* Fixes to CallFilter

* cargo +nightly fmt

* final fixes

* Default changed to $value

* Update frame/support/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2022-09-08 10:46:25 +00:00
Shawn Tabrizi 23a81ee1e3 Remove RefTimeWeight (#12157)
* update api

* update

* remove unused

* remove `one` api

* fix unused

* fmt

* add saturating accrue

* remove `Weight::new()`

* use some macros

* div makes no sense

* Update weight_v2.rs

* missed some

* more patch

* fixes

* more fixes

* more fix

* more fix

* remove RefTimeWeight

* Update frame/contracts/src/storage.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* not needed

* Fixes

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2022-09-02 18:19:49 +00:00
Shawn Tabrizi 30951822ba Weight v1.5: Opaque Struct (#12138)
* initial idea

* update frame_support

* update a bunch more

* add ord

* adjust RuntimeDbWeight

* frame_system builds

* re-export

* frame_support tests pass

* frame_executive compile

* frame_executive builds

* frame_system tests passing

* pallet-utility tests pass

* fix a bunch of pallets

* more

* phragmen

* state-trie-migration

* scheduler and referenda

* pallet-election-provider-multi-phase

* aura

* staking

* more

* babe

* balances

* bunch more

* sudo

* transaction-payment

* asset-tx-payment

* last pallets

* fix alliance merge

* fix node template runtime

* fix pallet-contracts cc @athei

* fix node runtime

* fix compile on runtime-benchmarks feature

* comment

* fix frame-support-test

* fix more tests

* weight regex

* frame system works

* fix a bunch

* more

* more

* more

* more

* more

* more fixes

* update templates

* fix contracts benchmarks

* Update lib.rs

* Update lib.rs

* fix ui

* make scalar saturating mul const

* more const functions

* scalar div

* refactor using constant functions

* move impl

* fix overhead template

* use compactas

* Update lib.rs
2022-08-31 11:26:13 +00:00
Bastian Köcher 73d9ae3284 Introduce trie level cache and remove state cache (#11407)
* trie state cache

* Also cache missing access on read.

* fix comp

* bis

* fix

* use has_lru

* remove local storage cache on size 0.

* No cache.

* local cache only

* trie cache and local cache

* storage cache (with local)

* trie cache no local cache

* Add state access benchmark

* Remove warnings etc

* Add trie cache benchmark

* No extra "clone" required

* Change benchmark to use multiple blocks

* Use patches

* Integrate shitty implementation

* More stuff

* Revert "Merge branch 'master' into trie_state_cache"

This reverts commit 947cd8e6d43fced10e21b76d5b92ffa57b57c318, reversing
changes made to 29ff036463.

* Improve benchmark

* Adapt to latest changes

* Adapt to changes in trie

* Add a test that uses iterator

* Start fixing it

* Remove obsolete file

* Make it compile

* Start rewriting the trie node cache

* More work on the cache

* More docs and code etc

* Make data cache an optional

* Tests

* Remove debug stuff

* Recorder

* Some docs and a simple test for the recorder

* Compile fixes

* Make it compile

* More fixes

* More fixes

* Fix fix fix

* Make sure cache and recorder work together for basic stuff

* Test that data caching and recording works

* Test `TrieDBMut` with caching

* Try something

* Fixes, fixes, fixes

* Forward the recorder

* Make it compile

* Use recorder in more places

* Switch to new `with_optional_recorder` fn

* Refactor and cleanups

* Move `ProvingBackend` tests

* Simplify

* Move over all functionality to the essence

* Fix compilation

* Implement estimate encoded size for StorageProof

* Start using the `cache` everywhere

* Use the cache everywhere

* Fix compilation

* Fix tests

* Adds `TrieBackendBuilder` and enhances the tests

* Ensure that recorder drain checks that values are found as expected

* Switch over to `TrieBackendBuilder`

* Start fixing the problem with child tries and recording

* Fix recording of child tries

* Make it compile

* Overwrite `storage_hash` in `TrieBackend`

* Add `storage_cache` to  the benchmarks

* Fix `no_std` build

* Speed up cache lookup

* Extend the state access benchmark to also hash a runtime

* Fix build

* Fix compilation

* Rewrite value cache

* Add lru cache

* Ensure that the cache lru works

* Value cache should not be optional

* Add support for keeping the shared node cache in its bounds

* Make the cache configurable

* Check that the cache respects the bounds

* Adds a new test

* Fixes

* Docs and some renamings

* More docs

* Start using the new recorder

* Fix more code

* Take `self` argument

* Remove warnings

* Fix benchmark

* Fix accounting

* Rip off the state cache

* Start fixing fallout after removing the state cache

* Make it compile after trie changes

* Fix test

* Add some logging

* Some docs

* Some fixups and clean ups

* Fix benchmark

* Remove unneeded file

* Use git for patching

* Make CI happy

* Update primitives/trie/Cargo.toml

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update primitives/state-machine/src/trie_backend.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Introduce new `AsTrieBackend` trait

* Make the LocalTrieCache not clonable

* Make it work in no_std and add docs

* Remove duplicate dependency

* Switch to ahash for better performance

* Speedup value cache merge

* Output errors on underflow

* Ensure the internal LRU map doesn't grow too much

* Use const fn to calculate the value cache element size

* Remove cache configuration

* Fix

* Clear the cache in between for more testing

* Try to come up with a failing test case

* Make the test fail

* Fix the child trie recording

* Make everything compile after the changes to trie

* Adapt to latest trie-db changes

* Fix on stable

* Update primitives/trie/src/cache.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Fix wrong merge

* Docs

* Fix warnings

* Cargo.lock

* Bump pin-project

* Fix warnings

* Switch to released crate version

* More fixes

* Make clippy and rustdocs happy

* More clippy

* Print error when using deprecated `--state-cache-size`

* 🤦

* Fixes

* Fix storage_hash linkings

* Update client/rpc/src/dev/mod.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Review feedback

* encode bound

* Rework the shared value cache

Instead of using a `u64` to represent the key we now use an `Arc<[u8]>`. This arc is also stored in
some extra `HashSet`. We store the key are in an extra `HashSet` to de-duplicate the keys accross
different storage roots. When the latest key usage is dropped in the lru, we also remove the key
from the `HashSet`.

* Improve of the cache by merging the old and new solution

* FMT

* Please stop coming back all the time :crying:

* Update primitives/trie/src/cache/shared_cache.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Fixes

* Make clippy happy

* Ensure we don't deadlock

* Only use one lock to simplify the code

* Do not depend on `Hasher`

* Fix tests

* FMT

* Clippy 🤦

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
2022-08-18 18:59:22 +00:00
Shawn Tabrizi 8e7adaf283 Storage Layer for All FRAME Extrinsics (#11431)
* add new trait

* implement DispatchableWithStorageLayer

* at least one transactional

* all dispatch is at least transactional

* storage_layer api

* add test

* storage layer tests

* deprecate transactional tag

* i guess no reason to deprecate

* remove transactional from batch_all

* update tests

* extend trait

* cargo run --quiet --profile=production --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --profile=production --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --profile=production --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix copy paste name

* cargo run --quiet --profile=production --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_utility --extrinsic=* --execution=wasm --wasm-execution=compiled --output=./frame/utility/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Create run_all_benchmarks.sh

* uncomment build

* update number of steps and repeats

* add skip build

* Update run_all_benchmarks.sh

* Update run_all_benchmarks.sh

* new benchmarks

* Update frame/support/src/traits/dispatch.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/support/src/traits/dispatch.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/support/test/tests/storage_layers.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/support/test/tests/storage_layers.rs

* weights

* Update dispatch.rs

* doc link

* decl_macro support

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2022-05-26 19:28:32 +00:00
Shawn Tabrizi d9f33dc7e9 Create Script to Run All Benchmarks (#11493)
* Create run_all_benchmarks.sh

* Update run_all_benchmarks.sh

* Update run_all_benchmarks.sh

* Review fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update scripts/run_all_benchmarks.sh

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* typo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add default for $1

* Typo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update run_all_benchmarks.sh

* new weights on benchmarking machine

* prefer `--chain=dev`

* fix compile

* fix command

* fmt

* dont use square brackets

* Extend doc

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove +nightly

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add error file an run execute everything optimistically

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-05-23 16:56:42 +00:00
Qinxuan Chen 8bce841d70 frame_support::pallet_prelude: reexport StorageVersion (#11408)
* frame_support::pallet_prelude: reexport StorageVersion

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* cargo fmt

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2022-05-17 20:28:34 +02:00
Falco Hirschenberger b581604aa7 Apply some clippy lints (#11154)
* Apply some clippy hints

* Revert clippy ci changes

* Update client/cli/src/commands/generate.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/inspect_key.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/transactions.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/protocol.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert due to missing `or_default` function.

* Fix compilation and simplify code

* Undo change that corrupts benchmark.

* fix clippy

* Update client/service/test/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

remove leftovers!

* Update client/tracing/src/logging/directives.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/fork-tree/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* added needed ref

* Update frame/referenda/src/benchmarking.rs

* Simplify byte-vec creation

* let's just not overlap the ranges

* Correction

* cargo fmt

* Update utils/frame/benchmarking-cli/src/shared/stats.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Giles Cope <gilescope@gmail.com>
2022-04-30 21:28:27 +00:00
Qinxuan Chen 51915ebe4a Remove deprecated RawEvent types from some pallets (#11316)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2022-04-29 08:28:17 +00:00
Kian Paimani 661d0ea5bb Store validator self-vote in bags-list, and allow them to be trimmed for election (#10821)
* Implement the new validator-in-bags-list scenario + migration

* Apply suggestions from code review

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* some review comments

* guard the migration

* some review comments

* Fix tests 🤦‍♂️

* Fix build

* fix weight_of_fn

* reformat line width

* make const

* use weight of fn cached

* SortedListProvider -> VoterList

* Fix all build and docs

* check post migration

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
2022-03-23 14:17:26 +00:00
Alexander Theißen f3168c3fa0 Add production profile to substrate-wasm-builder (#10747)
* Add production profile to wasm builder

* Fix profile detection

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Replace panic! by println! + exit

* Default to `release` for wasm on debug builds

* Replaced unwrap by expect

* Update all weights

Rerun on the bm2 server.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
2022-01-31 14:16:26 +00:00
Oliver Tale-Yazdi 9daea28085 Optimized weights (#10692)
* Add optimization flags to 'release' profile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Optimized weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add missing pallets

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add `production` profile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* pallet-collective: fix tests

the weight of System.remark went to 0, the collective test uses a
remark call to trigger an out-of-gas condition so I replaced it
with a `remark_with_event` call.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2022-01-24 12:35:25 +00:00
Oliver Tale-Yazdi 362a6d9b28 Require MaxEncodedLen per default (#10662)
* Remove generate_storage_info

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add without_storage_info where needed

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update doc tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more without_storage_info

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix TryBuild

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fix TryBuild tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2022-01-20 11:16:04 +00:00
Squirrel 3dd32d5255 Make clippy _a little_ more annoying (#10570)
* Clippy: +complexity

* Update client/cli/src/arg_enums.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/inspect/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/keystore/src/testing.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update frame/elections/src/lib.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Update primitives/npos-elections/fuzzer/src/reduce.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Incorporating feedback

* No need for Ok

* Additional

* Needed slice

* Wigy's suggestions on less derefs

* fix count

* reverting changes brought in by option_map_unit_fn

* add --all-targets

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2022-01-05 14:35:30 +00:00
Bastian Köcher f3662b4bba Happy new year 2022 (#10573) 2022-01-03 09:22:14 +01:00
cheme 4c651637f2 Inner hashing of value in state trie (runtime versioning). (#9732)
* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.

* useless closure

* remove remaining state_hash variables.

* Remove outdated comment

* useless inner hash

* fmt

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* Apply review change (docs and error).

* fmt

* use explicit runtime_interface in doc test

* fix ui test

* fix doc test

* fmt

* use default for path and specname when resolving version.

* small review related changes.

* doc value size requirement.

* rename old_state feature

* Remove macro changes

* feature rename

* state version as host function parameter

* remove flag for client api

* fix tests

* switch storage chain proof to V1

* host functions, pass by state version enum

* use WrappedRuntimeCode

* start

* state_version in runtime version

* rust fmt

* Update storage proof of max size.

* fix runtime version rpc test

* right intent of convert from compat

* fix doc test

* fix doc test

* split proof

* decode without replay, and remove some reexports.

* Decode with compatibility by default.

* switch state_version to u8. And remove RuntimeVersionBasis.

* test

* use api when reading embedded version

* fix decode with apis

* extract core version instead

* test fix

* unused import

* review changes.

Co-authored-by: kianenigma <kian@parity.io>
2021-12-24 08:54:07 +00:00
dharjeezy 1b0be8ae06 Replace parameter_types with ConstU32 &c. (#10402)
* remove parameter types and use const type

* remove parameter types and use const type

* Delete {

* Delete count,

* refractor for beefy, benchmarking, child bounties, and collective pallets

* refractor for pallet contracts

* refractor for elections

* refractor for more pallets

* fix CI issues

* fix CI issues

* fix CI issues

* fix CI issues

* remove warning to fix CI issue

* remove warning to fix CI issue
refractor for pallet preimage

* remove warning to fix CI issue
refractor for pallet proxy

* remove warning to fix CI issue
refractor for pallet recovery
refractor for pallet randomness-collective-flip

* remove warning to fix CI issue
refractor for pallet scored-pool
refractor for pallet scheduler
refractor for pallet session

* remove warning to fix CI issue
refractor for pallet society, support, system, timestamp, tips

* remove warning to fix CI issue
refractor for pallet transaction_payment, transaction_storage, treasury, uniques, utility

* remove warning to fix CI issue

* cargo +nightly fmt

* CI fix

* more param refractor on beefy-mmr

* refractor for beefy

* Update frame/babe/src/mock.rs

* Update frame/babe/src/mock.rs

* Update frame/bounties/src/tests.rs

* Update frame/tips/src/tests.rs

* Delete mock.rs

* Update frame/examples/basic/src/tests.rs

* Apply suggestions from code review

* Update frame/im-online/src/mock.rs

* Update frame/im-online/src/mock.rs

* Update frame/offences/benchmarking/src/mock.rs

* Update frame/session/benchmarking/src/mock.rs

* Update frame/support/test/tests/pallet_compatibility.rs

* Update frame/support/test/tests/pallet_compatibility_instance.rs

* Update frame/treasury/src/tests.rs

* Update test-utils/runtime/src/lib.rs

* some cleanup

* fmt

* remove unused

Co-authored-by: Damilare <dakinlose@teamapt.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-12-16 03:06:11 +00:00