Commit Graph

59 Commits

Author SHA1 Message Date
Léa Narzis d733c77ee2 Adapt RemoteExternalities and its related types to be used with generic hash parameters (#3953)
Closes  https://github.com/paritytech/polkadot-sdk/issues/3737

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2024-04-08 21:56:41 +00:00
Dcompoze 002d9260f9 Fix spelling mistakes across the whole repository (#3808)
**Update:** Pushed additional changes based on the review comments.

**This pull request fixes various spelling mistakes in this
repository.**

Most of the changes are contained in the first **3** commits:

- `Fix spelling mistakes in comments and docs`

- `Fix spelling mistakes in test names`

- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`

Other source code spelling mistakes are separated into individual
commits for easier reviewing:

- `Fix the spelling of 'authority'`

- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`

- `Fix the spelling of 'prev_enqueud_messages'`

- `Fix the spelling of 'endpoint'`

- `Fix the spelling of 'children'`

- `Fix the spelling of 'PenpalSiblingSovereignAccount'`

- `Fix the spelling of 'PenpalSudoAccount'`

- `Fix the spelling of 'insufficient'`

- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`

- `Fix the spelling of 'subtracted'`

- `Fix the spelling of 'CandidatePendingAvailability'`

- `Fix the spelling of 'exclusive'`

- `Fix the spelling of 'until'`

- `Fix the spelling of 'discriminator'`

- `Fix the spelling of 'nonexistent'`

- `Fix the spelling of 'subsystem'`

- `Fix the spelling of 'indices'`

- `Fix the spelling of 'committed'`

- `Fix the spelling of 'topology'`

- `Fix the spelling of 'response'`

- `Fix the spelling of 'beneficiary'`

- `Fix the spelling of 'formatted'`

- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`

- `Fix the spelling of 'succeeded'`

- `Fix the spelling of 'reopened'`

- `Fix the spelling of 'proposer'`

- `Fix the spelling of 'InstantiationNonce'`

- `Fix the spelling of 'depositor'`

- `Fix the spelling of 'expiration'`

- `Fix the spelling of 'phantom'`

- `Fix the spelling of 'AggregatedKeyValue'`

- `Fix the spelling of 'randomness'`

- `Fix the spelling of 'defendant'`

- `Fix the spelling of 'AquaticMammal'`

- `Fix the spelling of 'transactions'`

- `Fix the spelling of 'PassingTracingSubscriber'`

- `Fix the spelling of 'TxSignaturePayload'`

- `Fix the spelling of 'versioning'`

- `Fix the spelling of 'descendant'`

- `Fix the spelling of 'overridden'`

- `Fix the spelling of 'network'`

Let me know if this structure is adequate.

**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.

~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~

~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~

~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~

**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.

~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~
2024-03-26 13:57:57 +00:00
georgepisaltu bbd51ce867 Revert "FRAME: Create TransactionExtension as a replacement for SignedExtension (#2280)" (#3665)
This PR reverts #2280 which introduced `TransactionExtension` to replace
`SignedExtension`.

As a result of the discussion
[here](https://github.com/paritytech/polkadot-sdk/pull/3623#issuecomment-1986789700),
the changes will be reverted for now with plans to reintroduce the
concept in the future.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
2024-03-13 14:10:59 +00:00
Gavin Wood fd5f9292f5 FRAME: Create TransactionExtension as a replacement for SignedExtension (#2280)
Closes #2160

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature).
  - General transactions (without a hardcoded signature).

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now
permissible:
- 0b000000100: Bare (used to be called "Unsigned"): contains Signature
or Extra (extension data). After bare transactions are no longer
supported, this will strictly identify an Inherents only.
- 0b100000100: Old-school "Signed" Transaction: contains Signature and
Extra (extension data).
- 0b010000100: New-school "General" Transaction: contains Extra
(extension data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes two type parameters: `Call` and `Context`. `Call` is
the runtime call type which used to be an associated type; you can just
move it to become a type parameter for your trait impl. `Context` is not
currently used and you can safely implement over it as an unbounded
type.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

## TODO

- [x] Introduce `CheckSignature` impl of `TransactionExtension` to
ensure it's possible to have crypto be done wholly in a
`TransactionExtension`.
- [x] Deprecate `SignedExtension` and move all uses in codebase to
`TransactionExtension`.
  - [x] `ChargeTransactionPayment`
  - [x] `DummyExtension`
  - [x] `ChargeAssetTxPayment` (asset-tx-payment)
  - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
  - [x] `CheckWeight`
  - [x] `CheckTxVersion`
  - [x] `CheckSpecVersion`
  - [x] `CheckNonce`
  - [x] `CheckNonZeroSender`
  - [x] `CheckMortality`
  - [x] `CheckGenesis`
  - [x] `CheckOnlySudoAccount`
  - [x] `WatchDummy`
  - [x] `PrevalidateAttests`
  - [x] `GenericSignedExtension`
  - [x] `SignedExtension` (chain-polkadot-bulletin)
  - [x] `RefundSignedExtensionAdapter`
- [x] Implement `fn weight` across the board.
- [ ] Go through all pre-existing extensions which assume an account
signer and explicitly handle the possibility of another kind of origin.
- [x] `CheckNonce` should probably succeed in the case of a non-account
origin.
- [x] `CheckNonZeroSender` should succeed in the case of a non-account
origin.
- [x] `ChargeTransactionPayment` and family should fail in the case of a
non-account origin.
  - [ ] 
- [x] Fix any broken tests.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Marcin S. <marcin@realemail.net>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Javier Bullrich <javier@bullrich.dev>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Julian Eager <eagr@tutanota.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: yjh <yjh465402634@gmail.com>
Co-authored-by: Tom Mi <tommi@niemi.lol>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
Co-authored-by: PG Herveou <pgherveou@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: bader y <ibnbassem@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Parth <desaiparth08@gmail.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Jonathan Udd <jonathan@dwellir.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Evgeny Snitko <evgeny@parity.io>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro>
Co-authored-by: zhiqiangxu <652732310@qq.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Anwesh <anweshknayak@gmail.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com>
Co-authored-by: Muharem <ismailov.m.h@gmail.com>
Co-authored-by: joepetrowski <joe@parity.io>
Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com>
Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: command-bot <>
2024-03-04 19:12:43 +00:00
Bruno Galvao b2a62a56c9 Use TEST_WS in all remote-externalities tests (#3284)
Refactor in accordance with
https://github.com/paritytech/polkadot-sdk/issues/2245#issuecomment-1937025951

Prior to this PR, the `remote_tests` test module would either use
`TEST_WS` or `DEFAULT_HTTP_ENDPOINT`.

With the PR, `TEST_WS` is the default for the `remote_tests` test module
and the fallback is `DEFAULT_HTTP_ENDPOINT`.

The only downside I see to this PR is that for particular tests in the
`remote_tests` module, one would want to use a different http endpoint.
If that is the case, they would have to manually hardcode the http
endpoint for that particular test.

Note: The `TEST_WS` node should fulfill the role for all test cases e.g.
include child tries.

Give it a _try_:
```
TEST_WS=wss://rococo-try-runtime-node.parity-chains.parity.io:443 cargo test --features=remote-test -p frame-remote-externalities -- --nocapture
```

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2024-02-13 09:23:22 +00:00
Niklas Adolfsson e16ef0861f rpc: backpressured RPC server (bump jsonrpsee 0.20) (#1313)
This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
   - Less dependencies for the clients in particular
   - Return type requires Clone in method call responses
   - Moved to tokio channels
   - Async subscription API (not used in this PR)

Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate

The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.

Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.

The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.

Hopefully the last release prior to 1.0, sorry in advance for a big PR

Previous attempt: https://github.com/paritytech/substrate/pull/13992

Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627
2024-01-23 08:55:13 +00:00
Davide Galassi 4c10fd2a41 Move cryptographic hashing procedures to crypto folder. (#2306)
Step towards https://github.com/paritytech/polkadot-sdk/issues/1975

As reported
https://github.com/paritytech/polkadot-sdk/issues/1975#issuecomment-1774534225
I'd like to encapsulate crypto related stuff in a dedicated folder.

Currently all cryptographic primitive wrappers are all sparsed in
`substrate/core` which contains "misc core" stuff.

To simplify the process, as the first step with this PR I propose to
move the cryptographic hashing there.

The `substrate/crypto` folder was already created to contains `ec-utils`
crate.

Notes:
- rename `sp-core-hashing` to `sp-crypto-hashing`
- rename `sp-core-hashing-proc-macro` to `sp-crypto-hashing-proc-macro`
- As the crates name is changed I took the freedom to restart fresh from
version 0.1.0 for both crates

---------

Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
2024-01-22 23:36:14 +00:00
Julian Eager 1d5d4a4840 Enable parallel key scraping (#1985)
closes #174

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-11-29 17:46:10 +11:00
Liam Aharon 55f354429c remote-ext: fix state download stall on slow connections and reduce memory usage (#1295)
Original PR https://github.com/paritytech/substrate/pull/14746

---

## Fixing stall

### Introduction
I experienced an apparent stall downloading state from
`https://rococo-try-runtime-node.parity-chains.parity.io:443` which was
having networking difficulties only responding to my JSONRPC requests
with 50-200KB/s of bandwidth.

This PR fixes the issue causing the stall, and generally improves
performance remote-ext when it downloads state by greatly reducing the
chances of a timeout occuring.

### Description
Introduces a new `REQUEST_DURATION_TARGET` constant and modifies
`get_storage_data_dynamic_batch_size` to

- Increase or decrease the batch size of the next request depending on
whether the elapsed time of the last request was gt or lt the target
- Reset the batch size to 1 if the request times out

This fixes an issue on slow connections that can otherwise cause
multiple timeouts and a stalled download when:

1. The batch size increases rapidly as remote-ext downloads keys with
small associated storage values
2. remote-ext tries to process a large series of subsequent keys all
with extremely large associated storage values (Rococo has a series of
keys 1-5MB large)
3. The huge storage values download for 5 minutes until the request
times out
4. The partially downloaded keys are thrown out and remote-ext tries
again with a smaller batch size, but the batch size is still far too
large and takes 5 minutes to be reduced again
5. The download will be essentially stalled for many hours while the
above step cycles


After this PR, the request size will

- Not grow as large to begin with, as it is regulated downwards as the
request duration exceeds the target
- Drop immediately to 1 if the request times out. A timeout indicates
the keys next in line to download have extremely large storage values
compared to previously downloaded keys, and we need to reset the batch
size to figure out what our new ideal batch size is. By not resetting
down to 1, we risk the next request timing out again.

## Reducing memory

As suggested by @bkchr, I adjusted `get_storage_data_dynamic_batch_size`
from being recursive to a loop which allows removing a bunch of clones
that were chewing through a lot of memory. I noticed actually it was
using up to 50GB swap previously when downloading Polkadot keys on a
slow connection, because it needed to recurse and clone a lot.

After this change it uses only ~1.5GB memory.
2023-10-11 01:41:58 +11:00
Bastian Köcher ecf8035da6 Set StateBackend::Transaction to PrefixedMemoryDB (#14612)
* Yep

* Try to get it working everywhere

* Make `from_raw_storage` start with an empty db

* More fixes!

* Make everything compile

* Fix `child_storage_root`

* Fix after merge

* Cleanups

* Update primitives/state-machine/src/overlayed_changes/mod.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Review comments

* Fix issues

* Silence warning

* FMT

* Clippy

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
2023-08-17 10:49:38 +00:00
Marijn Schouten c9b54e10ff change HashFor to HashingFor (#14483)
* change HashFor to HashingFor

* fmt

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: command-bot <>
2023-07-25 16:24:14 +00:00
Liam Aharon 2119c80225 fix(test-externalities): include memory db reference counts in snapshots (#14411)
* persist memory db reference counts in snapshots

* update proxy_test snapshot

* clippy

* comment

* comment

* add snapshot versioning

* update proxy_test

* compact snapshot version

* kick ci

* kick ci

* check snapshot version without extra struct
2023-06-27 14:26:27 +00:00
Liam Aharon 38cc4ad6aa fix(remote-externalities): child trie not loaded (#14365)
* fix remote-ext chlid keys

* sanity check test that child trie is loaded
2023-06-21 14:16:33 +00:00
Liam Aharon 5100a0376e remote-externalities: retry for get_keys (#14167)
* retry get keys request

* remove redundant clone

* remove redundant at
2023-05-19 19:26:52 +00:00
Ankan c8b66415bb divide payload into atleast one chunk (#14086) 2023-05-07 16:06:06 +00:00
Liam Aharon ead46b9efd try-runtime-cli: 'instant' snapshots, threading refactor, better progress logs (#14057)
* remote externalities refactor

* remove redundant logs

* use const for parallel requests

* prefer functional

* improve variable naming

* handle requests error

* use overlayedchanges

* Revert "use overlayedchanges"

This reverts commit c0ddb87a5abdd52207597f5df66cbbdf9d79badc.

* Revert "Revert "use overlayedchanges""

This reverts commit 1d49362d9b999c045c8f970a0ab8b486bc47a90a.

* Revert "Revert "Revert "use overlayedchanges"""

This reverts commit 06df786488d94f249e9abccffac4af445f76e5a7.

* backup/load raw storage values

* test raw storage drain and restore

* update snapshot tests

* improve logs

* clippy suggestions

* address comments

* fix example

* fix test

* clippy
2023-05-05 17:15:40 +10:00
Liam Aharon 6e5141fc23 try-runtime-cli: improve ci stability (#14030)
* hardcode 1 thread

* ci stability

* fix comment

* improve comment

* kick ci

* kick ci

* update expect message

* improve comment

* kick ci

* kick ci

* configure threads with env var

* fix threads env var

* fix threads env var
2023-04-28 00:03:47 +10:00
Liam Aharon 27f89d653b remote-externalities: batch insert key/values (#14004)
* add batch inserting into remote externalities

* use into_iter

* remove redundant comment

* redundant batch insert

* avoid extra vec allocations
2023-04-27 18:12:29 +10:00
Liam Aharon bc1a599284 try-runtime: dynamic storage query sizes (#13923)
* improve batch rpc error message

* wip aimd storage data fetch

* complete aimd function refactor

* make batch_request function async

* improve function name

* fix load_child_remote issue

* slight efficiency improvement

* improve logs and variable name

* remove redundant comment

* improve comment

* address pr comments

* Update utils/frame/remote-externalities/src/lib.rs

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* simplify client handling

* fix type issue

* fix clippy issue

* try to trigger ci

* try to trigger ci

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
2023-04-21 09:16:40 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Kian Paimani c1ed1901e1 Try-runtime Revamp and Facelift (#12537)
* fix online/offline confusion

* unified cache file

* multi-threaded babyyy

* checkpoint for niklas

* compiles

* all tests pass with --test-threads 1

* child-tree scrape is also multi-threaded now.

* better thread splitting

* some suggestions (#12532)

* some suggestions

* tokio multithread

* move unused dependencies

* snapshot command

* fix rem

* a bit of cleanup

* support optional checks

* fix

* OCW command migrated to wasm-only, as an example

* state-version management fully in remote-ext

* almost everything move to wasm executor, some CLI flags reduced

* follow-chain works as well

* Master.into()

* everything builds now

* concurrent insertion and download for remote builds

* minor fix

* fix a bug

* checkpoint

* some updates

* fmt

* review comments

* fmt

* fix

* fmt

* update

* fmt

* rename

* fix the damn UI tests

* fmt

* remoe the thread abstraction for the time being

* cleanup

* fix CI

* fmt

* fix

* fix a few more things

* tweak log levels

* better error handling

* address grumbles: use futures::mpsc

* review comments

* fmt

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* better api version stuff

* some doc update

* a whole lot of docs

* fmt

* fix all docs

* fmt

* rpc rebase: Try-runtime Revamp and Facelift (#12921)

* Introduce sensible weight constants (#12868)

* Introduce sensible weight constants

* cargo fmt

* Remove unused import

* Add missing import

* ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery

Co-authored-by: command-bot <>

* Checkout to the branch HEAD explicitly in `build-linux-substrate` (#12876)

* cli: Improve pruning documentation (#12819)

* cli: Improve pruning documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Keep `finalized` notation and remove `canonical` one

* cli: Fix cargo doc

* cli: `PruningModeClap` IR enum

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Convert PruningModeClap into pruning modes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Use `PruningModeClap`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Rename to `DatabasePruningMode`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Implement `FromStr` instead of `clap::ValueEnum`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/cli/src/params/pruning_params.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Add option documentation back

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply suggestions from code review

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

* Revert "Move LockableCurrency trait to fungibles::Lockable and deprecate LockableCurrency (#12798)" (#12882)

This reverts commit 9a014d1ecd.

* Don't indefinitely block on shutting down Tokio (#12885)

* Don't indefinitely on shutting down Tokio

Now we wait in maximum 60 seconds before we shutdown the node. Tasks are may be leaked and leading
to some data corruption.

* Drink less :thinking_face:

* General Message Queue Pallet (#12485)

* The message queue

* Make fully generic

* Refactor

* Docs

* Refactor

* Use iter not slice

* Per-origin queues

* Multi-queue processing

* Introduce MaxReady

* Remove MaxReady in favour of ready ring

* Cleanups

* ReadyRing and tests

* Stale page reaping

* from_components -> from_parts

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Move WeightCounter to sp_weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add MockedWeightInfo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Deploy to kitchensink

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use WeightCounter

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Small fixes and logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add service_page

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Typo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Move service_page below service_queue

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add service_message

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use correct weight function

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Overweight execution

* Refactor

* Missing file

* Fix WeightCounter usage in scheduler

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix peek_index

Take into account that decoding from a mutable slice modifies it.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add tests and bench service_page_item

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add debug_info

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add no-progress check to service_queues

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Bound from_message and try_append_message

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add PageReaped event

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename BookStateOf and BookStateFor

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update tests and remove logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove redundant per-message origins; add footprint() and sweep_queue()

* Move testing stuff to mock.rs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add integration test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix no-progress check

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix debug_info

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fixup merge and tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix footprint tracking

* Introduce

* Formatting

* OverweightEnqueued event, auto-servicing config item

* Update tests and benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Provide change handler

* Add missing BookStateFor::insert and call QueueChangeHandler

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update benchmarks and weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use weight metering functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* weightInfo::process_message_payload is gone

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add defensive_saturating_accrue

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename WeightCounter to WeightMeter

Ctr+Shift+H should do the trick.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test on_initialize

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add module docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove origin from MaxMessageLen

The message origin is not encoded into the heap and does
therefore not influence the max message length anymore.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add BoundedVec::as_slice

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test Page::{from_message, try_append_message}

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fixup docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

* Do nothing in sweep_queue if the queue does not exist

... otherwise it inserts default values into the storage.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test ring (un)knitting

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Upgrade stress-test

Change the test to not assume that all queued messages will be
processed in the next block but split it over multiple.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Beauty fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename BoundedVec::as_slice to as_bounded_slice

Conflicts with deref().as_slice() otherwise.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix imports

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove ReadyRing struct

Was used for testing only. Instead use 'fn assert_ring' which also
check the service head and backlinks.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Beauty fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix stale page watermark

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test feature and clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* QueueChanged handler is called correctly

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Abstract testing functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Simplify tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make stuff compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Extend overweight execution benchmark

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove TODOs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test service queue with faulty MessageProcessor

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update pallet ui tests to 1.65

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review doc fixes

Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add weight_limit to extrinsic weight of execute_overweight

* Correctly return unused weight

* Return actual weight consumed in do_execute_overweight

* Review fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Set version 7.0.0-dev

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make it compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Switch message_size to u64

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Switch message_count to u64

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make CI green

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

* Update tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_message_queue

* Dont mention README.md in the Cargo.toml

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove reference to readme

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* zombienet timings adjusted (#12890)

* zombinet tests: add some timeout to allow net spin-up

Sometimes tests are failing at first try, as the pods were not up yet.
Adding timeout should allow the network to spin up properly.

* initial timeout increased to 30s

* Move import queue out of `sc-network` (#12764)

* Move import queue out of `sc-network`

Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.

This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.

* Fix tests

* Apply review comments

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/network/sync/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

* Trace response payload in default `jsonrpsee` middleware (#12886)

* Trace result in default `jsonrpsee` middleware

* `rpc_metrics::extra`

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

* Ensure that we inform all tasks to stop before starting the 60 seconds shutdown (#12897)

* Ensure that we inform all tasks to stop before starting the 60 seconds shutdown

The change of waiting in maximum 60 seconds for the node to shutdown actually introduced a bug. We
were actually waiting always 60 seconds as we didn't informed our tasks to shutdown. The solution to
this problem is to drop the task manager as this will then inform all tasks to end. It also adds
tests to ensure that the behaviors work as expected. (This should already have been done in the
first pr! :()

* ".git/.scripts/fmt.sh" 1

Co-authored-by: command-bot <>

* Safe desired targets call (#12826)

* checked call for desired targets

* fix compile

* fmt

* fix tests

* cleaner with and_then

* Fix typo (#12900)

* ValidateUnsigned: Improve docs. (#12870)

* ValidateUnsigned: Improve docs.

* Review comments

* rpc server with HTTP/WS on the same socket (#12663)

* jsonrpsee v0.16

add backwards compatibility

run old http server on http only

* cargo fmt

* update jsonrpsee 0.16.1

* less verbose cors log

* fix nit in log: WS -> HTTP

* revert needless changes in Cargo.lock

* remove unused features in tower

* fix nits; add client-core feature

* jsonrpsee v0.16.2

* `pallet-message-queue`: Fix license (#12895)

* Fix license

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add mock doc

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use explicit call indices (#12891)

* frame-system: explicit call index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use explicit call indices

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* pallet-template: explicit call index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* DNM: Temporarily require call_index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "DNM: Temporarily require call_index"

This reverts commit c4934e312e12af72ca05a8029d7da753a9c99346.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Pin canonincalized block (#12902)

* Remove implicit approval chilling upon slash. (#12420)

* don't read slashing spans when taking election snapshot

* update cargo.toml

* bring back remote test

* fix merge stuff

* fix npos-voters function sig

* remove as much redundant diff as you can

* Update frame/staking/src/pallet/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* fix

* Update frame/staking/src/pallet/impls.rs

* update lock

* fix all tests

* review comments

* fmt

* fix offence bench

* clippy

* ".git/.scripts/bench-bot.sh" pallet dev pallet_staking

Co-authored-by: Andronik <write@reusable.software>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: command-bot <>

* bounties calls docs fix (#12909)

Co-authored-by: parity-processbot <>

* pallet-contracts migration pre-upgrade fix for v8 (#12905)

* Only run pre-v8 migration check for versions older than 8

* Logix fix

* use custom environment for publishing crates (#12912)

* [contracts] Add debug buffer limit + enforcement (#12845)

* Add debug buffer limit + enforcement

Add debug buffer limit + enforcement

* use BoundedVec for the debug buffer

* revert schedule (debug buf len limit not needed anymore)

* return DispatchError

* addressed review comments

* frame/remote-externalities: Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/rpc: Add previous export

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fixup some wrong dependencies (#12899)

* Fixup some wrong dependencies

Dev dependencies should not appear in the feature list. If features are required, they should be
directly enabled for the `dev-dependency`.

* More fixups

* Fix fix

* Remove deprecated feature

* Make all work properly and nice!!

* FMT

* Fix formatting

* add numerator and denominator to Rational128 Debug impl and increase precision of float representation (#12914)

* Fix state-db pinning (#12927)

* Pin all canonicalized blocks

* Added a test

* Docs

* [ci] add job switcher (#12922)

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Vlad <vladimir@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: tgmichel <telmo@purestake.com>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Luke Schoen <ltfschoen@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Andronik <write@reusable.software>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Dino Pačandi <3002868+Dinonard@users.noreply.github.com>
Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>
Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

* Revert "rpc rebase: Try-runtime Revamp and Facelift (#12921)"

This reverts commit 4ce770a9cb8daf1401529bda7d974b8c703f6b3e.

* Lexnv/kiz revamp try runtime stuff (#12932)

* Introduce sensible weight constants (#12868)

* Introduce sensible weight constants

* cargo fmt

* Remove unused import

* Add missing import

* ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery

Co-authored-by: command-bot <>

* Checkout to the branch HEAD explicitly in `build-linux-substrate` (#12876)

* cli: Improve pruning documentation (#12819)

* cli: Improve pruning documentation

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Keep `finalized` notation and remove `canonical` one

* cli: Fix cargo doc

* cli: `PruningModeClap` IR enum

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Convert PruningModeClap into pruning modes

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Use `PruningModeClap`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Rename to `DatabasePruningMode`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Implement `FromStr` instead of `clap::ValueEnum`

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Update client/cli/src/params/pruning_params.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cli: Add option documentation back

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Apply suggestions from code review

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

* Revert "Move LockableCurrency trait to fungibles::Lockable and deprecate LockableCurrency (#12798)" (#12882)

This reverts commit 9a014d1ecd.

* Don't indefinitely block on shutting down Tokio (#12885)

* Don't indefinitely on shutting down Tokio

Now we wait in maximum 60 seconds before we shutdown the node. Tasks are may be leaked and leading
to some data corruption.

* Drink less :thinking_face:

* General Message Queue Pallet (#12485)

* The message queue

* Make fully generic

* Refactor

* Docs

* Refactor

* Use iter not slice

* Per-origin queues

* Multi-queue processing

* Introduce MaxReady

* Remove MaxReady in favour of ready ring

* Cleanups

* ReadyRing and tests

* Stale page reaping

* from_components -> from_parts

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Move WeightCounter to sp_weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add MockedWeightInfo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Deploy to kitchensink

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use WeightCounter

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Small fixes and logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add service_page

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Typo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Move service_page below service_queue

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add service_message

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use correct weight function

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Overweight execution

* Refactor

* Missing file

* Fix WeightCounter usage in scheduler

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix peek_index

Take into account that decoding from a mutable slice modifies it.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add tests and bench service_page_item

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add debug_info

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add no-progress check to service_queues

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Bound from_message and try_append_message

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add PageReaped event

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename BookStateOf and BookStateFor

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update tests and remove logging

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove redundant per-message origins; add footprint() and sweep_queue()

* Move testing stuff to mock.rs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add integration test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix no-progress check

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix debug_info

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fixup merge and tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix footprint tracking

* Introduce

* Formatting

* OverweightEnqueued event, auto-servicing config item

* Update tests and benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Provide change handler

* Add missing BookStateFor::insert and call QueueChangeHandler

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update benchmarks and weights

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use weight metering functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* weightInfo::process_message_payload is gone

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add defensive_saturating_accrue

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename WeightCounter to WeightMeter

Ctr+Shift+H should do the trick.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test on_initialize

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add module docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove origin from MaxMessageLen

The message origin is not encoded into the heap and does
therefore not influence the max message length anymore.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add BoundedVec::as_slice

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test Page::{from_message, try_append_message}

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fixup docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

* Do nothing in sweep_queue if the queue does not exist

... otherwise it inserts default values into the storage.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test ring (un)knitting

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Upgrade stress-test

Change the test to not assume that all queued messages will be
processed in the next block but split it over multiple.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Beauty fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Rename BoundedVec::as_slice to as_bounded_slice

Conflicts with deref().as_slice() otherwise.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix imports

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove ReadyRing struct

Was used for testing only. Instead use 'fn assert_ring' which also
check the service head and backlinks.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Beauty fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix stale page watermark

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test feature and clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* QueueChanged handler is called correctly

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Abstract testing functions

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Simplify tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make stuff compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Extend overweight execution benchmark

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove TODOs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Test service queue with faulty MessageProcessor

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update pallet ui tests to 1.65

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* More docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review doc fixes

Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add weight_limit to extrinsic weight of execute_overweight

* Correctly return unused weight

* Return actual weight consumed in do_execute_overweight

* Review fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Set version 7.0.0-dev

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make it compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Switch message_size to u64

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Switch message_count to u64

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix benchmarks

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make CI green

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Docs

* Update tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_message_queue

* Dont mention README.md in the Cargo.toml

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove reference to readme

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* zombienet timings adjusted (#12890)

* zombinet tests: add some timeout to allow net spin-up

Sometimes tests are failing at first try, as the pods were not up yet.
Adding timeout should allow the network to spin up properly.

* initial timeout increased to 30s

* Move import queue out of `sc-network` (#12764)

* Move import queue out of `sc-network`

Add supplementary asynchronous API for the import queue which means
it can be run as an independent task and communicated with through
the `ImportQueueService`.

This commit removes removes block and justification imports from
`sc-network` and provides `ChainSync` with a handle to import queue so
it can import blocks and justifications. Polling of the import queue is
moved complete out of `sc-network` and `sc_consensus::Link` is
implemented for `ChainSyncInterfaceHandled` so the import queue
can still influence the syncing process.

* Fix tests

* Apply review comments

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/network/sync/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

* Trace response payload in default `jsonrpsee` middleware (#12886)

* Trace result in default `jsonrpsee` middleware

* `rpc_metrics::extra`

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

* Ensure that we inform all tasks to stop before starting the 60 seconds shutdown (#12897)

* Ensure that we inform all tasks to stop before starting the 60 seconds shutdown

The change of waiting in maximum 60 seconds for the node to shutdown actually introduced a bug. We
were actually waiting always 60 seconds as we didn't informed our tasks to shutdown. The solution to
this problem is to drop the task manager as this will then inform all tasks to end. It also adds
tests to ensure that the behaviors work as expected. (This should already have been done in the
first pr! :()

* ".git/.scripts/fmt.sh" 1

Co-authored-by: command-bot <>

* Safe desired targets call (#12826)

* checked call for desired targets

* fix compile

* fmt

* fix tests

* cleaner with and_then

* Fix typo (#12900)

* ValidateUnsigned: Improve docs. (#12870)

* ValidateUnsigned: Improve docs.

* Review comments

* rpc server with HTTP/WS on the same socket (#12663)

* jsonrpsee v0.16

add backwards compatibility

run old http server on http only

* cargo fmt

* update jsonrpsee 0.16.1

* less verbose cors log

* fix nit in log: WS -> HTTP

* revert needless changes in Cargo.lock

* remove unused features in tower

* fix nits; add client-core feature

* jsonrpsee v0.16.2

* `pallet-message-queue`: Fix license (#12895)

* Fix license

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add mock doc

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use explicit call indices (#12891)

* frame-system: explicit call index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use explicit call indices

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* pallet-template: explicit call index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* DNM: Temporarily require call_index

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "DNM: Temporarily require call_index"

This reverts commit c4934e312e12af72ca05a8029d7da753a9c99346.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Pin canonincalized block (#12902)

* Remove implicit approval chilling upon slash. (#12420)

* don't read slashing spans when taking election snapshot

* update cargo.toml

* bring back remote test

* fix merge stuff

* fix npos-voters function sig

* remove as much redundant diff as you can

* Update frame/staking/src/pallet/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* fix

* Update frame/staking/src/pallet/impls.rs

* update lock

* fix all tests

* review comments

* fmt

* fix offence bench

* clippy

* ".git/.scripts/bench-bot.sh" pallet dev pallet_staking

Co-authored-by: Andronik <write@reusable.software>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: command-bot <>

* bounties calls docs fix (#12909)

Co-authored-by: parity-processbot <>

* pallet-contracts migration pre-upgrade fix for v8 (#12905)

* Only run pre-v8 migration check for versions older than 8

* Logix fix

* use custom environment for publishing crates (#12912)

* [contracts] Add debug buffer limit + enforcement (#12845)

* Add debug buffer limit + enforcement

Add debug buffer limit + enforcement

* use BoundedVec for the debug buffer

* revert schedule (debug buf len limit not needed anymore)

* return DispatchError

* addressed review comments

* frame/remote-externalities: Fix clippy

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* frame/rpc: Add previous export

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* Fixup some wrong dependencies (#12899)

* Fixup some wrong dependencies

Dev dependencies should not appear in the feature list. If features are required, they should be
directly enabled for the `dev-dependency`.

* More fixups

* Fix fix

* Remove deprecated feature

* Make all work properly and nice!!

* FMT

* Fix formatting

* add numerator and denominator to Rational128 Debug impl and increase precision of float representation (#12914)

* Fix state-db pinning (#12927)

* Pin all canonicalized blocks

* Added a test

* Docs

* [ci] add job switcher (#12922)

* Use LOG_TARGET in consensus related crates (#12875)

* Use shared LOG_TARGET in consensus related crates
* Rename target from "afg" to "grandpa"

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Vlad <vladimir@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: tgmichel <telmo@purestake.com>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Luke Schoen <ltfschoen@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Andronik <write@reusable.software>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Dino Pačandi <3002868+Dinonard@users.noreply.github.com>
Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>
Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>

* Revert "Lexnv/kiz revamp try runtime stuff (#12932)"

This reverts commit 378cfb26d984bcde467781f07ef8ddb6998212cb.

* fmt

* update

* fix publish

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Vlad <vladimir@parity.io>
Co-authored-by: Anthony Alaribe <anthonyalaribe@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: tgmichel <telmo@purestake.com>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Luke Schoen <ltfschoen@users.noreply.github.com>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Andronik <write@reusable.software>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: Muharem Ismailov <ismailov.m.h@gmail.com>
Co-authored-by: Dino Pačandi <3002868+Dinonard@users.noreply.github.com>
Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>
Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
2022-12-14 23:59:01 +00:00
Kian Paimani 0b29691688 Remove implicit approval chilling upon slash. (#12420)
* don't read slashing spans when taking election snapshot

* update cargo.toml

* bring back remote test

* fix merge stuff

* fix npos-voters function sig

* remove as much redundant diff as you can

* Update frame/staking/src/pallet/mod.rs

Co-authored-by: Andronik <write@reusable.software>

* fix

* Update frame/staking/src/pallet/impls.rs

* update lock

* fix all tests

* review comments

* fmt

* fix offence bench

* clippy

* ".git/.scripts/bench-bot.sh" pallet dev pallet_staking

Co-authored-by: Andronik <write@reusable.software>
Co-authored-by: Ankan <ankan.anurag@gmail.com>
Co-authored-by: command-bot <>
2022-12-12 17:05:13 +00:00
Niklas Adolfsson 84303ca75d rpc server with HTTP/WS on the same socket (#12663)
* jsonrpsee v0.16

add backwards compatibility

run old http server on http only

* cargo fmt

* update jsonrpsee 0.16.1

* less verbose cors log

* fix nit in log: WS -> HTTP

* revert needless changes in Cargo.lock

* remove unused features in tower

* fix nits; add client-core feature

* jsonrpsee v0.16.2
2022-12-12 10:32:55 +00:00
Niklas Adolfsson e3b269ab0f frame/utils: introduce substrate-rpc-client crate for RPC utils (#12212)
* hack together a PoC

* Update utils/frame/rpc-utils/Cargo.toml

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update utils/frame/rpc-utils/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* rpc_utils -> substrate_rpc_client

* try runtime: remove keep connection

* make CI happy

* cargo fmt

* fix ci

* update lock file

* fix

* fix

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>
2022-10-18 16:39:19 +00:00
Piotr Mikołajczyk 198f94f931 try-runtime::follow-chain - keep connection (#12167)
* Refactor RPC module

* Add flag to `follow-chain`

* Multithreading remark

* fmt

* O_O

* unused import

* cmon

* accidental removal reverted

* remove RpcHeaderProvider

* mut refs

* fmt

* no mutability

* now?

* now?

* arc mutex

* async mutex

* async mutex

* uhm

* connect in constructor

* remove dep

* old import

* another take

* trigger polkadot pipeline

* trigger pipeline
2022-09-06 08:01:35 +00:00
Kian Paimani f67c06ce22 Runtime State Test + Integration with try-runtime (#10174)
* add missing version to dependencies

* Huh

* add features more

* more fixing

* last touches

* it all finally works

* remove some feature gates

* remove unused

* fix old macro

* make it work again

* fmt

* remove unused import

* ".git/.scripts/fmt.sh" 1

* Cleanup more

* fix and rename everything

* a few clippy fixes

* Add try-runtime feature

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* small fixes

* fmt

* Update bin/node-template/runtime/src/lib.rs

* fix build

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: David <dvdplm@gmail.com>

* Update utils/frame/try-runtime/cli/src/commands/execute_block.rs

Co-authored-by: David <dvdplm@gmail.com>

* address all review comments

* fix typos

* revert spec change

* last touches

* update docs

* fmt

* remove some debug_assertions

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: David <dvdplm@gmail.com>
2022-09-01 10:33:22 +00:00
Sebastian Kunert 9f409dc0b8 Prepare for rust 1.62.1 (#11903)
* Update UI test output for rust 1.62.1

* switch ci to staging image to check that everything works

* fix artifacts node-bench-regression-guard

* Imeplement `scale_info::TypeInfo` manually to silence aggressive rust warning

* Fix more clippy lints

* Make clippy happy by relying on auto-deref were possible

* Add tracking issue to the comments

* pin ci image

Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-07-26 14:37:05 +02:00
Falco Hirschenberger b581604aa7 Apply some clippy lints (#11154)
* Apply some clippy hints

* Revert clippy ci changes

* Update client/cli/src/commands/generate.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/inspect_key.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/transactions.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/protocol.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert due to missing `or_default` function.

* Fix compilation and simplify code

* Undo change that corrupts benchmark.

* fix clippy

* Update client/service/test/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

remove leftovers!

* Update client/tracing/src/logging/directives.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/fork-tree/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* added needed ref

* Update frame/referenda/src/benchmarking.rs

* Simplify byte-vec creation

* let's just not overlap the ranges

* Correction

* cargo fmt

* Update utils/frame/benchmarking-cli/src/shared/stats.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Giles Cope <gilescope@gmail.com>
2022-04-30 21:28:27 +00:00
Squirrel 3ecb5306eb Updating shell.nix nightly (#11266)
* The 2022-02-10 nightly can't build some deps

There's a sig that returns `impl IntoIterator<Item = (&'static str, OsString)>` and an Option is being retured but it's incorrectly not allowing a `?` in the method.

* removing duplicate test

* [ci] add job cargo-check-nixos

* add dummy variables nix check

* fix check-dependent jobs

* fix check-dependent-project template

Co-authored-by: alvicsam <alvicsam@gmail.com>
2022-04-27 17:19:07 +00:00
Bastian Köcher f517e57f67 Prepare for rust stable 1.60 (#11138)
* Prepare for rust stable 1.59

Besides preparing the UI tests this also adds a new script update-rust-stable.sh script for
simplifying the update of a rust stable version. This script will run all UI tests for the new
rust stable version and updating the expected output.

* Ensure we run the UI tests in CI

* use staging ci image

* More test updates

* Unignore test (#11097)

* empty commit for pipeline rerun

* empty commit for pipeline rerun

* Try to make clippy happy

* More clippy fixes

* FMT

* ci image production

Co-authored-by: alvicsam <alvicsam@gmail.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
2022-04-11 11:21:54 +02:00
Kian Paimani 6cfdbe5caa Trie version migration pallet (#10073)
* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* migration pallet

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.

* fix test

* fix child migration calls.

* useless closure

* remove remaining state_hash variables.

* Fix and add more tests

* Remove outdated comment

* useless inner hash

* fmt

* remote tests

* finally ksm works

* batches are broken

* clean the benchmarks

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>

* Update frame/state-trie-migration/src/lib.rs

* brand new version

* fix build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update primitives/storage/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* new test structure

* new testing stuff from emeric

* Add commit_all, still not working

* Fix all tests

* add comment

* we have PoV tracking baby

* document stuff, but proof size is still wrong

* FUCK YEAH

* a big batch of review comments

* add more tests

* tweak test

* update config

* some remote-ext stuff

* delete some of the old stuff

* sync more files with master to minimize the diff

* Fix all tests

* make signed migration a bit more relaxed

* add witness check to signed submissions

* allow custom migration to also go above limit

* Fix these pesky tests

* ==== removal of the unsigned stuff ====

* Make all tests work again

* separate the tests from the logic so it can be reused easier

* fix overall build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Slightly better termination

* some final tweaks

* Fix tests

* Restrict access to signed migrations

* address most of the review comments

* fix defensive

* New simplified code

* Fix weights

* fmt

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* make the tests correctly fail

* Fix build

* Fix build

* try and fix the benchmarks

* fix build

* Fix cargo file

* Fix runtime deposit

* make rustdoc happy

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
2022-03-04 15:54:23 +00:00
Niklas Adolfsson 3f69a2e1ff remote ext: make it possible to use an existing ws connection (#10737)
* remote ext: make it possible to pass ws client

* grumbles: introduce `map_uri`
2022-01-28 09:04:52 +00:00
Bastian Köcher f3662b4bba Happy new year 2022 (#10573) 2022-01-03 09:22:14 +01:00
Guillaume Thiolliere cf1eb73046 Deny warning when building doc (#10387)
* deny warning

* add new job

* fix doc

* fmt
2021-12-15 20:28:18 +01:00
Kian Paimani 4775f11edc allow try-runtime and TestExternalities to report PoV size (#10372)
* allow try-runtime and test-externalities to report proof size

* self review

* fix test

* Fix humanized dispaly of bytes

* Fix some test

* Fix some review grumbles

* last of the review comments

* fmt

* remove unused import

* move test

* fix import

* Update primitives/state-machine/src/testing.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* last touches

* fix

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-12-04 06:11:25 +00:00
David eeb80f9e0b Upgrade jsonrpsee to v0.4.1 (#10022)
* Upgrade jsonrpsee to v0.4.1

* remove needless BlockT trait bound

* use default wss port in URL

* Fix try_runtime build

* Partially fix for "remote-tests" feature

* Review feedback

* fmt

* Sort out trait bounds for benches

* Fmt

* fmt again?

* fmt with nightly-2021-09-13

* Upgrade try-runtime as well

* fmt

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
2021-11-11 11:29:28 +00:00
Kian Paimani e01edb0840 new remote-ext mode: (#10192) 2021-11-09 21:26:39 +00:00
Kian Paimani bba9cfd8a2 introduce remote-tests for pallet-bags-list (#10036)
* make a few primitives in bags-list public

* push new stuff

* update

* update log target

* bring remote tests here

* revert pub

* Update frame/bags-list/remote-tests/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* some rev commnets

* Fix

* cleanup

* Update Cargo.lock

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-11-01 14:41:38 +00:00
Kian Paimani aae41d777a allow remote-ext to exclude some keys (#10020)
* allow remote-ext to exclude some keys

* don't use deprecated, just remove.

* rename
2021-10-13 11:12:26 +00:00
Kian Paimani 8f5f89324d follow-chain testing mode for try-runtime (and revamp CLI configs). (#9788)
* deadlock, need to ask someone to help now

* Finally it seems to be working.. at least for a few blocks

* self-review

* major mega revamp

* some small fixes

* another mega refactor

* add license

* Apply suggestions from code review

* hack around signature verification

* Some fixes

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* final tweaks, hopefully.

* a little self-review

* Add the ext root check

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
2021-09-21 15:04:28 +00:00
Kian Paimani 7f53a7f9df Display warning on try-runtime spec_name mismatch (#9593)
* Display warning on try-runtime version mismatch

* detect spec-name mismatch in try-runtime

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/remote-externalities/src/rpc_api.rs

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* remove unused import

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
2021-08-20 18:41:21 +00:00
Kian Paimani 7a32c3504d remove the uselsss weight return type from election provider API (#9569)
* remove the uselsss weight return type from election provider API

* fix everything, should be ready for final benchmark

* simplify on_init a bit furhter

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* remove unwraps

* fmt

* Update lock file

* whitelist block weight

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix warning

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
2021-08-19 07:45:54 +00:00
Kian Paimani abd08e29ce Rewrap all comments to 100 line width (#9490)
* reformat everything again

* manual formatting

* last manual fix

* Fix build
2021-08-11 14:56:55 +00:00
Squirrel aafe64315a Remove extra commas made redundent after rustfmt (#9404)
* Remove extra commas made redundent after rustfmt
2021-07-22 11:06:17 +01:00
Bastian Köcher 7b56ab15b4 Run cargo fmt on the whole code base (#9394)
* Run cargo fmt on the whole code base

* Second run

* Add CI check

* Fix compilation

* More unnecessary braces

* Handle weights

* Use --all

* Use correct attributes...

* Fix UI tests

* AHHHHHHHHH

* 🤦

* Docs

* Fix compilation

* 🤷

* Please stop

* 🤦 x 2

* More

* make rustfmt.toml consistent with polkadot

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-07-21 14:32:32 +00:00
David f5da2ecbeb Upgrade remote-externalities to use jsonrpsee v0.3 (#9342) 2021-07-13 20:01:36 +00:00
Zeke Mostov b9d03f25da try-runtime-cli: Add execute-block subcommand (#9077)
* Refactor remote_externalities::rpc_api

* try-runtime-cli: Adde `execute-block` subcommand

* Trivial

* Address some comments

* Use required_if & remove header-at usage

* Improve doc

* Update comment

* small tweaks

* add overwrite-code to shared params

* Update utils/frame/try-runtime/cli/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* make url a shared param

* add helper for block_at (#9153)

* add helper for block_at

* remove redundant bound

* doc for fn block_at

* Update error message

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
2021-06-22 20:36:12 +00:00
Zeke Mostov f0120d3b62 [try-runtime-cli] Offchain worker support (#8966)
* make remote-ext work with ws and safe RPCs

* Update docs.

* Update utils/frame/remote-externalities/Cargo.toml

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* Fix test

* Update lock file

* Update utils/frame/remote-externalities/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix build again.

* checkpoint, merging the paged rpc now

* revert lifetime stuff

* WIP: remote client init not working

* Small cleanups

* use jsonrpsee alpha.7

* WIP

* Executiing without errors

* Reorg & cleanup

* Trivial cleaning

* Add txpool & keystore extension

* Small cleaning

* More :cleaning

* Flags: page-size, override-code

* WIP

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove heap_pages

* Dry code extraction from state

* Formatting

* More formatting

* Add issue todo

* Use jsonrpsee 0.2.0

* Try trigger gitlab

* Fix "block_import_works" test

* fix native_big_block_import_fails_on_fallback test

* fix commit should work

* Rewrite UI tests

* Revert "Rewrite UI tests"

This reverts commit ada7f670f701c21fb399946a3f6918453f537bcb.

* try again with UI

* Use const for legacy heap pages val

* Move parse module to its own file

* Move rpc_api module to its own file

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* trait names: Block, not B

* Corect HEAP_PAGES_TEST_LEGACY export

* Update utils/frame/remote-externalities/src/rpc_api.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Revert test_ext heap_page insert; adjust storage root instead

* Doc comments for try_runtime::cli::Command

* TryRuntime stub

* trailing comma

* Remove unused dev dep in frame-executive

* Improve parse::hash variable name & error index

* Use Result for rpc_api fns

* Richer err messagges

* Remove HEAP_PAGE_TEST_LEGACY

* Update bin/node/executor/tests/basic.rs

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
2021-06-09 22:05:28 +00:00
Kian Paimani ac3c88fd34 Inject hashed prefix for remote-ext (#8960)
* Inject for remote-ext

* Update utils/frame/remote-externalities/src/lib.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Update utils/frame/remote-externalities/src/lib.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
2021-06-01 13:49:06 +01:00
Kian Paimani b3fabde4e3 make remote ext use batch ws-client (#8916)
* make remote ext use batch ws-client

* Add debug log for key length

* better assertions

* new sanity_checl

* try and make it work with batch

* update test

* remove exctra uri

* add missing at

* remove unused rpc stuff

* improve

Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
2021-05-29 04:20:25 +00:00