Commit Graph

101 Commits

Author SHA1 Message Date
Koute b179b83264 Make sp-keystore no_std-compatible and fix the build-runtimes-polkavm CI job (#3363)
Fixes https://github.com/paritytech/polkadot-sdk/issues/3352
2024-02-18 15:32:01 +00:00
Liam Aharon 3717ec3802 Sync Cargo.toml and crates.io versions (#3034)
Related https://github.com/paritytech/polkadot-sdk/issues/3032

---

Using https://github.com/liamaharon/cargo-workspace-version-tools/ 

`cargo run -- sync --path ../polkadot-sdk`

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2024-01-26 18:14:03 +00:00
Squirrel be8e626806 Set clippy lints in workspace (requires rust 1.74) (#2390)
We currently use a bit of a hack in `.cargo/config` to make sure that
clippy isn't too annoying by specifying the list of lints.

There is now a stable way to define lints for a workspace. The only down
side is that every crate seems to have to opt into this so there's a
*few* files modified in this PR.

Dependencies:

- [x] PR that upgrades CI to use rust 1.74 is merged.

---------

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
2023-12-13 15:11:07 +01:00
Liam Aharon 4a293bc5a2 Enforce consistent and correct toml formatting (#2518)
Using taplo, fixes all our broken and inconsistent toml formatting and
adds CI to keep them tidy.

If people want we can customise the format rules as described here
https://taplo.tamasfe.dev/configuration/formatter-options.html

@ggwpez, I suggest zepter is used only for checking features are
propagated, and leave formatting for taplo to avoid duplicate work and
conflicts.

TODO
- [x] Use `exclude = [...]` syntax in taplo file to ignore zombienet
tests instead of deleting the dir

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-12-01 07:38:02 +00:00
Sebastian Kunert 4902db2198 Use Extensions to register offchain worker custom extensions (#1719)
Closes #1671 

Adds a `type_id` function to the `Extension` trait, allowing to properly store an retrieve
boxed `Extensions`.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-09-29 09:37:51 +02:00
Chevdor a30092ab42 Markdown linter (#1309)
* Add markdown linting

- add linter default rules
- adapt rules to current code
- fix the code for linting to pass
- add CI check

fix #1243

* Fix markdown for Substrate
* Fix tooling install
* Fix workflow
* Add documentation
* Remove trailing spaces
* Update .github/.markdownlint.yaml

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
* Fix mangled markdown/lists
* Fix captalization issues on known words
2023-09-04 12:02:32 +03:00
Oliver Tale-Yazdi dcda0e50f5 Fix build profiles (#1229)
* Fix build profiles

Closes https://github.com/paritytech/polkadot-sdk/issues/1155

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Manually set version to 1.0.0

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use workspace repo

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* 'Authors and Edition from workspace

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-08-29 13:39:41 +02:00
alvicsam f441a5fc93 Diener workspacify
Signed-off-by: alvicsam <alvicsam@gmail.com>
2023-08-25 11:05:17 +02:00
Oliver Tale-Yazdi 8b9455465b Format and Sort features in Cargo.toml files (#14803)
* CI: Add feature sorting check

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Sort all features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add some mistakes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Add some mistakes"

This reverts commit b2b1099f979f6decb22d09b46689c1554bb72e81.

* CI job naming

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add oneliner formatting

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Explain tool

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use latest version

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Better erorr message

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Format after master merge

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use --check option

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Messed up the merge commit...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-08-23 14:21:52 +00:00
Bastian Köcher 5eb816d7a6 Removal of execution strategies (#14387)
* Start

* More work!

* Moar

* More changes

* More fixes

* More worrk

* More fixes

* More fixes to make it compile

* Adds `NoOffchainStorage`

* Pass the extensions

* Small basti making small progress

* Fix merge errors and remove `ExecutionContext`

* Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`

Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
`ExecutionExtension` which provides the default extensions.

* Fix compilation

* Register the global extensions inside runtime api instance

* Fixes

* Fix `generate_initial_session_keys` by passing the keystore extension

* Fix the grandpa tests

* Fix more tests

* Fix more tests

* Don't set any heap pages if there isn't an override

* Fix small fallout

* FMT

* Fix tests

* More tests

* Offchain worker custom extensions

* More fixes

* Make offchain tx pool creation reusable

Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
registered in the runtime externalities context. This factory will be required for a later pr to
make the creation of offchain transaction pools easier.

* Fixes

* Fixes

* Set offchain transaction pool in BABE before using it in the runtime

* Add the `offchain_tx_pool` to Grandpa as well

* Fix the nodes

* Print some error when using the old warnings

* Fix merge issues

* Fix compilation

* Rename `babe_link`

* Rename to `offchain_tx_pool_factory`

* Cleanup

* FMT

* Fix benchmark name

* Fix `try-runtime`

* Remove `--execution` CLI args

* Make clippy happy

* Forward bls functions

* Fix docs

* Update UI tests

* Update client/api/src/execution_extensions.rs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/cli/src/params/import_params.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-authored-by: Koute <koute@users.noreply.github.com>

* Pass the offchain storage to the MMR RPC

* Update client/api/src/execution_extensions.rs

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Review comments

* Fixes

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
2023-07-11 14:21:38 +00:00
Koute 9d1741eb3c Bump parity-scale-codec to 3.6.1 (#14428) 2023-06-21 11:37:11 +00:00
Alexandru Vasile 3da9449067 Bump sp-crates from latest crates.io version + release (#14265)
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-05-31 12:11:01 +00:00
Alexandru Vasile 98a0550ea2 Release: Bump SP crates to release on crates.io (#14237)
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-05-29 10:40:59 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Gavin Wood 2d7fa18e73 Make DispatchError impl MEL (#13169)
* Make DispatchError impl MEL

* Upgrade SCALE codec to support `codec(skip)` for MEL

Co-authored-by: Bastian Köcher <info@kchr.de>
2023-01-19 16:47:47 +00:00
Bastian Köcher 0c934a9352 ed25519_verify: Support using dalek for historical blocks (#12661)
* ed25519_verify: Support using dalek for historical blocks

The switch from `ed25519-dalek` to `ed25519-zebra` was actually a breaking change. `ed25519-zebra`
is more permissive. To support historical blocks when syncing a chain this pull request introduces
an externalities extension `UseDalekExt`. This extension is just used as a signaling mechanism to
`ed25519_verify` to use `ed25519-dalek` when it is present. Together with `ExtensionBeforeBlock` it
can be used to setup a node in way to sync historical blocks that require `ed25519-dalek`, because
they included a transaction that verified differently as when using `ed25519-zebra`.

This feature can be enabled in the following way. In the chain service file, directly after the
client is created, the following code should be added:

```
use sc_client_api::ExecutorProvider;
client.execution_extensions().set_extensions_factory(
	sc_client_api::execution_extensions::ExtensionBeforeBlock::<Block, sp_io::UseDalekExt>::new(BLOCK_NUMBER_UNTIL_DALEK_SHOULD_BE_USED)
);
```

* Fix doc

* More fixes

* Update client/api/src/execution_extensions.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix merge and warning

* Fix docs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2022-11-27 15:34:07 +00:00
Niklas Adolfsson 2b8af8cb1a release sp-core 7.0.0 and sp-runtime 7.0.0 (#12599)
* chore(release): sp-core v7.0.0

* chore(release): sp-runtime v7.0.0

* fix bad merge
2022-11-15 14:54:14 +00:00
Gavin Wood ecbd65fb95 Safe and sane multi-item storage removal (#11490)
* Fix overlay prefix removal result

* Second part of the overlay prefix removal fix.

* Report only items deleted from storage in clear_prefix

* Fix kill_prefix

* Formatting

* Remove unused code

* Fixes

* Fixes

* Introduce clear_prefix host function v3

* Formatting

* Use v2 for now

* Fixes

* Formatting

* Docs

* Child prefix removal should also hide v3 for now

* Fixes

* Fixes

* Formatting

* Fixes

* apply_to_keys_whle takes start_at

* apply_to_keys_whle takes start_at

* apply_to_keys_whle takes start_at

* Cursor API; force limits

* Use unsafe deprecated functions

* Formatting

* Fixes

* Grumbles

* Fixes

* Docs

* Some nitpicks 🙈

* Update primitives/externalities/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Formatting

* Fixes

* cargo fmt

* Fixes

* Update primitives/io/src/lib.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Formatting

* Fixes

Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2022-05-29 12:56:26 +01:00
Shawn Tabrizi 7d5202341b Basic TOML Lint (#11348)
* basic lint

* lint ordering
2022-05-04 13:38:54 +00:00
Falco Hirschenberger b581604aa7 Apply some clippy lints (#11154)
* Apply some clippy hints

* Revert clippy ci changes

* Update client/cli/src/commands/generate.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/inspect_key.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/transactions.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/protocol.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert due to missing `or_default` function.

* Fix compilation and simplify code

* Undo change that corrupts benchmark.

* fix clippy

* Update client/service/test/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

remove leftovers!

* Update client/tracing/src/logging/directives.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/fork-tree/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* added needed ref

* Update frame/referenda/src/benchmarking.rs

* Simplify byte-vec creation

* let's just not overlap the ranges

* Correction

* cargo fmt

* Update utils/frame/benchmarking-cli/src/shared/stats.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Giles Cope <gilescope@gmail.com>
2022-04-30 21:28:27 +00:00
Andrew Jones a3a7fe9041 Release primitive crates (#10937)
* cargo set-version --bump major -p pallet-contracts-primitives

* cargo set-version --bump major -p sp-core

* cargo set-version --bump major -p sp-runtime-interface

* cargo set-version --bump major -p sp-wasm-interface

* cargo set-version --bump major -p sp-runtime

* cargo set-version --bump major -p sp-storage

* cargo set-version --bump major -p sp-rpc

* cargo set-version --bump major -p sp-io

* cargo set-version --bump major -p sp-trie

* cargo set-version -p sp-state-machine  -- 0.12.0

* cargo set-version -p sp-externalities  -- 0.12.0

* cargo set-version -p sp-keystore -- 0.12.0

* cargo set-version --bump major -p sp-keyring

* cargo set-version --bump major -p sp-version

* cargo set-version --bump major -p sp-tracing

* cargo set-version --bump major -p sp-application-crypto

* cargo set-version --bump major -p sp-arithmetic

* cargo unleash version bump-major -p sp-runtime-interface-proc-macro

* Add codec max-encoded-len feature to sp-arithmetic

* cargo unleash version bump-major -p sp-core-hashing-proc-macro
2022-03-01 14:35:23 +01:00
wigy 535325d2e6 Upgrading parity-scale-codec to v3 (#10825)
* Upgraded dependencies

* Adapting code to scale v3

* Empty commit to trigger CI

* Triggering CI

* Fixing UI test

* Remove superfluous dev-dep added by #9228

* Cryout for CI
2022-02-24 15:34:34 +01:00
Alexander Theißen 3938c1cb62 Release pallet-contracts-primitives (#10772)
* Bump sp-storage

* Bump sp-rpc

* Bump sp-io

* Bump sp-trie

* Bump sp-state-machine

* Bump sp-externalities

* Bump sp-keystore

* Bump sp-application-crypto

* Bump pallet-contracts-primitives

* Bump sp-core

* Bump sp-runtime-interface

* Bump sp-wasm-interface

* Bump sp-runtime

* Bump sp-storage

* Update lock file

* Revert "Bump sp-storage"

This reverts commit f86c6db1ae334e28207b658fd90714ade0332e3b.

* Update lock file
2022-02-01 20:38:02 +01:00
Bastian Köcher f3662b4bba Happy new year 2022 (#10573) 2022-01-03 09:22:14 +01:00
cheme 4c651637f2 Inner hashing of value in state trie (runtime versioning). (#9732)
* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.

* useless closure

* remove remaining state_hash variables.

* Remove outdated comment

* useless inner hash

* fmt

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* Apply review change (docs and error).

* fmt

* use explicit runtime_interface in doc test

* fix ui test

* fix doc test

* fmt

* use default for path and specname when resolving version.

* small review related changes.

* doc value size requirement.

* rename old_state feature

* Remove macro changes

* feature rename

* state version as host function parameter

* remove flag for client api

* fix tests

* switch storage chain proof to V1

* host functions, pass by state version enum

* use WrappedRuntimeCode

* start

* state_version in runtime version

* rust fmt

* Update storage proof of max size.

* fix runtime version rpc test

* right intent of convert from compat

* fix doc test

* fix doc test

* split proof

* decode without replay, and remove some reexports.

* Decode with compatibility by default.

* switch state_version to u8. And remove RuntimeVersionBasis.

* test

* use api when reading embedded version

* fix decode with apis

* extract core version instead

* test fix

* unused import

* review changes.

Co-authored-by: kianenigma <kian@parity.io>
2021-12-24 08:54:07 +00:00
David 92dc635f3c Release sp-core 4.0.0 (#10447)
* Bump versions of sp-core and dependencies to v4.0.0

* Update references from `4.0.0-dev` –> `4.0.0`

* Funny whitespace

* Funny whitespace 2
2021-12-08 21:06:50 +01:00
Arkadiy Paronyan 4cbbf0cf43 Kill the light client, CHTs and change tries. (#10080)
* Remove light client, change tries and CHTs

* Update tests

* fmt

* Restore changes_root

* Fixed benches

* Cargo fmt

* fmt

* fmt
2021-11-12 14:15:01 +01:00
Bastian Köcher b08d3017a8 Switch to Rust 2021 (#10170)
* Switch to Rust 2021

* Update trybuild to fix errors
2021-11-04 19:54:48 +01:00
Dan Shields 8db7ec9cc9 Migrate to docs.substrate.io throughout (#10007)
* all reverences of substrate.dev replaced with *correct* docs.substrate.io links

* fix links in non-cargo files, errors in replace

* homepage https://substrate.io

* cargo +nightly fmt

* FMT

Co-authored-by: Bastian Köcher <info@kchr.de>
2021-10-13 12:56:56 -06:00
Kian Paimani abd08e29ce Rewrap all comments to 100 line width (#9490)
* reformat everything again

* manual formatting

* last manual fix

* Fix build
2021-08-11 14:56:55 +00:00
Bastian Köcher 7b56ab15b4 Run cargo fmt on the whole code base (#9394)
* Run cargo fmt on the whole code base

* Second run

* Add CI check

* Fix compilation

* More unnecessary braces

* Handle weights

* Use --all

* Use correct attributes...

* Fix UI tests

* AHHHHHHHHH

* 🤦

* Docs

* Fix compilation

* 🤷

* Please stop

* 🤦 x 2

* More

* make rustfmt.toml consistent with polkadot

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-07-21 14:32:32 +00:00
Benjamin Kampmann c78c2892a4 Attempting to fix publishing (#9140)
* mark template and utils as non-publish

* switch to development version for testing

* activate unleash check

* maybe if I disable all rules...

* Fix isolated compilation of `max-encoded-len-derive` with `syn`

error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
  --> src/lib.rs:88:29
   |
88 |             .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
   |                            --------- ^^ ----------------------------------- _
   |                            |
   |                            syn::Path

error: aborting due to previous error

For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`

* WIP: bump changes crates since v3 tag to next breaking

cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0

FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`

* Update lockfile

* WIP: Bump sp-transaction-pool as well

* WIP: Bump sp-offchain as well

* WIP: Bump frame-system-rpc-runtime-api as well

* WIP: Bump sp-authority-discovery as well

* Manually deactivate dev-deps before `cargo unleash check`

Otherwise we run into `Cycle detected` error.

* Bump sp-consensus-slots

* Add missing Cargo.lock change

* Bump sp-consensus-vrf as well

* Bump sp-keyring as well

* Bump sp-consensus-pow as well

* Try to speed up the `unleash-check` job

Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help

* fixup: Ensure the temp target dir exists for unleash check

* Bump pallet-transaction-payment-rpc-runtime-api as well

Needed for Polkadot

* Bump pallet-transaction-payment-rpc as well

Needed for Polkadot

* Try updating crates after patching in the Polkadot CI job

* Use another approach to update patched Substrate crates

* Try to update all sp-core versions in Polkadot CI job

* Simplify sp-core version checking

* Apply another shellcheck lint

* Just do the simplest thing I guess

* Welp don't do --offline then

* Clean up `unleash-check` job triggers

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Fix a note in unleash-check cache step

* Add a note about temporary optimization in cargo-unleash

* Pin a newer version of cargo-unleash

Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2021-07-11 12:17:53 +00:00
Shawn Tabrizi b42b8fc5fb Include StorageInfo in Benchmarking Pipeline (#9090)
* extend storageinfo

* extend_storage_info

* use vec

* add storage info to pipeline

* get read and written keys

* undo storageinfo move

* refactor keytracker

* return read / write count

* playing with key matching

* add basic `StorageInfo` constructor

* add whitelisted to returned info

* fix some test stuff

* pipe comments into benchmark data

* add_storage_comments

* add comments to template

* track only storage prefix

* Update frame/benchmarking/src/lib.rs

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix test

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* remove test logs

* add temp benchmark script

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* remove keytracker and use trackedstoragekey

* add comment for unknown keys

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_timestamp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/timestamp/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* remove duplicate comments with unknown keys

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_timestamp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/timestamp/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* refactor bench tracker, and fix results

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/balances/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix child tries in new tracker

* extra newline

* fix unused warning

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_timestamp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/timestamp/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix master merge

* storage info usage refactor

* remove now unused

* fix refactor

* use a vec for prefix

* fix tests

* also update writer to use vec

* disable read and written keys for now

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=frame_system --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/system/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update frame/system/src/weights.rs

* fix test

* Delete weights.rs

* reset weights

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-07-07 22:06:06 +00:00
cheme 693b39f43b Frame remove_all with size limit. (#9106)
* remove prefixed content with limit.

* test match

* factor comment and factor ext limit removal.

* fix benchmark

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-06-15 13:23:58 +00:00
Arkadiy Paronyan 84811dae00 Storage chain: Runtime module (#8624)
* Transaction storage runtime module

* WIP: Tests

* Tests, benchmarks  and docs

* Made check_proof mandatory

* Typo

* Renamed a crate

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Added weight for on_finalize

* Fixed counter mutations

* Reorganized tests

* Fixed build

* Update for the new inherent API

* Reworked for the new inherents API

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Store transactions in a Vec

* Added FeeDestination

* Get rid of constants

* Fixed node runtime build

* Fixed benches

* Update frame/transaction-storage/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-06-04 06:50:59 +00:00
Nikolay Volf 31c5ddc1ce update environmental (#8789) 2021-05-12 14:48:01 +02:00
Guillaume Thiolliere bb343c7c26 fix doc and dyn (#8631) 2021-04-17 17:01:31 +02:00
Shawn Tabrizi c04b44b0a3 Add PoV Tracking to Benchmarking Pipeline (#8559)
* Added a function to estimate proof size for benchmarking

* integrate proof_size into benchmarking pipeline

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fix tests

* one more test

* Update bench.rs

* Update utils/frame/benchmarking-cli/src/writer.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2021-04-08 12:54:46 +02:00
Arkadiy Paronyan 4a0d6d9490 Storage chains: indexing, renewals and reference counting (#8265)
* Transaction indexing

* Tests and fixes

* Fixed a comment

* Style

* Build

* Style

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Code review suggestions

* Add missing impl

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* impl JoinInput

* Don't store empty slices

* JoinInput operates on slices

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-03-18 11:46:27 +00:00
Shawn Tabrizi d25229bc89 Return number of keys removed when calling storage_kill on child trie (#8166)
* Initial piping of returning amount of keys killed

* One more test for `None` limit

* forgot to update

* fix return value

* use version 3

* Update to return `KillOutcome`

* Update name to KillChildStorageResult
2021-02-22 19:24:12 +00:00
Benjamin Kampmann 6a964c5df0 Releasing 3.0 (#8098)
* bumping version for next release
* add changelog
* add guide
2021-02-10 19:23:18 +01:00
Guillaume Thiolliere f48296e9ad Update parity-scale-codec to 2.0 (#7994)
* update cargo.toml

* use 2.0 in mmmr
2021-01-29 12:22:45 +00:00
Guillaume Thiolliere 014e6f03c6 bumpd minor version (#7873) 2021-01-12 09:25:47 +01:00
Igor Matuszewski 028f3d2674 Merge 2.0.1 backport branch into mainline master (#7842)
* Backport paritytech/substrate#7381

* Bring back genesis storage build in aura/timestamp

To not change spec version, see
https://github.com/paritytech/substrate/pull/7686#discussion_r540032743

* Backport paritytech/substrate#7238

* Backport paritytech/substrate#7395

* Bump impl_version

* Fix UI tests and bump trybuild dep

See https://github.com/rust-lang/rust/pull/73996

Backports:
https://github.com/paritytech/substrate/pull/7764
https://github.com/paritytech/substrate/pull/7656

* Partially backport paritytech/substrate#7838

* Release frame-support with a dep compilation fix

* Bump patch level for remaining crates

This is done because at the time of writing cargo-unleash does not fully
support partial workspace publishing and mixes both local and crates.io
versions of the packages, leading to errors in the release check workflow.

* Backport paritytech/substrate#7854

...to fix compilation error when using futures-* v0.3.9.

* Adding Changelog  entry for patch release

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
2021-01-11 11:24:29 +01:00
kaichao d2eb87f5fc minor fix (#7828) 2021-01-05 12:55:21 +01:00
Bastian Köcher e3e651f72c Happy new year (#7814)
* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL
2021-01-04 09:03:13 +00:00
Alexander Theißen ec2c70396e Define ss58 prefix inside the runtime (#7810)
* Add SS58Prefix type to the frame_system config trait

* Remove unused chain_id runtime interface
2020-12-30 22:07:37 +00:00
Alexander Theißen 9ce24fe1f4 Allow capping the amount of work performed when deleting a child trie (#7671)
* Allow Backend::for_keys_in_child_storage to be aborted by the closure

* Ext::kill_child_storage now takes an upper limit for backend deletion

* Add Storage::storage_kill_limited() runtime interface

* review: Use a new version of kill_storage instead of a new interface

* review: Simplify boolean expression

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* review: Rename for_keys_in_child_storage

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
2020-12-09 01:17:28 +00:00
Nikolay Volf a062bc2f1d Runtime worker threads (#7089)
* std variant

* principal work

* format and naming

* format and naming continued

* working nested fork

* add comment

* naming and tabs

* line width

* fix wording

* address review

* refactor dynamic dispatch

* update wasmtime

* some care

* move ext

* more refactor

* doc effort

* simplify

* doc effort

* tests and docs

* address review

* naming

* explain some args

* add example

* unwinding for native and tests

* rename stray

* fix refs

* fix tests

* fix warnings

* stray naming

* fixes and comments

* Update primitives/io/src/tasks.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* make examples "compile"

* dyn_dispatch -> spawn_call

* fix impl

* address review

* Update primitives/io/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update primitives/io/src/async_externalities.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/example-parallel/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* fix compilation

* Update client/executor/common/src/wasm_runtime.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* address review

* Update client/executor/wasmtime/src/instance_wrapper.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Update client/executor/src/native_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/native_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/wasmtime/src/instance_wrapper.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* address some issues

* address more issues

* wasm_only interface

* define sp_tasks

* avoid anyhow

* fix example

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-10-20 12:41:51 +00:00
Bastian Köcher dbf4558633 Remove runtime registered extensions after execution (#7236)
* Remove runtime registered extensions after execution

This prevents a bug when an extension was registered in native, but the
native execution aborted without removing the extension again. Now, when
executing the wasm code the extension is still registered and it fails
of being registered. So, the wasm execution fails as well. This can
happen for example for the `VerificationExt`.

* Make it better
2020-10-01 09:12:05 +00:00