Commit Graph

8915 Commits

Author SHA1 Message Date
Liam Aharon d715caa63a Improve try-state developer experience & fix bug (#2019)
Making some devex improvements as I audit our chains adherence to
try-state invariants, in preparation for automated try-state checks and
alerting.

Note to reviewer: while you're here, if you have time would be great to
get your eyes on https://github.com/paritytech/polkadot-sdk/pull/1297
also since it touches a similar file and I'd like to avoid merge
conflicts :P

## Devex Improvements

- Changes the log level of logs informing the user that try-state checks
are being run for a pallet from debug to info
- Improves how errors are communicated
- Errors are logged when they are encountered, rather than after
everything has been executed
- Exact pallet the error originated from is included with the error log
  - Clearly see all errors and how many there are, rather than only one
  - Closes #136 

### Example of new logs

<img width="1185" alt="Screenshot 2023-10-25 at 15 44 44"
src="https://github.com/paritytech/polkadot-sdk/assets/16665596/b75588a2-1c64-45df-bbc8-bcb8bf8b0fe0">

### Same but with old logs (run with RUST_LOG=debug)

Notice only informed of one of the errors, and it's unclear which pallet
it originated

<img width="1185" alt="Screenshot 2023-10-25 at 15 39 01"
src="https://github.com/paritytech/polkadot-sdk/assets/16665596/e3429cb1-489e-430a-9716-77c052e5dae6">
 

## Bug fix

When dry-running migrations and `checks.try_state()` is `true`, only run
`try_state` checks after migrations have been executed. Otherwise,
`try_state` checks that expect state to be in at a HIGHER storage
version than is on-chain could incorrectly fail.

---------

Co-authored-by: command-bot <>
2023-10-30 09:43:32 +01:00
Davide Galassi 7035034710 Improve Client CLI help readability (#2073)
Currently the CLI `-h/--help` commad output is almost unreadable as (for
some commands) it:
- doesn't provide a short brief of what the command does.
- doesn't separate the options description in smaller paragraphs.
- doesn't use a smart wrap strategy for lines longer than the number of
columns in the terminal.

Follow some pics taken with a 100 cols wide term

## Short help (./node -h)

### Before


![20231028-174531-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/11b62c3c-dcd5-43f4-ac58-f1b299e3f4b9)

### After


![20231028-175041-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/dc08f6fd-b287-40fb-8b33-71a185922104)


## Long help (./node --help)

### Before


![20231028-175257-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/9ebdc0ae-54ee-4760-b873-a7e813523cb6)

### After


![20231028-175155-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/69cbe5cb-eb2f-46a5-8ebf-76c0cf8c4bad)

---------

Co-authored-by: command-bot <>
2023-10-29 18:25:33 +01:00
Vadim Smirnov 8ce16ee6ff fix(frame-benchmarking-cli): Pass heap_pages param to WasmExecutor (#2075)
In https://github.com/paritytech/substrate/pull/13740 the use of the
`heap-pages` param inside the `frame-benchmarking-cli` has been removed.
This results in running out of memory and this PR fixes the heap
allocation strategy for benchmarks wasm executor.
2023-10-29 13:20:40 +01:00
Sam Johnson f46f5a90f6 upgrade to docify 0.2.6 (#2069)
Updates `docify` to 0.2.6, which fixes a bug that was preventing nesting
`#[docify::export]` within sub-items of items that already have
`#[docify::export]` attached to them from working properly.

Release notes here:
https://github.com/sam0x17/docify/releases/tag/v0.2.6

cc @ggwpez @kianenigma
2023-10-27 21:43:51 +02:00
Juan Girini 43415ef58c feat: FRAME umbrella crate. (#1337)
### Original PR https://github.com/paritytech/substrate/pull/14137

This PR brings in the first version of the "_`frame` umbrella crate_".
This crate is intended to serve two purposes:

1. documentation
2. easier development with frame. Ideally, we want most users to be able
to build a frame-based pallet and runtime using just `frame` (plus
`scale-codec` and `scale-info`).

The crate is not finalized and is not yet intended for external use.
Therefore, the version is set to `0.0.1-dev`, this PR is `silent`, and
the entire crate is hidden behind the `experimental` flag. The main
intention in merging it early on is to be able to iterate on it in the
rest of
[`developer-hub`](https://github.com/paritytech/polkadot-sdk-docs/)
efforts.

The public API of the `frame` crate is at the moment as follows: 

```
pub mod frame
pub use frame::log
pub use frame::pallet
pub mod frame::arithmetic
pub use frame::arithmetic::<<sp_arithmetic::*>>
pub use frame::arithmetic::<<sp_arithmetic::traits::*>>
pub mod frame::deps
pub use frame::deps::codec
pub use frame::deps::frame_executive
pub use frame::deps::frame_support
pub use frame::deps::frame_system
pub use frame::deps::scale_info
pub use frame::deps::sp_api
pub use frame::deps::sp_arithmetic
pub use frame::deps::sp_block_builder
pub use frame::deps::sp_consensus_aura
pub use frame::deps::sp_consensus_grandpa
pub use frame::deps::sp_core
pub use frame::deps::sp_inherents
pub use frame::deps::sp_io
pub use frame::deps::sp_offchain
pub use frame::deps::sp_runtime
pub use frame::deps::sp_std
pub use frame::deps::sp_version
pub mod frame::derive
pub use frame::derive::CloneNoBound
pub use frame::derive::Debug
pub use frame::derive::Debug
pub use frame::derive::DebugNoBound
pub use frame::derive::Decode
pub use frame::derive::Decode
pub use frame::derive::DefaultNoBound
pub use frame::derive::Encode
pub use frame::derive::Encode
pub use frame::derive::EqNoBound
pub use frame::derive::PartialEqNoBound
pub use frame::derive::RuntimeDebug
pub use frame::derive::RuntimeDebugNoBound
pub use frame::derive::TypeInfo
pub use frame::derive::TypeInfo
pub mod frame::prelude
pub use frame::prelude::<<frame_support::pallet_prelude::*>>
pub use frame::prelude::<<frame_system::pallet_prelude::*>>
pub use frame::prelude::<<sp_std::prelude::*>>
pub use frame::prelude::CloneNoBound
pub use frame::prelude::Debug
pub use frame::prelude::Debug
pub use frame::prelude::DebugNoBound
pub use frame::prelude::Decode
pub use frame::prelude::Decode
pub use frame::prelude::DefaultNoBound
pub use frame::prelude::Encode
pub use frame::prelude::Encode
pub use frame::prelude::EqNoBound
pub use frame::prelude::PartialEqNoBound
pub use frame::prelude::RuntimeDebug
pub use frame::prelude::RuntimeDebugNoBound
pub use frame::prelude::TypeInfo
pub use frame::prelude::TypeInfo
pub use frame::prelude::frame_system
pub mod frame::primitives
pub use frame::primitives::BlakeTwo256
pub use frame::primitives::H160
pub use frame::primitives::H256
pub use frame::primitives::H512
pub use frame::primitives::Hash
pub use frame::primitives::Keccak256
pub use frame::primitives::U256
pub use frame::primitives::U512
pub mod frame::runtime
pub mod frame::runtime::apis
pub use frame::runtime::apis::<<frame_system_rpc_runtime_api::*>>
pub use frame::runtime::apis::<<sp_api::*>>
pub use frame::runtime::apis::<<sp_block_builder::*>>
pub use frame::runtime::apis::<<sp_consensus_aura::*>>
pub use frame::runtime::apis::<<sp_consensus_grandpa::*>>
pub use frame::runtime::apis::<<sp_offchain::*>>
pub use frame::runtime::apis::<<sp_session::runtime_api::*>>
pub use frame::runtime::apis::<<sp_transaction_pool::runtime_api::*>>
pub use frame::runtime::apis::ApplyExtrinsicResult
pub use frame::runtime::apis::CheckInherentsResult
pub use frame::runtime::apis::InherentData
pub use frame::runtime::apis::OpaqueMetadata
pub use frame::runtime::apis::impl_runtime_apis
pub use frame::runtime::apis::sp_api
pub mod frame::runtime::prelude
pub use frame::runtime::prelude::<<frame_executive::*>>
pub use frame::runtime::prelude::ConstBool
pub use frame::runtime::prelude::ConstI128
pub use frame::runtime::prelude::ConstI16
pub use frame::runtime::prelude::ConstI32
pub use frame::runtime::prelude::ConstI64
pub use frame::runtime::prelude::ConstI8
pub use frame::runtime::prelude::ConstU128
pub use frame::runtime::prelude::ConstU16
pub use frame::runtime::prelude::ConstU32
pub use frame::runtime::prelude::ConstU64
pub use frame::runtime::prelude::ConstU8
pub use frame::runtime::prelude::NativeVersion
pub use frame::runtime::prelude::RuntimeVersion
pub use frame::runtime::prelude::construct_runtime
pub use frame::runtime::prelude::create_runtime_str
pub use frame::runtime::prelude::derive_impl
pub use frame::runtime::prelude::frame_support
pub use frame::runtime::prelude::ord_parameter_types
pub use frame::runtime::prelude::parameter_types
pub use frame::runtime::prelude::runtime_version
pub mod frame::runtime::testing_prelude
pub use frame::runtime::testing_prelude::BuildStorage
pub use frame::runtime::testing_prelude::Storage
pub mod frame::runtime::types_common
pub type frame::runtime::types_common::AccountId = <<frame::runtime::types_common::Signature as sp_runtime::traits::Verify>::Signer as sp_runtime::traits::IdentifyAccount>::AccountId
pub type frame::runtime::types_common::BlockNumber = u32
pub type frame::runtime::types_common::BlockOf<T, Extra> = sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<frame::runtime::types_common::BlockNumber, sp_runtime::traits::BlakeTwo256>, sp_runtime::generic::unchecked_extrinsic::UncheckedExtrinsic<sp_runtime::multiaddress::MultiAddress<frame::runtime::types_common::AccountId, ()>, <T as frame_system::pallet::Config>::RuntimeCall, frame::runtime::types_common::Signature, Extra>>
pub type frame::runtime::types_common::OpaqueBlock = sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<frame::runtime::types_common::BlockNumber, sp_runtime::traits::BlakeTwo256>, sp_runtime::OpaqueExtrinsic>
pub type frame::runtime::types_common::Signature = sp_runtime::MultiSignature
pub type frame::runtime::types_common::SystemSignedExtensionsOf<T> = (frame_system::extensions::check_non_zero_sender::CheckNonZeroSender<T>, frame_system::extensions::check_spec_version::CheckSpecVersion<T>, frame_system::extensions::check_tx_version::CheckTxVersion<T>, frame_system::extensions::check_genesis::CheckGenesis<T>, frame_system::extensions::check_mortality::CheckMortality<T>, frame_system::extensions::check_nonce::CheckNonce<T>, frame_system::extensions::check_weight::CheckWeight<T>)
pub mod frame::testing_prelude
pub use frame::testing_prelude::<<frame_executive::*>>
pub use frame::testing_prelude::<<frame_system::mocking::*>>
pub use frame::testing_prelude::BuildStorage
pub use frame::testing_prelude::ConstBool
pub use frame::testing_prelude::ConstI128
pub use frame::testing_prelude::ConstI16
pub use frame::testing_prelude::ConstI32
pub use frame::testing_prelude::ConstI64
pub use frame::testing_prelude::ConstI8
pub use frame::testing_prelude::ConstU128
pub use frame::testing_prelude::ConstU16
pub use frame::testing_prelude::ConstU32
pub use frame::testing_prelude::ConstU64
pub use frame::testing_prelude::ConstU8
pub use frame::testing_prelude::NativeVersion
pub use frame::testing_prelude::RuntimeVersion
pub use frame::testing_prelude::Storage
pub use frame::testing_prelude::TestState
pub use frame::testing_prelude::assert_err
pub use frame::testing_prelude::assert_err_ignore_postinfo
pub use frame::testing_prelude::assert_error_encoded_size
pub use frame::testing_prelude::assert_noop
pub use frame::testing_prelude::assert_ok
pub use frame::testing_prelude::assert_storage_noop
pub use frame::testing_prelude::construct_runtime
pub use frame::testing_prelude::create_runtime_str
pub use frame::testing_prelude::derive_impl
pub use frame::testing_prelude::frame_support
pub use frame::testing_prelude::frame_system
pub use frame::testing_prelude::if_std
pub use frame::testing_prelude::ord_parameter_types
pub use frame::testing_prelude::parameter_types
pub use frame::testing_prelude::runtime_version
pub use frame::testing_prelude::storage_alias
pub mod frame::traits
pub use frame::traits::<<frame_support::traits::*>>
pub use frame::traits::<<sp_runtime::traits::*>>
```

---

The road to full stabilization is

- [ ] https://github.com/paritytech/polkadot-sdk/issues/127
- [ ] have a more intentional version bump, as opposed to the current bi
weekly force-major-bump
- [ ] revise the internal API of `frame`, especially what goes into the
`prelude`s.
- [ ] migrate all internal pallets and runtime to use `frame`

---------

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
2023-10-27 11:38:16 +02:00
Sam Johnson 6ca5789db8 upgrade docify to 0.2.5 (#2052)
Updates `docify` to 0.2.5, which fixes some indentation bugs and adds
the new `#[docify::export_content]` attribute which can be used like
regular `#[docify::export]` but will only export the _underlying
contents_ of the item it is attached to, if applicable (otherwise it
just behaves exactly like `#[docify::export]`).

Release notes here:
https://github.com/sam0x17/docify/releases/tag/v0.2.5

cc @kianenigma
2023-10-27 19:54:56 +11:00
yjh 86228fa45e add authorities_len for aura (#2040) 2023-10-26 21:55:49 +02:00
Bastian Köcher 42707bc98b sp-version: Improve the docs (#2027)
Co-authored-by: ordian <write@reusable.software>
2023-10-26 21:53:15 +02:00
Alin Dima 1b08bdd2dd cumulus: fix test runtimes panic (#2039)
the min slot duration should be 0 only if the `experimental` feature is
enabled. otherwise, the runtime will panic on a division by 0.
2023-10-26 22:00:51 +03:00
Dastan 0bcebac4fc Expose collection attributes from Inspect trait (#1914)
# Description

- What does this PR do?

While working with `pallet_nfts` through `nonfungibles_v2` traits
`Inspect, Mutate`, I found out that once you have set the collection
attribute with `<Nfts as Mutate>::set_collection_attribute()`, it's not
possible to read it with `<Nfts as Inspect>::collection_attribute()`
since they use different `namespace` values. When setting the attribute,
`AttributeNamespace::Pallet` is used, while
`AttributeNamespace::CollectionOwner` is used when reading.

more context:
https://github.com/freeverseio/laos/issues/7#issuecomment-1766137370

This PR makes `item` an optional parameter in
`Inspect::system_attribute()`, to be able to read collection attributes.

- Why are these changes needed?

To be able to read collection level attributes when reading attributes
of the collection. It will be possible to read collection attributes by
passing `None` for `item`

- How were these changes implemented and what do they affect?

`NftsApi` is also affected and `NftsApi::system_attribute()` now accepts
optional `item` parameter.

## Breaking change

Because of the change in the `NftsApi::system_attribute()` method's
`item` param, parachains who integrated the `NftsApi` need to update
their API code and frontend integrations accordingly. AssetHubs are
unaffected since the NftsApi wasn't released on those parachains yet.
2023-10-26 09:52:12 +00:00
Oliver Tale-Yazdi c86b633695 Sort the benchmarks before listing them (#2026)
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-25 14:46:49 +02:00
Xiliang Chen b7a8532db1 publish pallet-root-testing (#2017)
so we can use it in our tests
2023-10-25 10:59:40 +02:00
PG Herveou 3148063a6f Contracts: Add benchmarks to include files (#2022)
Project that includes pallet-contracts via crates.io will fail to run 
```bash
cargo check --features=runtime-benchmarks
```

without the currently not included benchmarks files
2023-10-25 09:47:54 +02:00
Liam Aharon ff3a3bca44 Small optimisation to --profile dev wasm builds (#1851)
`wasm-builder` was adjusted to default to building wasm blobs in
`release` mode even when cargo is in `debug` because `debug` wasm is too
slow.

A side effect of this was `.compact` and `.compact.compressed` getting
built when the dev is running build in `debug`, adding ~5s to the build
time of every wasm runtime.

I think it's reasonable to assume if the dev is running `debug` build
they want to optimise speed and do not care about the size of the wasm
binary. Compacting a blob has negligible impact on its actual
performance.

In this PR, I adjusted the behavior of the wasm builder so it does not
produce `.compact` or `.compact.compressed` wasm when the user is
running in `debug`. The builder will continue to produce the bloaty wasm
in release mode unless it is overriden with an env var.

As suggested by @koute in review, also refactored the
`maybe_compact_wasm_and_copy_blobs` into multiple funuctions, and
renamed things to better support RISC-V in the future.

---

There is no `T-runtime` label so @KiChjang told me to put `T1-FRAME` :)

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-10-25 11:02:13 +11:00
Bastian Köcher 2e11ae89dc basic-authorship: Improve time recording and logging (#2010) 2023-10-25 00:30:48 +02:00
drskalman fbd5777118 Application Crypto and BEEFY Support for paired (ECDSA,BLS) crypto (#1815)
Next step in process of making BEEFY being able to generate both ECDSA
and BLS signature after #1705. It allows BEEFY to use a pair of ECDSA
and BLS key as a AuthorityId.

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
2023-10-24 20:37:34 +02:00
Oliver Tale-Yazdi 4a44356773 Improve features dev-ex (#1831)
Adds a config file that allows to run `zepter` without any arguments in
the workspace to address all issues.
A secondary workflow for the CI is provided as `zepter run check`. Both
the formatting and linting are now in one check for efficiancy.

The latest version also detects some more things that `featalign` was
already showing.

Error message [in the
CI](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3916205)
now looks like this:
```pre
...
crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml)
  feature 'std'
    must propagate to:
      parity-scale-codec
Found 55 issues (run with --fix to fix).
Error: Command 'lint propagate-feature' failed with exit code 1

Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration.
It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter`.
Otherwise please ask directly in the Merge Request, GitHub Discussions or on Matrix Chat, thank you.

For more information, see:
  - https://github.com/paritytech/polkadot-sdk/issues/1831
  - https://github.com/ggwpez/zepter
```

TODO:
- [x] Check that CI fails correctly

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-24 17:59:38 +02:00
Marcin S e39253c022 PVF: Add worker check during tests and benches (#1771) 2023-10-24 16:22:15 +02:00
georgepisaltu 9185195185 Make IdentityInfo generic in pallet-identity (#1661)
Fixes #179 

# Description

This PR makes the structure containing identity information used in
`pallet-identity` generic through the pallet `Config`. Additionally, the
old structure is now available in a separate module called `simple`
(pending rename) and is compatible with the new interface.

Another change in this PR is that while the `additional` field in
`IdentityInfo` stays for backwards compatibility reasons, the associated
costs are stil present in the pallet through the `additional` function
in the `IdentityInformationProvider` interface. This function is marked
as deprecated as it is only a temporary solution to the backwards
compatibility problem we had. In short, we could have removed the
additional fields in the struct and done a migration, but we chose to
wait and do it off-chain through the genesis of the system parachain.
After we move the identity pallet to the parachain, additional fields
will be migrated into the existing fields and the `additional` key-value
store will be removed. Until that happens, this interface will provide
the necessary information to properly account for the associated costs.

Additionally, this PR fixes an unrelated issue; the `IdentityField` enum
used to represent the fields as bitflags couldn't store more than 8
fields, even though it was marked as `#[repr(u64)]`. This was because of
the `derive` implementation of `TypeInfo`, which assumed `u8` semantics.
The custom implementation of this trait in
https://github.com/paritytech/polkadot-sdk/commit/0105cc0396b7a53d0b290f48b1225847f6d17321
fixes the issue.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
2023-10-24 13:47:11 +02:00
Kian Paimani 35eb133baa Ensure correct variant count in Runtime[Hold/Freeze]Reason (#1900)
closes https://github.com/paritytech/polkadot-sdk/issues/1882

## Breaking Changes

This PR introduces a new item to `pallet_balances::Config`:

```diff
trait Config {
++    type RuntimeFreezeReasons;
}
```

This value is only used to check it against `type MaxFreeze`. A similar
check has been added for `MaxHolds` against `RuntimeHoldReasons`, which
is already given to `pallet_balances`.

In all contexts, you should pass the real `RuntimeFreezeReasons`
generated by `construct_runtime` to `type RuntimeFreezeReasons`. Passing
`()` would also work, but it would imply that the runtime uses no
freezes at all.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-24 12:01:04 +02:00
Bastian Köcher a5a2432d22 CheckWeight: Add more logging (#1996)
This adds more logging to `CheckWeight` to get a better understanding
why a transaction exhausts resources.
2023-10-24 11:55:46 +02:00
Brian Anderson 39c04fdd96 Update wasm-opt to 0.116 (#1995)
Just keeping wasm-opt up to date.

I don't see anything in the [binaryen
changelog](https://github.com/WebAssembly/binaryen/blob/main/CHANGELOG.md)
that should affect substrate.

This release includes dwarf passes that were accidentally omitted from
previous versions of the wasm-opt crate. I suspect this will not affect
substrate as their omission hasn't been noticed until recently.
2023-10-24 12:52:30 +09:00
Muharem d0fd266003 Resolve Credit to Account impls of OnUnbalanced trait (#1876)
Implements the `OnUnbalanced` trait to resolve received credits to the
specified account. Credits that fail to resolve are dropped.

### Motivation 
Throughout the codebase, several types implement the trait with the same
behavior. While some currently utilize older currency trait, a migration
to the new fungible/s is anticipated for all. Examples:
[1](https://github.com/paritytech/polkadot-sdk/blob/1b34571c0c423115813034783ddf524aac257bf5/cumulus/parachains/common/src/impls.rs#L37),
[2](https://github.com/paritytech/polkadot-sdk/blob/1b34571c0c423115813034783ddf524aac257bf5/polkadot/runtime/common/src/impls.rs#L36),
[3](https://github.com/paritytech/polkadot-sdk/blob/1b34571c0c423115813034783ddf524aac257bf5/substrate/bin/node/runtime/src/impls.rs#L40),
[4](https://github.com/paritytech/polkadot-sdk/blob/1b34571c0c423115813034783ddf524aac257bf5/substrate/bin/node/runtime/src/lib.rs#L1969),
[5](https://github.com/paritytech/polkadot-sdk/blob/1b34571c0c423115813034783ddf524aac257bf5/substrate/frame/broker/src/mock.rs#L198),
[6](https://github.com/paritytech/polkadot-sdk/blob/1b34571c0c423115813034783ddf524aac257bf5/substrate/frame/society/src/lib.rs#L2031),
[7](https://github.com/paritytech/polkadot-sdk/blob/1b34571c0c423115813034783ddf524aac257bf5/substrate/frame/treasury/src/lib.rs#L1118)
2023-10-23 19:14:52 +02:00
aj3n 38c3c62588 wasm-builder: manually set CARGO_TARGET_DIR (#1951)
✄
-----------------------------------------------------------------------------

Thank you for your Pull Request! 🙏 Please make sure it follows the
contribution guidelines outlined in
[this
document](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md)
and fill
out the sections below. Once you're ready to submit your PR for review,
please
delete this section and leave only the text under the "Description"
heading.

# Description

*Please include a summary of the changes and the related issue. Please
also include relevant motivation and context,
including:*

- What does this PR do?

make 'substrate-wasm-builder' manually set 'CARGO_TARGET_DIR' to
'$project_dir/target' while building instead of unset
'CARGO_TARGET_DIR';

- Why are these changes needed?

If you using this in the `build.rs` with following content in your
`~/.cargo/config.toml':

    [build]
    target-dir = "target"

the build process will stuck because of dead lock -- two `cargo build`
on same target directory in the same time.
There is already an attempt to avoid such dead lock by unset the
`CARGO_TARGET_DIR`, but for users with config above in his build
enviroment (like me), this workaround won't work.

- How were these changes implemented and what do they affect?

Instead of unset 'CARGO_TARGET_DIR', we set 'CARGO_TARGET_DIR' to
'$project/target/', which is already assumed to be true by rest of the
code.

*Use [Github semantic

linking](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)
to address any open issues this PR relates to or closes.*

Fixes # (issue number, *if applicable*)

Closes # (issue number, *if applicable*)

# Checklist

- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

✄
-----------------------------------------------------------------------------

I have built my project with this fix, there's still some warnings with
`build.target-dir` set but the building process won't hang.
I haven't found related issue in this repo. But I did find one issue
[here](https://github.com/substrate-developer-hub/substrate-node-template/issues/116).
2023-10-23 11:54:06 +02:00
Bastian Köcher e2b21d00c7 sc-executor: Increase maximum instance count (#1856)
Changes the maximum instances count for `wasmtime` to `64`. It also
allows to only pass in maximum `32` for `--max-runtime-instances` as
`256` was way too big. With `64` instances in total and `32` that can be
configured in maximum, there should be enough space to accommodate for
extra instances that are may required to be allocated adhoc.

---------

Co-authored-by: Koute <koute@users.noreply.github.com>
2023-10-22 10:44:34 +02:00
Dmitry Markin 69c986f405 Revert "Check for parent of first ready block being on chain (#1812)" (#1950)
This reverts https://github.com/paritytech/polkadot-sdk/pull/1812 until
we know why it causes syncing issues reported in
https://github.com/subspace/subspace/issues/2122.
2023-10-20 13:03:08 +02:00
Bastian Köcher f3bf5c1acd xcm: Change TypeInfo::path to not include staging (#1948)
The `xcm` crate was renamed to `staging-xcm` to be able to publish it to
crates.io as someone as squatted `xcm`. The problem with this rename is
that the `TypeInfo` includes the crate name which ultimately lands in
the metadata. The metadata is consumed by downstream users like
`polkadot-js` or people building on top of `polkadot-js`. These people
are using the entire `path` to find the type in the type registry. Thus,
their code would break as the type path would now be [`staging_xcm`,
`VersionedXcm`] instead of [`xcm`, `VersionedXcm`]. This pull request
fixes this by renaming the path segment `staging_xcm` to `xcm`.

This requires: https://github.com/paritytech/scale-info/pull/197

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
2023-10-20 11:21:19 +02:00
cheme f4c4c0fe29 Switch trie cache random seed (#1935)
Use a more secure seed for hashsets of cache.
2023-10-20 12:13:19 +09:00
Oliver Tale-Yazdi 099ef8fe11 [FRAME] Message Queue use proper overweight limit (#1873)
Changes:
- Use a sensible limit for the overweight-cutoff of a single messages
instead of the full configured `ServiceWeight`.
- Add/Update tests

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-19 12:02:13 +02:00
Keith Yeung 3dece311be Introduce XcmFeesToAccount fee manager (#1234)
Combination of paritytech/polkadot#7005, its addon PR
paritytech/polkadot#7585 and its companion paritytech/cumulus#2433.

This PR introduces a new XcmFeesToAccount struct which implements the
`FeeManager` trait, and assigns this struct as the `FeeManager` in the
XCM config for all runtimes.

The struct simply deposits all fees handled by the XCM executor to a
specified account. In all runtimes, the specified account is configured
as the treasury account.

XCM __delivery__ fees are now being introduced (unless the root origin
is sending a message to a system parachain on behalf of the originating
chain).

# Note for reviewers

Most file changes are tests that had to be modified to account for the
new fees.
Main changes are in:
- cumulus/pallets/xcmp-queue/src/lib.rs <- To make it track the delivery
fees exponential factor
- polkadot/xcm/xcm-builder/src/fee_handling.rs <- Added. Has the
FeeManager implementation
- All runtime xcm_config files <- To add the FeeManager to the XCM
configuration

# Important note

After this change, instructions that create and send a new XCM (Query*,
Report*, ExportMessage, InitiateReserveWithdraw, InitiateTeleport,
DepositReserveAsset, TransferReserveAsset, LockAsset and RequestUnlock)
will require the corresponding origin account in the origin register to
pay for transport delivery fees, and the onward message will fail to be
sent if the origin account does not have the required amount. This
delivery fee is on top of what we already collect as tx fees in
pallet-xcm and XCM BuyExecution fees!

Wallet UIs that want to expose the new delivery fee can do so using the
formula:

```
delivery_fee_factor * (base_fee + encoded_msg_len * per_byte_fee)
```

where the delivery fee factor can be obtained from the corresponding
pallet based on which transport you are using (UMP, HRMP or bridges),
the base fee is a constant, the encoded message length from the message
itself and the per byte fee is the same as the configured per byte fee
for txs (i.e. `TransactionByteFee`).

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Giles Cope <gilescope@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2023-10-18 17:22:25 +02:00
Alex Bean 3aaf62add4 Trading trait and deal with metadata in Mutate trait for nonfungibles_v2 (#1561)
I have added some Traits that are missing and are useful for dealing
with non-fungible tokens on other pallets and their implementations for
NFTs pallet.

- In the Mutate trait, added methods for dealing with the metadata:
`set_metadata`, `set_collection_metadata`, `clear_metadata` and
`clear_collection_metadata`.
The motivation of adding this methods coming from a StackExchange
question asking for it: [Setting metadata of an item of the Nfts pallet
in a custom
pallet](https://substrate.stackexchange.com/questions/9974/setting-metadata-of-an-item-of-the-nfts-pallet-in-a-custom-pallet)

- A Trait for trading non-fungible items. The methods in that Trait are
`buy_item`, `set_price` and `item_price`
An example of where this Trait can be useful is a pallet that deals with
[NFT
Royalties](https://forum.polkadot.network/t/nfts-royalty-pallet/3766)
and needs to perform this actions.

---------

Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com>
2023-10-18 13:33:29 +03:00
Oliver Tale-Yazdi 58b7927284 Fix para-scheduler migration on Rococo (#1921)
Closes https://github.com/paritytech/polkadot-sdk/issues/1916

Changes:
- Trivially wrap the migration into a version migration to enforce
idempotency.
- Opinionated logging nits

@liamaharon maybe we can add a check to the `try-runtime-cli` that
migrations are idempotent? It should be possible to check that the
storage root is identical after executing a second time (and that it
does not panic like it did here 😆).

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
2023-10-18 00:19:34 +02:00
Squirrel d9e266f432 nit: use traits::tokens::fungible => use traits::fungible (#1753)
Slightly less verbose use of fungible(s).

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
2023-10-17 11:55:59 +02:00
Branislav Kontur 5cdd819ed2 [xcm] Small enhancements for NetworkExportTable and xcm-builder (#1848)
## Summary

This PR introduces several enhancements.

The current implementation of `NetworkExportTable` lacks remote location
filtering support beyond `NetworkId` lookup. To provide more control and
granularity, it's essential to allow configuration for bridging to
different consensus `NetworkId` while restricting access e.g. to
particular remote parachains.

Additionally, the `StartsWith` and `Equals` and
`StartsWithExplicitGlobalConsensus` helper functions, which are in
active use, are moved to the `xcm-builder` and `frame_support` modules
for better code organization.

Adds a new `LocationWithAssetFilters` filter to enable location-based
and asset-related filtering. This filter is useful for configuring the
`pallet_xcm` filter for
[XcmTeleportFilter](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L212)
and
[XcmReserveTransferFilter](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L216)
to restrict specific assets.

Furthermore, the `BridgeMessage` fields are not accessible outside of
`xcm-builder`, limiting the ability to create custom logic dependent on
it.

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
2023-10-17 11:11:03 +02:00
Kian Paimani e10de2e283 cleanup a few hidden imports in frame-support (#1770)
Just making a few hidden imports cleaner and hidden in docs.

---------

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2023-10-17 10:09:03 +02:00
Liam Aharon fcc1bb414b Allow Locks/Holds/Reserves/Freezes by default when using pallet_balances TestDefaultConfig (#1880)
Allow Locks/Holds/Reserves/Freezes by default when using
`pallet_balances` `TestDefaultConfig`.
2023-10-17 08:06:03 +11:00
Oliver Tale-Yazdi 9c1a2b38e5 Workspace maintenance (#1884)
Changes:
- Add missing crate to the workspace
- Remove versions from local dependency links

Maybe it is finally worth it to add this scrip to the CI to find these
things earlier:
[check-deps.py](https://github.com/ggwpez/substrate-scripts/blob/master/import-runtime-repos/check-deps.py).

@paritytech/ci what would be the best location for that check?  
It takes only a second to run, so maybe we can squeeze it into one of
the existing checks?
Otherwise creating a new GH workflow feels a bit wasteful... maybe i can
group it with https://github.com/paritytech/polkadot-sdk/pull/1831

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
2023-10-16 20:03:53 +02:00
Muharem c422e3f577 extract amount method for fungible/s Imbalance (#1847)
Introduces an `extract` amount method for `fungible/s` `Imbalance`.
2023-10-16 17:16:59 +02:00
Adrian Catangiu 646ecd0edb sc-consensus-beefy: fix initialization when state is unavailable (#1888)
Fix situation where BEEFY initial validator set could not be determined.

If state is unavailable at BEEFY genesis block to get initial validator
set, get the info from header digests. For this, we need to walk back
the chain starting from BEEFY genesis looking for the BEEFY digest
announcing the active validator set for that respective session.

This commit fixes a silly bug where walking back the chain was stopped
when reaching BEEFY genesis block, which is incorrect when BEEFY genesis
is not session boundary block. When BEEFY genesis is set to some random
block within a session, we need to walk back to the start of the session
to see the validator set announcement.

Added regression test for this fix.

Fixes https://github.com/paritytech/polkadot-sdk/issues/1885

Signed-off-by: Adrian Catangiu <adrian@parity.io>
2023-10-16 17:35:42 +03:00
Bastian Köcher 4e98bec3f1 sp-api: Improve error message for duplicate runtime apis (#1877)
Co-authored-by: command-bot <>
2023-10-16 14:41:44 +02:00
Bulat Saifullin 73ec161ebc Update the alerts to use a new metric substrate_unbounded_channel_size (#1568)
# Description

Follow up for https://github.com/paritytech/polkadot-sdk/pull/1489.
Closes #611 

Before we calculated the channel size during alert expression but in
#1489 a new metric was introduced that reports channel size.
## Changes:
1. updated alert rule to use new metric.
2023-10-16 13:15:50 +02:00
Davide Galassi 38ef04eb53 Arkworks Elliptic Curve utils overhaul (#1870)
- Removal of Arkworks unit tests. These tests were just testing the
arkworks upstream implementation which should be assumed correct. This
is not the place to test well known dependencies.
- Removal of some over-engineering. We just store the calls to Arkworks
in one file. Per-curve sources are not required.
- Docs formatting

---

I also took the opportunity to bump the `bandersnatch-vrfs` crate
revision internally providing some new shiny stuff.
2023-10-16 10:43:52 +02:00
Gonçalo Pestana 8ee4042c3b Refactor staking ledger (#1484)
This PR refactors the staking ledger logic to encapsulate all reads and
mutations of `Ledger`, `Bonded`, `Payee` and stake locks within the
`StakingLedger` struct implementation.

With these changes, all the reads and mutations to the `Ledger`, `Payee`
and `Bonded` storage map should be done through the methods exposed by
StakingLedger to ensure the data and lock consistency of the operations.
The new introduced methods that mutate and read Ledger are:

- `ledger.update()`: inserts/updates a staking ledger in storage;
updates staking locks accordingly (and ledger.bond(), which is synthatic
sugar for ledger.update())
- `ledger.kill()`: removes all Bonded and StakingLedger related data for
a given ledger; updates staking locks accordingly;
`StakingLedger::get(account)`: queries both the `Bonded` and `Ledger`
storages and returns a `Option<StakingLedger>`. The pallet impl exposes
fn ledger(account) as synthatic sugar for `StakingLedger::get(account)`.

Retrieving a ledger with `StakingLedger::get()` can be done by providing
either a stash or controller account. The input must be wrapped in a
`StakingAccount` variant (Stash or Controller) which is treated
accordingly. This simplifies the caller API but will eventually be
deprecated once we completely get rid of the controller account in
staking. However, this refactor will help with the work necessary when
completely removing the controller.

Other goals:

- No logical changes have been introduced in this PR;
- No breaking changes or updates in wallets required;
- No new storage items or need to perform storage migrations;
- Centralise the changes to bonds and ledger updates to simplify the
OnStakingUpdate updates to the target list (related to
https://github.com/paritytech/polkadot-sdk/issues/443)

Note: it would be great to prevent or at least raise a warning if
`Ledger<T>`, `Payee<T>` and `Bonded<T>` storage types are accessed
outside the `StakingLedger` implementation. This PR should not get
blocked by that feature, but there's a tracking issue here
https://github.com/paritytech/polkadot-sdk/issues/149

Related and step towards
https://github.com/paritytech/polkadot-sdk/issues/443
2023-10-15 22:50:07 +02:00
drskalman 1b34571c0c Paired-key Crypto Scheme (#1705)
BEEFY needs two cryptographic keys at the same time. Validators should
sign BEEFY payload using both ECDSA and BLS key. The network will gossip
a payload which contains a valid ECDSA key. The prover nodes aggregate
the BLS keys if aggregation fails to verifies the validator which
provided a valid ECDSA signature but an invalid BLS signature is subject
to slashing.

As such BEEFY session should be initiated with both key. Currently there
is no straight forward way of doing so, beside having a session with
RuntimeApp corresponding to a crypto scheme contains both keys.

This pull request implement a generic paired_crypto scheme as well as
implementing it for (ECDSA, BLS) pair.

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
2023-10-15 11:42:40 +02:00
S E R A Y A c9b51cd49c add link to rfc-0001 in broker README (#1862)
# Description
- What does this PR do?
  - link added
- Why are these changes needed?
  - improve docs
- How were these changes implemented and what do they affect?
  - only concerns docs
2023-10-15 10:00:58 +02:00
Julian Eager 9f7656df15 Discard Executor (#1855)
closes #622 

Pros:
* simpler interface, just functions:
`create_runtime_from_artifact_bytes()` and `execute_artifact()`

Cons:
* extra overhead of constructing executor semantics each time

I could make it a combination of
* `create_runtime_config(params)` (such that we could clone the
constructed semantics)
* `create_runtime(blob, config)`
* `execute_artifact(blob, config, params)`

Not sure if it's worth it though.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
2023-10-14 23:06:00 +02:00
Juan 7c87d61f5a Macros to use path instead of ident (#1474) 2023-10-14 08:26:19 +02:00
0xmovses 24840290af Refactor alliance benchmarks to v2 (#1868)
- This PR refactors `alliance/src/benchmarkings.rs` to use benchmarking
v2. These changes are needed to improve the readability and
maintainability of the benchmarking code.

- No known issue to backlink.

## Local Testing 
1. `cargo build --features runtime-benchmarks` 
2. `cargo run --locked --release -p node-cli --bin substrate-node
--features runtime-benchmarks -- benchmark pallet --execution wasm
--wasm-execution compiled --chain dev --pallet "*" --extrinsic "*"
--steps 2 --repeat 1`
2023-10-13 21:54:45 +01:00
Adrian Catangiu 82bfe28424 frame: use derive-impl for beefy and mmr pallets (#1867)
Part of #171
2023-10-13 11:37:48 +03:00
Adrian Catangiu 832060000b sc-consensus-beefy: improve gossip logic (#1852)
- Remove cached messages used for deduplication in `GossipValidator`
since they're already deduplicated in upper layer `NetworkGossip`.
- Add cache for "justified rounds" to quickly discard any further (even
if potentially different) justifications at the gossip level, once a
valid one (for a respective round) is submitted to the worker.
- Add short-circuit in worker `finalize()` method to not attempt to
finalize same block multiple times (for example when we get
justifications for same block from multiple components like
block-import, gossip or on-demand).
- Change a test which had A LOT of latency in syncing blocks for some
weird reason and would only run after ~150seconds. It now runs
instantly.

Fixes https://github.com/paritytech/polkadot-sdk/issues/1728
2023-10-13 09:12:30 +02:00