We currently use a bit of a hack in `.cargo/config` to make sure that
clippy isn't too annoying by specifying the list of lints.
There is now a stable way to define lints for a workspace. The only down
side is that every crate seems to have to opt into this so there's a
*few* files modified in this PR.
Dependencies:
- [x] PR that upgrades CI to use rust 1.74 is merged.
---------
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Our executor currently only supports the WASM MVP feature set, however
nowadays when compiling WASM the Rust compiler has more features enabled
by default.
We do set the `-C target-cpu=mvp` flag to make sure that *our* code gets
compiled in a way that is compatible with our executor, however this
doesn't affect Rust's standard library crates (`std`, `core` and
`alloc`) which are by default precompiled and still can make use of
these extra features.
So in this PR we force the compiler to also compile the standard library
crates for us to make sure that they also only use the MVP features.
I've added the `WASM_BUILD_STD` environment variable which can be used
to disable this behavior if set to `0`.
Unfortunately this *will* slow down the compile times when building
runtimes, but there isn't much that we can do about that.
Fixes https://github.com/paritytech/polkadot-sdk/issues/1755
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
Optimizes the `rerun-if-changed` logic by ignoring `dev-dependencies`
and also not outputting paths. Because outputting paths could lead to
include unwanted crates in the rerun checks.
`wasm-builder` was adjusted to default to building wasm blobs in
`release` mode even when cargo is in `debug` because `debug` wasm is too
slow.
A side effect of this was `.compact` and `.compact.compressed` getting
built when the dev is running build in `debug`, adding ~5s to the build
time of every wasm runtime.
I think it's reasonable to assume if the dev is running `debug` build
they want to optimise speed and do not care about the size of the wasm
binary. Compacting a blob has negligible impact on its actual
performance.
In this PR, I adjusted the behavior of the wasm builder so it does not
produce `.compact` or `.compact.compressed` wasm when the user is
running in `debug`. The builder will continue to produce the bloaty wasm
in release mode unless it is overriden with an env var.
As suggested by @koute in review, also refactored the
`maybe_compact_wasm_and_copy_blobs` into multiple funuctions, and
renamed things to better support RISC-V in the future.
---
There is no `T-runtime` label so @KiChjang told me to put `T1-FRAME` :)
---------
Co-authored-by: Koute <koute@users.noreply.github.com>
Just keeping wasm-opt up to date.
I don't see anything in the [binaryen
changelog](https://github.com/WebAssembly/binaryen/blob/main/CHANGELOG.md)
that should affect substrate.
This release includes dwarf passes that were accidentally omitted from
previous versions of the wasm-opt crate. I suspect this will not affect
substrate as their omission hasn't been noticed until recently.
✄
-----------------------------------------------------------------------------
Thank you for your Pull Request! 🙏 Please make sure it follows the
contribution guidelines outlined in
[this
document](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md)
and fill
out the sections below. Once you're ready to submit your PR for review,
please
delete this section and leave only the text under the "Description"
heading.
# Description
*Please include a summary of the changes and the related issue. Please
also include relevant motivation and context,
including:*
- What does this PR do?
make 'substrate-wasm-builder' manually set 'CARGO_TARGET_DIR' to
'$project_dir/target' while building instead of unset
'CARGO_TARGET_DIR';
- Why are these changes needed?
If you using this in the `build.rs` with following content in your
`~/.cargo/config.toml':
[build]
target-dir = "target"
the build process will stuck because of dead lock -- two `cargo build`
on same target directory in the same time.
There is already an attempt to avoid such dead lock by unset the
`CARGO_TARGET_DIR`, but for users with config above in his build
enviroment (like me), this workaround won't work.
- How were these changes implemented and what do they affect?
Instead of unset 'CARGO_TARGET_DIR', we set 'CARGO_TARGET_DIR' to
'$project/target/', which is already assumed to be true by rest of the
code.
*Use [Github semantic
linking](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)
to address any open issues this PR relates to or closes.*
Fixes # (issue number, *if applicable*)
Closes # (issue number, *if applicable*)
# Checklist
- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
required)
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)
You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!
✄
-----------------------------------------------------------------------------
I have built my project with this fix, there's still some warnings with
`build.target-dir` set but the building process won't hang.
I haven't found related issue in this repo. But I did find one issue
[here](https://github.com/substrate-developer-hub/substrate-node-template/issues/116).
* wasm-builder: Make `hash` and `date` optional
Apparently there are installations where the `hash` and `date` is optional.
Closes: https://github.com/paritytech/substrate/issues/14335
* ".git/.scripts/commands/fmt/fmt.sh"
---------
Co-authored-by: command-bot <>
* frame-benchmarking-cli: Remove native dispatch requirement
No need for this, we can just use the `WasmExecutor` directly.
* Fixes
* Pass benchmarking host functions
* Ensure we can pass custom host functions
* wasm-builder: Enforce `runtime_version` wasm section
This pr changes the `wasm-builder` to enforce the `runtime_version` wasm section. This wasm section
is being created by the `sp_version::runtime_version` attribute macro. This attribute macro now
exists since quite some time and `runtime_version` also is the only way for parachains to support
reading the `RuntimeVersion` from the runtime.
\# Disabling the check
By default the `WasmBuilder` will now check for this wasm section and if not found, exit with an
error. However, there are situations where you may want to disable this check (like for tests). In
this case there exists the `disable_runtime_version_section_check` function.
```
WasmBuilder::new()
...
...
...
.disable_runtime_version_section_check()
.build()
```
By using this method you get back the old behavior.
* Review comment
* Fix
* Fix issue with `enum-as-inner`
* Support stable rust for compiling the runtime
This pull request brings support for compiling the runtime with stable Rust. This requires at least
rust 1.68.0 to work on stable. The code is written in a way that it is backwards compatible and
should automatically work when someone compiles with 1.68.0+ stable.
* We always support nightlies!
* 🤦
* Sort by version
* Review feedback
* Review feedback
* Fix version parsing
* Apply suggestions from code review
Co-authored-by: Koute <koute@users.noreply.github.com>
---------
Co-authored-by: Koute <koute@users.noreply.github.com>
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* Bump `wasmtime` to 4.0.0 (and a few other deps)
* Use `Error::msg` instead of `anyhow!`
* Bump `wasmtime` to 5.0.0
* Update `Cargo.lock`
* Add `wasmtime` feature to `sp-wasm-interface` dependency
* Fixup some wrong dependencies
Dev dependencies should not appear in the feature list. If features are required, they should be
directly enabled for the `dev-dependency`.
* More fixups
* Fix fix
* Remove deprecated feature
* Make all work properly and nice!!
* FMT
* Fix formatting
* implement crate publishing from CI
* fix indentation
* use resource_group for job exclusivity
ensure that at most one instance of the publish-crates job is running at any given time to prevent race conditions
* correct publish = false
* Remove YAML anchors as GitLab's `extends:` doesn't need it
* Temporarily force cache upload for the new jobs
* Revert `RUSTY_CACHIER_FORCE_UPLOAD`
* pin libp2p-tcp=0.37.0 for sc-telemetry
* Revert "pin libp2p-tcp=0.37.0 for sc-telemetry"
This reverts commit 29146bfad6c31e8cf0e2f17ad92a71bb81a373af.
* always collect generated crates
* increase timeout for publish-crates-template
* Force upload the new job cache again
* Revert "Force upload the new job cache again"
This reverts commit 5a5feee1b2c51fdef768b25a76be4c3949ec1c99.
* reformat
* improve timeout explanation
* s/usual/average
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
* Use wasm-opt on runtime
* Optimize for size
* Simplify fn compact_wasm_file
* Run a lighter pass for non production builds
* Disable optimizations and keep name section
* Update wasm-opt
* Remove dward sections
* Update wasm-opt
* Update wasm-opt
Recently we added the wasm binaries to the `rerun-if-changed` list. The problem with that is that
they have a later mtime than the `invoked.timestamp` file and this file's mtime is used to determine
if the `build.rs` script needs to be re-run. The solution to this is that we copy the mtime of this
`invoked.timestamp` file and add it to the wasm binaries. Then cargo/rustc doesn't constantly wants
to rerun the `build.rs` script.
* wasm-builder: Support latest nightly
With latest nightly, aka rust version 1.60+ namespaced features are added. This changes the handling
of optional dependencies. We currently have features that enable optional dependencies when `std` is
enabled. This was before no problem, but now the wasm-builder detects them as enabled. To support
the transition period until 1.60 is released as stable, this pr adds an heuristic to not enable these
optional crates in the wasm build when they are enabled in the `std` feature. This heuristic fails
when someones enables these optional dependencies from the outside as well as via the `std` feature,
however we hope that no one is doing this at the moment. When namespaced features are enabled, these
dependencies needs to be enabled using `dep:dependency-name` to solve this properly.
https://doc.rust-lang.org/cargo/reference/unstable.html#namespaced-features
* Remove accidentally added features
This is required for projects like Cumulus that have dependencies on `westend-runtime`, but this
runtime is only added when the `runtime-benchmarks` feature is enabled. By having all features
"enabled" in `cargo metadata` we ensure that all crates can be found.
When building a wasm binary from a different repo inside a local workspace, we did not used the
correct `Cargo.toml` to find the correct patches and features. The solution to this is to just walk
up from the target directory until we find the workspace we are currently compiling. If this
heuristic isn't working, we print a warning and let the user set an env variable
`WASM_BUILD_WORKSPACE_HINT` to tell the `wasm-builder` where the actual workspace is.
* Intend to reactivate cargo-unleash check
It appears the bug it was deactivated for has been resolved a while ago. Trying to reactivate the checks.
* adding missing cargo.toml metadata for BEEFY crates
* fix wrong version reference
* matching up versions
* disable faulty cache
* switching more versions to prerelease
* Revert "disable faulty cache"
This reverts commit 411a12ae444a9695a8bfea4458a868438d870b06.
* bump minor of sc-allocator to fix already-published-issue
* fixup another pre-released dependency problem
* temp switch to latest unleash
* fixing dependency version and features
* prometheus endpoint has also been changed
* fixing proposer metrics versioning
* fixing hex feature for beefy
* fix generate-bags feature selection
* fixup Cargo.lock
* upgrade prometheus dependencies
* missed one
* switch to latest release
Niklas Adolfsson