* implement crate publishing from CI
* fix indentation
* use resource_group for job exclusivity
ensure that at most one instance of the publish-crates job is running at any given time to prevent race conditions
* correct publish = false
* Remove YAML anchors as GitLab's `extends:` doesn't need it
* Temporarily force cache upload for the new jobs
* Revert `RUSTY_CACHIER_FORCE_UPLOAD`
* pin libp2p-tcp=0.37.0 for sc-telemetry
* Revert "pin libp2p-tcp=0.37.0 for sc-telemetry"
This reverts commit 29146bfad6c31e8cf0e2f17ad92a71bb81a373af.
* always collect generated crates
* increase timeout for publish-crates-template
* Force upload the new job cache again
* Revert "Force upload the new job cache again"
This reverts commit 5a5feee1b2c51fdef768b25a76be4c3949ec1c99.
* reformat
* improve timeout explanation
* s/usual/average
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
* Use wasm-opt on runtime
* Optimize for size
* Simplify fn compact_wasm_file
* Run a lighter pass for non production builds
* Disable optimizations and keep name section
* Update wasm-opt
* Remove dward sections
* Update wasm-opt
* Update wasm-opt
Recently we added the wasm binaries to the `rerun-if-changed` list. The problem with that is that
they have a later mtime than the `invoked.timestamp` file and this file's mtime is used to determine
if the `build.rs` script needs to be re-run. The solution to this is that we copy the mtime of this
`invoked.timestamp` file and add it to the wasm binaries. Then cargo/rustc doesn't constantly wants
to rerun the `build.rs` script.
* wasm-builder: Support latest nightly
With latest nightly, aka rust version 1.60+ namespaced features are added. This changes the handling
of optional dependencies. We currently have features that enable optional dependencies when `std` is
enabled. This was before no problem, but now the wasm-builder detects them as enabled. To support
the transition period until 1.60 is released as stable, this pr adds an heuristic to not enable these
optional crates in the wasm build when they are enabled in the `std` feature. This heuristic fails
when someones enables these optional dependencies from the outside as well as via the `std` feature,
however we hope that no one is doing this at the moment. When namespaced features are enabled, these
dependencies needs to be enabled using `dep:dependency-name` to solve this properly.
https://doc.rust-lang.org/cargo/reference/unstable.html#namespaced-features
* Remove accidentally added features
This is required for projects like Cumulus that have dependencies on `westend-runtime`, but this
runtime is only added when the `runtime-benchmarks` feature is enabled. By having all features
"enabled" in `cargo metadata` we ensure that all crates can be found.
When building a wasm binary from a different repo inside a local workspace, we did not used the
correct `Cargo.toml` to find the correct patches and features. The solution to this is to just walk
up from the target directory until we find the workspace we are currently compiling. If this
heuristic isn't working, we print a warning and let the user set an env variable
`WASM_BUILD_WORKSPACE_HINT` to tell the `wasm-builder` where the actual workspace is.
* Prepare UI tests for rust 1.54
* Delete wrong_page.stderr
* CI: run with a staging CI image
* Revert "CI: run with a staging CI image"
This reverts commit 66f5b00d14b50fd9d8fbf773f7e884f380697591.
* CI: debug, again
* LOG_TARGET is only used in std
* Remove unnecessary unsafe
* Fixes
* Use correct rustc locally
* FMT
* Compile with benchmarking
* Review feedback
* Some ui tests
* I know...
* Fix wasm tests
Co-authored-by: Denis P <denis.pisarev@parity.io>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* validation extension in sp_io
* need paths
* arc impl
* missing host function in executor
* io to pkdot
* decode function.
* encode primitive.
* trailing tab
* multiple patch
* fix child trie logic
* restore master versionning
* bench compact proof size
* trie-db 22.3 is needed
* line width
* split line
* fixes for bench (additional root may not be needed as original issue was
with empty proof).
* revert compact from block size calculation.
* New error type for compression.
* Adding test (incomplete (failing)).
Also lacking real proof checking (no good primitives in sp-trie crate).
* There is currently no proof recording utility in sp_trie, removing
test.
* small test of child root in proof without a child proof.
* remove empty test.
* remove non compact proof size
* Missing revert.
* proof method to encode decode.
* Init `RuntimeLogger` automatically for each runtime api call
This pr change the runtime api in such a way to always and automatically
enable the `RuntimeLogger`. This enables the user to use `log` or
`tracing` from inside the runtime to create log messages. As logging
introduces some extra code and especially increases the size of the wasm
blob. It is advised to disable all logging completely with
`sp-api/disable-logging` when doing the wasm builds for the on-chain
wasm runtime.
Besides these changes, the pr also brings most of the logging found in
frame to the same format "runtime::*".
* Update frame/im-online/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update test-utils/runtime/Cargo.toml
* Fix test
* Don't use tracing in the runtime, as we don't support it :D
* Fixes
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Build every wasm crate in its own project with wasm-builder
Building all wasm crates in one workspace was a nice idea, however it
just introduced problems:
1. We needed to prune old members, but this didn't worked for old git
deps.
2. We locked the whole wasm workspace while building one crate. This
could lead to infinitely locking the workspace on a crash.
Now we just build every crate in its own project, this means we will
build the dependencies multiple times. While building the dependencies
multiple times, we still decrease the build time by around 30 seconds
for Polkadot and Substrate because of the new parallelism ;)
* Remove the requirement on wasm-builder-runner
This removes the requirement on wasm-builder-runner by using the new
`build_dep` feature of cargo. We use nightly anyway and that enables us
to use this feature. This solves the problem of not mixing
build/proc-macro deps with normal deps. By doing this we get rid off
this complicated project structure and can depend directly on
`wasm-builder`. This also removes all the code from wasm-builder-runner
and mentions that it is deprecated.
* Copy the `Cargo.lock` to the correct folder
* Remove wasm-builder-runner
* Update docs
* Fix deterministic check
Modified-by: Bastian Köcher <git@kchr.de>
* Try to make the ui test happy
* Switch to `SKIP_WASM_BUILD`
* Rename `SKIP_WASM_BINARY` to the correct name...
* Update utils/wasm-builder/src/builder.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update utils/wasm-builder/src/builder.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Make `wasm-builder` print the rustc version
This makes `wasm-builder` print the rustc version that is being used to
compile the project. This is rather useful, because people can check
faster if the used rustc version is maybe known for being broken with
Substrate.
* Apply suggestions from code review
* Add some comments
* Fixes bug in wasm-builder with cargo publish
There was a bug in wasm-builder which resulted in generating a
`Cargo.lock` in the project directory because of running `cargo
metadata`. This resulted in commands like `cargo publish` to fail (if
there was no `Cargo.lock` before building), because it checks that the
project directory isn't modified.
* Update utils/wasm-builder/src/wasm_project.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
This improves the error message of wasm-builder when the wasm toolchain
isn't installed. Currently we print that the wasm toolchain is not
installed, but the actual problem is that there is a bug in the
packaging in rust. This will now be much easier to debug, by printing
the full error message of the compiler.
* Enable the `runtime-wasm` for wasm builds
This enables the `runtime-wasm` feature for wasm builds. The feature is
not mandatory and will only be activated if it exists in the
`Cargo.toml`.
* Fix compilation
* Update docs
* Uprgade version
* Apply suggestions from code review
There was a bug which related in required files not being tracked of
being modified. This pr fixes this bug by making sure we ignore version
requirements for path dependencies and git dependencies. This also
ensures that we only track `.rs` or `.toml` files. Another improvement
is that we only include paths which don't contain a `Cargo.toml` if this
`Cargo.toml` does not belongs to the package being processed. This
prevents that sub-crates are added to the tracked files, while not being
part of the dependencies.
Up to now `wasm-builder` iterated over all packages that were not part
of the wasm workspace and added the files to the `rerun-if-changed`
list. However, this included to many files and resulted in needless
re-compilations in `Cumulus` for example.
Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.
* Make debug builds more usable
This pr makes debug builds more usable in terms of `cargo run -- --dev`.
1. `--dev` activates `--execution native`, iff `--execution` is not
given or no sub `--execution-*` is given.
2. It was probably a mistake to compile WASM in debug for a debug build.
So, we now build the WASM binary always as `release` (if not requested
differently by the user). So, we trade compilation time for a better
debug experience.
* Make sure we only overwrite default values
* Make it work
* Apply suggestion
* Make wasm-builder remove invalid members
Wasm-builder now removes members that point to packages that do not
exist anymore or that were renamed. Up to now, the build would fail and
required manual fixing.
* Fixes build
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
João Paulo Silva de Souza