Commit Graph

31 Commits

Author SHA1 Message Date
zjb0807 617e2cc75e Upgrade wasm builder (#10226)
* add TypeInfo for DispatchTime

* upgrade wasm-builder to Rust 2021

* remove resolver

* revert resolver in virtual workspace
2021-11-10 08:03:52 +00:00
Bastian Köcher 62187b5916 Prepare UI tests for rust 1.55 (#9637)
* Prepare UI tests for rust 1.54

* Delete wrong_page.stderr

* CI: run with a staging CI image

* Revert "CI: run with a staging CI image"

This reverts commit 66f5b00d14b50fd9d8fbf773f7e884f380697591.

* CI: debug, again

* LOG_TARGET is only used in std

* Remove unnecessary unsafe

* Fixes

* Use correct rustc locally

* FMT

* Compile with benchmarking

* Review feedback

* Some ui tests

* I know...

* Fix wasm tests

Co-authored-by: Denis P <denis.pisarev@parity.io>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2021-09-24 10:42:08 +00:00
Kian Paimani abd08e29ce Rewrap all comments to 100 line width (#9490)
* reformat everything again

* manual formatting

* last manual fix

* Fix build
2021-08-11 14:56:55 +00:00
Squirrel aafe64315a Remove extra commas made redundent after rustfmt (#9404)
* Remove extra commas made redundent after rustfmt
2021-07-22 11:06:17 +01:00
Bastian Köcher 7b56ab15b4 Run cargo fmt on the whole code base (#9394)
* Run cargo fmt on the whole code base

* Second run

* Add CI check

* Fix compilation

* More unnecessary braces

* Handle weights

* Use --all

* Use correct attributes...

* Fix UI tests

* AHHHHHHHHH

* 🤦

* Docs

* Fix compilation

* 🤷

* Please stop

* 🤦 x 2

* More

* make rustfmt.toml consistent with polkadot

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-07-21 14:32:32 +00:00
cheme f01c396677 Compact proof utilities in sp_trie. (#8574)
* validation extension in sp_io

* need paths

* arc impl

* missing host function in executor

* io to pkdot

* decode function.

* encode primitive.

* trailing tab

* multiple patch

* fix child trie logic

* restore master versionning

* bench compact proof size

* trie-db 22.3 is needed

* line width

* split line

* fixes for bench (additional root may not be needed as original issue was
with empty proof).

* revert compact from block size calculation.

* New error type for compression.

* Adding test (incomplete (failing)).
Also lacking real proof checking (no good primitives in sp-trie crate).

* There is currently no proof recording utility in sp_trie, removing
test.

* small test of child root in proof without a child proof.

* remove empty test.

* remove non compact proof size

* Missing revert.

* proof method to encode decode.
2021-06-07 09:06:38 +00:00
Michael Müller 536cee37f1 Upgrade cargo_metadata to 0.13.1 (#8670)
* Upgrade `cargo_metadata` to 0.13.1

* Update utils/wasm-builder/src/wasm_project.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-04-27 07:17:38 +00:00
Chris D'Costa 0cbd3b4f46 #8597 Update features resolver wasm build (#8598) 2021-04-12 18:43:22 +00:00
Robert Habermeier a600e278ed Support code blobs compressed with zstd (#8549)
* begin maybe-compressed-blob

* fix build

* implement blob compression / decompression

* add some tests

* decode -> decompress

* decompress code if compressed

* make API of compresseed blob crate take limit as parameter

* use new API in sc-executro

* wasm-builder: compress wasm

* fix typo

* simplify

* address review

* fix wasm_project.rs

* Update primitives/maybe-compressed-blob/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-04-07 20:44:45 +00:00
Bastian Köcher 68390d4085 Init RuntimeLogger automatically for each runtime api call (#8128)
* Init `RuntimeLogger` automatically for each runtime api call

This pr change the runtime api in such a way to always and automatically
enable the `RuntimeLogger`. This enables the user to use `log` or
`tracing` from inside the runtime to create log messages. As logging
introduces some extra code and especially increases the size of the wasm
blob. It is advised to disable all logging completely with
`sp-api/disable-logging` when doing the wasm builds for the on-chain
wasm runtime.

Besides these changes, the pr also brings most of the logging found in
frame to the same format "runtime::*".

* Update frame/im-online/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update test-utils/runtime/Cargo.toml

* Fix test

* Don't use tracing in the runtime, as we don't support it :D

* Fixes

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-03-01 15:29:17 +01:00
Bastian Köcher e3e651f72c Happy new year (#7814)
* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL
2021-01-04 09:03:13 +00:00
Bastian Köcher 923cb8eea1 Wasm-builder 3.0 (#7532)
* Build every wasm crate in its own project with wasm-builder

Building all wasm crates in one workspace was a nice idea, however it
just introduced problems:

1. We needed to prune old members, but this didn't worked for old git
deps.
2. We locked the whole wasm workspace while building one crate. This
could lead to infinitely locking the workspace on a crash.

Now we just build every crate in its own project, this means we will
build the dependencies multiple times. While building the dependencies
multiple times, we still decrease the build time by around 30 seconds
for Polkadot and Substrate because of the new parallelism ;)

* Remove the requirement on wasm-builder-runner

This removes the requirement on wasm-builder-runner by using the new
`build_dep` feature of cargo. We use nightly anyway and that enables us
to use this feature. This solves the problem of not mixing
build/proc-macro deps with normal deps. By doing this we get rid off
this complicated project structure and can depend directly on
`wasm-builder`. This also removes all the code from wasm-builder-runner
and mentions that it is deprecated.

* Copy the `Cargo.lock` to the correct folder

* Remove wasm-builder-runner

* Update docs

* Fix deterministic check

Modified-by: Bastian Köcher <git@kchr.de>

* Try to make the ui test happy

* Switch to `SKIP_WASM_BUILD`

* Rename `SKIP_WASM_BINARY` to the correct name...

* Update utils/wasm-builder/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update utils/wasm-builder/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-11-24 10:18:36 +01:00
Bastian Köcher 23382db1b4 Make wasm-builder print the rustc version (#7351)
* Make `wasm-builder` print the rustc version

This makes `wasm-builder` print the rustc version that is being used to
compile the project. This is rather useful, because people can check
faster if the used rustc version is maybe known for being broken with
Substrate.

* Apply suggestions from code review

* Add some comments
2020-10-19 14:45:09 +02:00
Bastian Köcher 1a01bca647 Fixes bug in wasm-builder with cargo publish (#7299)
* Fixes bug in wasm-builder with cargo publish

There was a bug in wasm-builder which resulted in generating a
`Cargo.lock` in the project directory because of running `cargo
metadata`. This resulted in commands like `cargo publish` to fail (if
there was no `Cargo.lock` before building), because it checks that the
project directory isn't modified.

* Update utils/wasm-builder/src/wasm_project.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-10-12 12:29:53 +02:00
Bastian Köcher 9dd5812eaf Improve error output of wasm-builder when wasm ins't installed (#7105)
This improves the error message of wasm-builder when the wasm toolchain
isn't installed. Currently we print that the wasm toolchain is not
installed, but the actual problem is that there is a bug in the
packaging in rust. This will now be much easier to debug, by printing
the full error message of the compiler.
2020-09-15 11:42:23 +02:00
cheme 186fa587cf Ignore wasm_gc for debug build. (#6962)
* Ignore gc for debug build.

* alternate implementation

* Update utils/wasm-builder/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fix

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-09-08 09:27:10 +02:00
pscott 046fda914a Improve overall performance (#6699)
* Improve overall performance

* Clean up code

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-21 14:46:49 +02:00
Bastian Köcher c9cc46de3b Wasm-builder runtime-wasm feature (#6177)
* Enable the `runtime-wasm` for wasm builds

This enables the `runtime-wasm` feature for wasm builds. The feature is
not mandatory and will only be activated if it exists in the
`Cargo.toml`.

* Fix compilation

* Update docs

* Uprgade version

* Apply suggestions from code review
2020-05-28 21:06:25 +01:00
Bastian Köcher 1ea45a541e Make wasm-builder check before copy/write files if the content is the same (#6149)
* Make `wasm-builder` check before copy/write files if the content is the
same

* Update utils/wasm-builder/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-05-27 00:28:41 +02:00
Bastian Köcher 0f28f4bf49 Fixes wasm-builder rerun if changed logic (#6144)
There was a bug which related in required files not being tracked of
being modified. This pr fixes this bug by making sure we ignore version
requirements for path dependencies and git dependencies. This also
ensures that we only track `.rs` or `.toml` files. Another improvement
is that we only include paths which don't contain a `Cargo.toml` if this
`Cargo.toml` does not belongs to the package being processed. This
prevents that sub-crates are added to the tracked files, while not being
part of the dependencies.
2020-05-26 21:23:57 +02:00
Benjamin Kampmann be8c96adec Relax substrate licensing scheme (#5947) 2020-05-15 13:21:56 +02:00
Demi Obenour d34a0f4e22 Fix escaping of the wasm file path (#6018)
* Fix escaping of the wasm file path

* Add separate methods for escaped paths
2020-05-14 00:55:24 +02:00
Bastian Köcher 60b65695e2 Make wasm-builder select dependencies more sensitive (#5162)
Up to now `wasm-builder` iterated over all packages that were not part
of the wasm workspace and added the files to the `rerun-if-changed`
list. However, this included to many files and resulted in needless
re-compilations in `Cumulus` for example.
2020-03-10 20:52:36 +01:00
Benjamin Kampmann 3bd088d41c Support enabling features with wasm-builder (#5131)
This adds support for enabling features in the wasm build. The `default` and
`std` feature are ignored in the build.
2020-03-05 08:52:23 +01:00
Bastian Köcher 36770979bb Extend rust nightly detection in wasm-builder (#5021)
Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.
2020-02-25 15:50:45 +01:00
Shawn Tabrizi c7a14db792 Fix various spelling errors (#4940)
* meaningfull -> meaningful

* initialise -> initialize

* tokans -> tokens

* incentivise -> incentivize

* lenght -> length

* incentivisation -> incentivization

* doesnt't -> doesn't

* overwriten -> overwritten

* lifecycle -> life cycle

* lifecycle -> life cycle

* usefull -> useful

* noone -> no one

* spaming -> spamming

* defered -> deferred

* hieght -> height

* sumation -> summation

* ingore -> ignore

* registed -> registered

* Auxialary -> Auxiliary

* loggin -> logging

* independance -> independence

* trailling -> trailing

* responsability -> responsibility

* trunkated -> truncated

* Weither -> Whether

* informations -> information

* Runtume -> Runtime

* choosen -> chosen

* delcataion -> declaration

* Unchekced -> Unchecked

* defintion -> definition

* scrach -> scratch

* imput -> input

* transfered -> transferred

* endownment -> endowment

* Determinator -> Determiner

* relevent -> relevant

* emited -> emitted

* acocunt -> account

* proprotional -> proportional

* instantiaion -> instantiation

* commited -> committed

* tombstonedead -> tombstone

* uwnrap -> unwrap

* acount -> account

* specialised -> specialized

* existant -> existent

* requried -> required

* Anull -> Annul

* AUTHORITES -> AUTHORITIES

* underyling -> underlying

* recognisable -> recognizable

* Capitalise -> Capitalize

* reportfor -> report for

* hearbeat -> heartbeat

* onlineness -> being online

* creater -> creator

* Bytearray -> Byte array

* Despoit -> Deposit

* substratced -> subtracted

* Curent -> Current

* imbalanes -> imbalances

* countfown -> countdown

* inexisting -> inexistent

* additionaly -> additionally

* substracted -> subtracted

* auxilary -> auxiliary

* parital -> partial

* in't -> isn't

* compatability -> compatibility

* infomation -> information

* etected -> detected

* extrinsiscs -> extrinsics

* reprensentation -> representation

* coonfiguration -> configuration

* primtives -> primitives

* miscelanious -> miscellaneous

* VERISON -> VERSION

* endcoded -> encoded

* Genrates -> Generates

* miliseconds -> milliseconds

* occured -> occurred

* trully -> truely

* truely -> truly

* conjuction -> conjunction

* encouters -> encounters

* customised -> customized

* deterministicly -> deterministically

* finalisation -> finalization

* pluggable -> plugable

* wakeup -> wake-up

* interemdiate -> intermediate

* intepreting -> interpreting

* finalzied -> finalized

* throgh -> through

* extinsic -> extrinsic

* convient -> convenient

* allocater -> allocator

* propagateable -> propagatable

* succesfuly -> successfully

* finalising -> finalizing

* publically -> publicly

* phrasee -> phrase

* substration -> substractions

* substractions -> subtractions

* neccessarily -> necessarily

* Inlucde -> Include

* unefficient -> inefficient

* thay -> they

* funtion -> function

* datastructures -> data structures

* infromation -> information

* propagatable -> propagable

* ecountered -> encountered

* recognise -> recognize

* intergration -> integration

* lastet -> latest

* datatypes -> data types

* datatype -> data type

* Strongarming -> Strong Arming

* avaible -> available

* Commiting -> Committing

* Retreiving -> Retrieving

* shoud -> should

* canonicaliziation -> canonicalization

* comitted -> committed

* clonable -> cloneable

* Uknown -> Unknown

* reponse -> response

* arbitary -> arbitrary

* Capapbilities -> Capabilities

* responsbile -> responsible

* initialisation -> initialization

* cames -> came

* intemediate -> intermediate

* reqeust -> request

* intance -> instance

* explcitly -> explicitly

* neighor -> neighbor

* reolving -> resolving

* untill -> until

* Validte -> Validate

* deserailize -> deserialize

* literaly -> literally

* preceeding -> preceding

* abpve -> above

* chcecked -> checked

* numbet -> number

* Unknow -> Unknown

* halfs -> halves

* gossup -> gossip

* givent -> given

* immediatelly -> immediately

* slicable -> sliceable

* conensus -> consensus

* Mimicks -> Mimics

* acccept -> accept

* serialise -> serialize

* exstrinsics -> extrinsics

* panicks -> panics

* maintaince -> maintenance

* repeatidely -> repeatedly

* anecstor -> ancestor

* becasue -> because

* processer -> processor

* Prunning -> Pruning

* insterested -> interested

* unuseful -> not useful

* yeided -> yielded

* descendfing -> descending

* corresponts -> corresponds

* survivew -> survive

* keps -> keeps

* ligh -> light

* prerequisities -> prerequisites

* positiion -> position

* depedency -> dependency

* extrinisic -> extrinsic

* atomicaly -> atomically

* staticly -> statically

* resul -> result

* timestamb -> timestamp

* Utilites -> Utilities

* ammount -> amount

* pocess -> process

* exteral -> external

* Update client/finality-grandpa/src/tests.rs

* Update primitives/io/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update primitives/blockchain/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update frame/support/src/weights.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update bin/node/cli/tests/common.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/cli/src/params.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/executor/common/src/sandbox.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/finality-grandpa/src/communication/mod.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/state-db/src/pruning.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update frame/contracts/src/tests.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

* bump impl

* timestamb -> timestamp

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
2020-02-17 15:07:24 +01:00
Bastian Köcher 85aa632278 Make debug builds more usable (#4683)
* Make debug builds more usable

This pr makes debug builds more usable in terms of `cargo run -- --dev`.

1. `--dev` activates `--execution native`, iff `--execution` is not
given or no sub `--execution-*` is given.
2. It was probably a mistake to compile WASM in debug for a debug build.
So, we now build the WASM binary always as `release` (if not requested
differently by the user). So, we trade compilation time for a better
debug experience.

* Make sure we only overwrite default values

* Make it work

* Apply suggestion
2020-01-21 12:33:28 +01:00
Shawn Tabrizi 7efa62725e Update copyright year (#4532) 2020-01-05 14:05:44 +01:00
Bastian Köcher 43e34bb909 Make wasm-builder remove invalid members (#4510)
* Make wasm-builder remove invalid members

Wasm-builder now removes members that point to packages that do not
exist anymore or that were renamed. Up to now, the build would fail and
required manual fixing.

* Fixes build
2019-12-29 21:54:10 +01:00
Bastian Köcher 94b6921d03 Fix check_polkadot CI check (#4317)
* Fix `check_polkadot` CI check

* Another try

* Use path overrides

* Last fix

* Update .gitlab-ci.yml

* Fix wasm builder

* Update wasm_project.rs
2019-12-07 23:36:47 +01:00
Benjamin Kampmann 60e5011c72 Reorganising the repository - external renames and moves (#4074)
* Adding first rough ouline of the repository structure

* Remove old CI stuff

* add title

* formatting fixes

* move node-exits job's script to scripts dir

* Move docs into subdir

* move to bin

* move maintainence scripts, configs and helpers into its own dir

* add .local to ignore

* move core->client

* start up 'test' area

* move test client

* move test runtime

* make test move compile

* Add dependencies rule enforcement.

* Fix indexing.

* Update docs to reflect latest changes

* Moving /srml->/paint

* update docs

* move client/sr-* -> primitives/

* clean old readme

* remove old broken code in rhd

* update lock

* Step 1.

* starting to untangle client

* Fix after merge.

* start splitting out client interfaces

* move children and blockchain interfaces

* Move trie and state-machine to primitives.

* Fix WASM builds.

* fixing broken imports

* more interface moves

* move backend and light to interfaces

* move CallExecutor

* move cli off client

* moving around more interfaces

* re-add consensus crates into the mix

* fix subkey path

* relieve client from executor

* starting to pull out client from grandpa

* move is_decendent_of out of client

* grandpa still depends on client directly

* lemme tests pass

* rename srml->paint

* Make it compile.

* rename interfaces->client-api

* Move keyring to primitives.

* fixup libp2p dep

* fix broken use

* allow dependency enforcement to fail

* move fork-tree

* Moving wasm-builder

* make env

* move build-script-utils

* fixup broken crate depdencies and names

* fix imports for authority discovery

* fix typo

* update cargo.lock

* fixing imports

* Fix paths and add missing crates

* re-add missing crates
2019-11-14 21:51:17 +01:00