Build WASM binaries as part of cargo build (#2868)

* Introduce `wasm-builder` and `wasm-builder-runner` to retire `build.sh`

Make use of `wasm-builder` in `test-runtime`.

* Add build script and remove the wasm project

* Port `node-runtime` to new wasm-builder

* Make `substrate-executor` tests work with `wasm-builder`

* Move `node-template` to `wasm-builder`

* Remove `build.sh` :)

* Remove the last include_bytes

* Adds the missing build.rs files

* Remove `build.sh` from CI

* Debug CI

* Make it work in CI

* CI attempt 3

* Make `substrate-runtime-test` compile on stable

* Ahhh, some missed `include_bytes!`

* AHH

* Add suggestions

* Improve search for `Cargo.lock` and don't panic if it is not found

* Searching from manifest path was no good idea

* Make the `wasm-builder` source better configurable

* Expose the bloaty wasm binary as well

* Make sure to rerun WASM recompilation on changes in dependencies

* Introduce new `WASM_BUILD_TYPE` env and make sure to call `build.rs` on
changes to env variables

* Remove `build.sh` from READMEs

* Rename the projects

* Fixes CI

* Update lock file

* Fixes merge-conflict

* Apply suggestions from code review

Co-Authored-By: TriplEight <denis.pisarev@parity.io>

* Try to make windows happy

* Replace all back slashes in paths with slashes

* Apply suggestions from code review

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Use cargo from `CARGO` env variable

* Fix compilation

* Use `rustup` for running the nightly build

* Make individual projects skipable

* Fix compilation

* Fixes compilation

* Build all WASM projects in one workspace

* Replace more back slashes!

* Remove `inlcude_bytes!`

* Adds some documentation

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* More review comments

* Update `Cargo.lock`

* Set license

* Apply suggestions from code review

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* More review comments + adds `TRIGGER_WASM_BUILD` env

* Fix doc tests

* Increase version + update README

* Switch crates.io version of `wasm-builder`

* Update README

* Switch to released version of `wasm-builder-runner`

substrate/.gitlab-ci.yml

@@ -95,8 +95,7 @@ cargo-check-benches:
   stage:                           test
   <<:                              *docker-env
   script:
-    - ./scripts/build.sh --locked
-    - time cargo check --benches
+    - BUILD_DUMMY_WASM_BINARY=1 time cargo check --benches
     - sccache -s
 
 
@@ -107,7 +106,7 @@ cargo-check-subkey:
     - /^v[0-9]+\.[0-9]+.*$/        # i.e. v1.0, v2.1rc1
   script:
     - cd ./subkey
-    - time cargo check --release   # makes sense to save artifacts for building it
+    - BUILD_DUMMY_WASM_BINARY=1 time cargo check --release   # makes sense to save artifacts for building it
     - sccache -s
 
 
@@ -122,7 +121,6 @@ test-linux-stable:                 &test-linux
     variables:
       - $DEPLOY_TAG
   script:
-    - ./scripts/build.sh --locked
     - time cargo test --all --release --verbose --locked
     - sccache -s
 
@@ -160,7 +158,6 @@ test-linux-stable-int:
     variables:
       - $DEPLOY_TAG
   script:
-    - ./scripts/build.sh --locked
     - time RUST_LOG=sync=trace,consensus=trace,client=trace,state-db=trace,db=trace,forks=trace,state_db=trace,storage_cache=trace
         cargo test -p node-cli --release --verbose --locked -- --ignored --test-threads=1
     - sccache -s
@@ -215,7 +212,6 @@ build-linux-release:
     variables:
       - $DEPLOY_TAG
   script:
-    - ./scripts/build.sh --locked
     - time cargo build --release --verbose
     - mkdir -p ./artifacts
     - mv ./target/release/substrate ./artifacts/.
@@ -245,7 +241,6 @@ build-rust-doc-release:
     - ./crate-docs
   <<:                              *build-only
   script:
-    - ./scripts/build.sh --locked
     - rm -f ./crate-docs/index.html # use it as an indicator if the job succeeds
     - time cargo +nightly doc --release --all --verbose
     - cp -R ./target/doc ./crate-docs