Commit Graph

96 Commits

Author SHA1 Message Date
Alexander Samusev 13fb87c8f3 [ci] Optimizing pipeline (#5001)
* [DO NOT MERGE] Optimizing pipeline

* add stages

* remove lock for tests

* return locked

* empty commit for pipeline rerun

* add comment about DAG

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>

Co-authored-by: Denis Pisarev <17856421+TriplEight@users.noreply.github.com>
2022-03-03 16:26:54 +01:00
Andronik 55e2ddd91c CI: add rococo to the runtime check (#5013)
* ci: add rococo for runtime changes check

* rococo: bump spec_version to 9180

* Update Rococo to 1-day lease (#4761)

* changed lease period to 1 day

* bumped version

* bumped version again

* changed 356 to 365 days to mimic one year

Co-authored-by: Santi Balaguer <santibalaguer@Santis-MacBook-Pro.local>

* rococo: bump spec_version to 9170 for consistency

Co-authored-by: Santi Balaguer <santiago.balaguer@gmail.com>
Co-authored-by: Santi Balaguer <santibalaguer@Santis-MacBook-Pro.local>
2022-03-02 09:14:38 +00:00
Bastian Köcher 550e81b63f Check that try-runtime compiles in CI (#4736)
It also moves some simple scripts directly into the .gitlab.yaml file.
2022-01-18 16:17:07 +01:00
sandreim 0da05cac26 Remove sleep from tests (#4639)
* Remove sleep and use polkadot test service

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* updates

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix other tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Run metrics tests separately

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* copy some substrate utilities

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update runtime metric test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove sleep from cli tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Polkadot companion for Substrate#10463 (#4519)

* Grandpa and Beefy protocol names include chain id

Signed-off-by: acatangiu <adrian@parity.io>

* chain_spec: include fork id

* use simplified protocol name

* fix after merge

* avoid using hash default, even for protocol names

* update lockfile for substrate

Co-authored-by: parity-processbot <>

* configuration: Update upgrade validation delay doc (#4662)

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* use testnet profile

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Don't run with runtime-benchmark feature

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* conditional compile up one level

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2022-01-10 11:00:46 +00:00
Sergei Shulepov d752f8664a pvf-precheck: PVF pre-checker subsystem (#4643)
This commit implements the last major piece of #3211: the subsystem that
tracks PVFs that require voting, issues pre-check requests to
candidate-validation and makes sure that the votes are submitted to the
chain.
2022-01-07 18:10:23 +00:00
Andronik a44349f652 custom cargo profiles for production (#4627)
* add custom profiles and specify MSVR as 1.57

* fix host-perf-check command

* use testnet profile for CI images

* do not do lto for testnet profile

* fix artifact path

* test with testnet profile to reuse build artifacts

* Revert "fix host-perf-check command"

This reverts commit f1d15492204b8251685a97636cbb5a5f394f21da.

* bump zombienet version

Co-authored-by: Javier Viola <javier@parity.io>
2021-12-29 14:10:40 +00:00
Bernhard Schuster 2457c26a08 enable disputes for known chains, except for polkadot (#4464)
* enable disputes, for all known chains but polkadot

* chore: fmt

* don't propagate disputes either

* review

* remove disputes feature

* remove superfluous line

* Update node/service/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* fixup

* allow being a dummy

* rialto

* add an enum, to make things work better

* overseer

* fix test

* comments

* move condition out

* excess arg

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-12-17 17:50:48 +00:00
sandreim e75ad88ae8 Add support for wasm runtime metrics try #2 (#4483)
* Add runtime metrics provider

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Runner changes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Some sample metrics in paras_inherent

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update cargo toml

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bug

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fmt after merge

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Refactor metric prefix override

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove bug comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add runtime metric primitives

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Impl trace event parsing

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update metrics

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix target check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Runtime metrics primitives

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Runtime metrics crate

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Node side runtime metric changes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* use runtime CounterVec instead of macro

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt nice

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove dead code

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* base58 decoding

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* base58 encoding

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix warn

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Finish label support

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* please compile

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add feature gate

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Comment cargo toml

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix cargo toml description

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update doc.

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* switch to `runtime-metrics` feature

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo toml

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* better ux

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* from_utf8_unchecked is safe

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add Counter and refactor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add integration test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* dev deps

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* gitlab script update

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Run tests twice

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* small fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck happy ?

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Guard tracing init

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* missing copyright

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update lockfile for substrate

Co-authored-by: parity-processbot <>
2021-12-16 11:56:25 +00:00
Steve Degosserie b3d08c0a8e Rococo: ability to programatically assign slots to teams (#3943)
* Permanent & Temp parachain slots on Rococo - WIP

* Revert test change

* Revert test change

* Fix formatting

* Extract logic to separate assigned_slots pallet

* Formatting

* Parachain downgrade logic

* Pallet doc comment

* Revert unnecessary changes

* Fix few issues, tweak temp slots allocation logic; add a bunch of tests

* Address review comments; track active temp slots

* Update runtime/common/src/assigned_slots.rs

* Update runtime/common/src/assigned_slots.rs

* Remove assigned_slots calls from paras_sudo_wrapper

* Unassign is a perfectly valid verb

* Remove unneeded collect

* Update code following #3980

* Cleanup

* Generate storage info for pallet

* Address review comments

* Add ForceOrigin to slots pallet

* Track permanent slot duration in storage

* Fix tests build

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
2021-12-02 22:23:45 +01:00
Sergei Shulepov 8f75230e42 pvf-precheck: Candidate Validation Changes (#4409)
* pvf-precheck: Candidate Validation Changes

Co-authored-by: Chris Sosnin <chris125_@live.com>

* Rename `MockValidationBackend` and specialize it

* Add pre-check tests

Co-authored-by: Chris Sosnin <chris125_@live.com>
2021-12-01 00:53:39 +01:00
Lldenaurois 2243121401 Revert "remove provisioner checks (#4254)" (#4375)
* Revert "remove provisioner checks (#4254)"

This reverts commit d5d916a915.

* Remove TODO in implementer's guide
2021-11-30 11:01:55 +00:00
sandreim 0912d6a95e Fix spellcheck (#4363)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-11-24 16:15:04 +01:00
Javier Viola ea5dbd0475 introduce malus + zombienet based integration tests (#4131)
* test/malus: craft the first maliciously disputing actor

* initial draft

* Add Dockerfile and instructions how to use it to build malus image locally

* Forgot one flag for the build cmd

* we are not docker specific, we are happy to use any containerruntime

* shuffle things around

* add initial tera based integration test

* chores

* fixins

* simple setup to start

* other samples (WIP)

* add Docker version with cargo-chef

* update substarte and small change of orders in commands in the container file

* metrics one

* fmt

* minor

* fixin

* fix metric names

* -d

* add open gauge

* fmt

* spellcheck

* fix test

* adjust to changed error messages

* refactor, more malus impls

* more malus changes

* foo

* minor cleanup

* suggest garbage candidate

* chore

* fix suggest garabge malus

* malus: back garbage candidate

* cargo lock

* re-introduce metrics

* chore: cargo fmt

* undoe 1.54.0 output, CI uses 1.53.0 rustc

* update location of js types

* Fix trybuild

* add tag to image name also; this will be replaced in the prod version

* Tests fixed

* add some fix me

* add dockerfile for ci

* Add docker file for malus for ci

* use variables in .toml file

* add chnages for malus test

* some fixes

* some more fixes

* Update .gitlab-ci.yml

* add local build for polkadot and malus

* some fixes

* enable disputes feature in CI

* ok, ok

* rename: MsgFilter -> MessageInterceptor

* remove TODO that would not have worked

* intercept

* refactor

* fix README and containers

* fix

* chore: cargo fmt

* avoid some more malus-$VARIANT references

* fix argument order

* chore: add about

* Update sanity check in relay chain selection

* fix order, add dispute-unavailable-block malus

* fixup: avoid underflow issue

* it's all u32

* fix conditional use

* Revert "it's all u32"

This reverts commit 6b3ae25bfd0bbf0b51d90d743642a75a4a815d6e.

* Revert "fixup: avoid underflow issue"

This reverts commit 336cbe2938e9720f870d37d8feeab7ca69200f47.

* Revert "Update sanity check in relay chain selection"

This reverts commit 970647f35e1116136e12fd91cd9f2fb7e18ad28d.

* update the malus bin

* Update node/malus/integrationtests/0003-dispute-unavailable-block.feature

Co-authored-by: Andronik Ordian <write@reusable.software>

* add some FIXME reminders

* update path to index.js

* Update .gitlab-ci.yml

* Update node/malus/integrationtests/0001-dispute-valid-block.toml

* try 1: make malus test run

* chore: cargo fmt

* temporary fix

* use subcommand syntax from latest gurke

* cargo +nightly fmt

* add collator to a a test

* docs: add env vars to README

* update ci to run dispute-valid-block test

* needs the polkadot image

* Fix path for nodejs container

* post merge fix

* download proper dir  containg configs for malus test

* update the malus ci job

* rm a whitespace

* temp build for malus

* use correct build command for temp malus

* remove subcommands for now

* set max validators per core in the default HostConfig

* tabs

* update beefy

* fixup

* fixup II

* make one variant compile

* make other variants compile

* revert changes to chain_spec

* fmt

* build malus image from polkadot-test-malus again

* revert unrelated changes

* try fixing build-malus job

* Revert "remove subcommands for now"

This reverts commit 5d8292bc49252124937affec4b7c28181a5deb7e.

* try fixing build-malus job II

* MVP working dispute-ancestor

* renames

* fix PVF execution on malus

* fix test

* fix typo

* fmt

* checkmate

* try something

* make it actually work

* some tweaks to 01 feature test

* fmt

* sleep a bit more

* complete wococoization

* some tweaks to 01 feature test

* typo fix

* use correct metric names

* fix

* ffs

* .

* try some rearrangement

* Attempt to wait till initial node bootstrap in test

* Fix test syntax

* Run malus tests with v2 script

* Proper symlink created

* simnet v14

* add zombienet tests

* add zombie net test - draft

* add more tests to dispute suite

* add within to fix tests

* replace test directory and start test migration

* migrate all the tests

* add timeout to tests

* reduce debug

* make easy to test in dev

* set appropriated debug

* use image from ci

* fix config for test

* set images from ci

* fix config

* add COLIMAGE env

* tweek tests

* reduce debug

* typo

* wip, migrate old test to zombie-net

* adjunt test config for zombie-net

* run mauls 0001 test only

* clean and setup smoke-test in zombie-net

* add extra time to assertinons

* clean code to merge and improve README

* add info to access logs

* improved readme

* merge master and resolve conflicts

* Update zombienet_tests/README.md

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* clean and consolidate zombienet name

* change runner in gitlab

* add comment explain why we use wococo

* change tag for runner

* remove unused tests

* remove dup Dockerfile and update description

* fmt

* fix compilation post-merge

* fmt

* cut me Some slack

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Anton Gavrilov <AntonE.Gavrilov@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2021-11-20 14:03:28 +00:00
Bernhard Schuster d5d916a915 remove provisioner checks (#4254)
* chore/provisioner: move metrics to a separate module

* avoid the duplicate names

* reduce all checks

* fixup tests

* Update node/core/provisioner/src/lib.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* chore: fmt

* chore: spellcheck

* doc

* remove the enum anti-pattern

* guide update - remove all the responsibilities

* add another trivial check

* Update node/core/provisioner/src/metrics.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update roadmap/implementers-guide/src/node/utility/provisioner.md

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update node/core/provisioner/src/metrics.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
2021-11-19 18:15:59 +00:00
Chris Sosnin f5fbaa139f PVF host prechecking support v2 (#4123)
* pvf host: store only compiled artifacts on disk

* Correctly handle failed artifacts

* Serialize result of PVF preparation uniquely

* Set the artifact state depending on the result

* Return the result of PVF preparation directly

* Move PrepareError to the error module

* Update doc comments

* Update misleading comment

* pvf host: turn off parallel compilation

* pvf host: implement precheck requests

* Fix warnings

* Unnecessary clone

* Add a note about timed out outcome

* Revert the pool outcome handling behavior

* Move the prepare result type into error mod

* Test prepare done

* fmt

* Add an explanation to wasmtime config

* Split pvf host test

* Add precheck to dictionary

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2021-11-13 17:25:59 +01:00
Denis Pisarev 9d5724f926 CI: chore (#3957)
* CI: chore

* CI: lsof and jq are installed

* CI: generate docs with deps

* CI: remove check width, fmt checks it

* CI: simplify rules

* CI: make CI image variable

* CI: more chore

* rm check_line_width.sh

* CI: fix spellcheck; fix check-transaction

* CI: return installing jq

* lint: remove dupes
2021-11-11 17:38:05 +01:00
Andronik Ordian 8aba3ea8ca some spelling fixes (#4088)
* quotes around Job::ToJob
* add new words to dictionary
2021-10-18 11:59:46 +02:00
Keith Yeung 769595eea5 Create a README for XCMv1 detailing notable changes (#4058)
* Create a README for XCMv1 detailing notable changes

* Update xcm/src/v1/README.md

Co-authored-by: Gavin Wood <gavin@parity.io>

* Update xcm/src/v1/README.md

Co-authored-by: David <dvdplm@gmail.com>

* Update xcm/src/v1/README.md

Co-authored-by: David <dvdplm@gmail.com>

* Update xcm/src/v1/README.md

Co-authored-by: David <dvdplm@gmail.com>

* Update xcm/src/v1/README.md

Co-authored-by: David <dvdplm@gmail.com>

* Link to relevant PR for a full list of changes

* Typo

* Add notable changes to top-level XCM rustdoc

* Appease spellchecker

Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: David <dvdplm@gmail.com>
2021-10-13 11:17:30 +00:00
Sergejs Kostjucenko f31eac6ec6 Remove bridges from checks (#4040)
* Remove bridges from check_line_width

* Remove bridges from spellcheck
2021-10-08 14:43:17 +03:00
Bernhard Schuster 30bdc8f5d4 collect included disputes from on-chain (#3924)
* dummy: impl another runtime API

* query the on chain disputes, and inform self

* make use of the refactor

* minro

* SPLIT ME

* write dispute values

* wip

* impl for all runtimes

* chore: fmt

* [] -> get

* fixup mock runtime

* fixup

* fixup discovery for overseer init

* chore: fmt

* spellcheck

* rename imported_on_chain_disputes -> on_chain_votes

* reduction

* make it mockable

* rename and refactor

* don't query on chain info if it's not needed

* yikes

* fmt

* fix test

* minimal fix for existing tests

* attempt to fetch the session info from the rolling window before falling back

* moved

* comments

* comments

* test for backing votes

* rename

* Update runtime/polkadot/src/lib.rs

* chore: spellcheck + dict

* chore: fmt

* fixup cache size

* add warning

* logging, rationale, less defense

* introduce new unchecked, that still checks in debug builds

* fix

* draft alt approach

* fix unused imports

* include the session

* Update node/core/dispute-coordinator/src/real/mod.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* provide where possible

* expand comment

* fixin

* fixup

* ValidityVote <-> ValidityAttestation <-> CompactStatement has a 1:1 representation

* mark TODO

* Update primitives/src/v1/mod.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* address review comments

* update docs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2021-10-06 19:16:23 +00:00
Gavin Wood 3bab876bc1 Tidy up XCM errors in preparation for v2. (#3988)
* Tidy up XCM errors

* Tidy up errors

* Re-order

* Fixes

* Formatting

* map undefined errors

* add functor to dictionary

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-10-02 00:14:13 +02:00
Andronik Ordian d19b1de5f0 CI: run disputes tests (#3962)
* CI: run disputes tests

* Revert "minor chore changes (#3944)"

This reverts commit d64394ef59.

* fix em

* Revert "Revert "minor chore changes (#3944)""

This reverts commit 02a1ecf3cacb52d6dd467e8f226a69bae651d319.

* Update handle_import_statement function

* fmt

Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2021-09-29 11:19:00 +00:00
Alexander Samusev f29b6db6e4 Docker files chore (#3880)
* Renamed staking-miner Dockerfiles

* change readme stacking-miner

* mv docker/ to scripts/docker/polkadot

* renamed dockerfiles in scripts/docker

* renamed Dockerfiles

* Update scripts/docker/staking-miner/staking_miner_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* renamed release.Dockerfile

* renamed staking-miner dockerfiles

* add label polkadot_builder.Dockerfile

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot/polkadot_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/polkadot_injected_release.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/collator_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_builder.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* Update scripts/docker/staking-miner/staking-miner_injected.Dockerfile

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

* move gpg server as variable to GHA config

* simplified polkadot_builder.Dockerfile

* gitlab-ci modify cp command in test-build-linux-stable

* fix tests

* renamed scripts/docker to scripts/dockerfiles

* fix spellcheck

* fix spellcheck

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
2021-09-28 17:47:52 +02:00
Chevdor 35f25c3f54 Fix spelling (#3845)
* Fix spelling

* Add missing entry

* Proper fix for KeyPair
2021-09-25 22:44:39 +02:00
Sergei Shulepov 13906d62f0 UMP: Support Overweight messages (#3575)
* Introduce new config: ump_max_individual_weight

* Implement overweight msg stashing

* Test

* Add migration module.

Also introduces a test for migration

* Integrate ExecuteOverweightOrigin to runtimes

* Fix more stuff

* Add `yeet` into dictionary

* Use suggested `Error` variant names

* typo

* Use 20ms as the maximum individual message weight

* Update the test value

* rustfmt

* Clean up

* Remove deprecated field from host config

* Remove missed _hrmp_open_request_ttl

* Apply typo fix suggestion

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Rename `migration::migrate_to_latest`

* Restore `_hrmp_open_request_ttl` in `v0::HostConfiguration`

* Apply suggestion for a rustdoc

* Apply the suggestion

* Test v0 config with the raw production data fetched from Kusama

* Update runtime/parachains/src/ump.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Expose migration functions

* Fix spellcheck

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
2021-09-15 11:13:44 +02:00
Andronik Ordian 33d7f9dd7b make polkadot-runtime optional feature (#3820)
* make polkadot-runtime optional feature

* sprinkle some cfg statements

* ok

* ok, ok

* add CI check

* set -e

* chmod +x

* fixes

* fmt

* nicer compile errors

* Update outdated comments
2021-09-13 10:28:33 +00:00
Sergei Shulepov 83a35874ee Do not expire HRMP open channel requests (#3543)
* Do not expire HRMP open channel requests

* Fix the build and update the docs

* Implement canceling requests and do not remove them automatically

* Fix a borked merge

* Fix fmt

* Please spellchecker

* Apply suggestions from code review

Co-authored-by: Amar Singh <asinghchrony@protonmail.com>

* Use `mutate_exists` for maintaining request counts

* Apply `rustfmt`

* Move newly introduced entrypoint to end to preserve ordering

Co-authored-by: Amar Singh <asinghchrony@protonmail.com>
2021-09-09 14:06:13 +02:00
Chevdor bb6a3fb0a2 staking-miner: docker images (#3682)
* staking-miner: docker image

* renaming

* add doc

* fix readme

* remove the creation of home for the polkadot user

* new builder image for the staking-miner

* remove mentions of the polkadot binary and add env

* foc: fix doc

* fix typos and dictionnary

* doc + sync up the 2 images

* doc: fine tuning the doc

* doc: security warning
2021-09-07 08:53:30 +00:00
Squirrel 80f8c5106e Removed unneeded deps (#3658)
* Removed unneeded deps (except under bridge)

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-08-30 18:18:33 +01:00
Keith Yeung 64cbeab3b3 Return a Result in invert_location (#3730)
* Return a Result in invert_location

* Add invertible to spellcheck dictionary

* Fix Some -> Ok
2021-08-28 02:17:51 +02:00
Gavin Wood 8b80b283d4 XCM v2: Scripting, Query responses, Exception handling and Error reporting (#3629)
* Intoduce XCM v2

Also some minor fix for v0/v1

* Minor version cleanup

* Minor version cleanup

* Introduce SendError for XcmSend trait to avoid cycles with having Outcome in Xcm

* comment

* Corrent type

* Docs

* Fix build

* Fixes

* Introduce the basic impl

* Docs

* Add function

* Basic implementation

* Weighed responses and on_report

* Make XCM more script-like

* Remove BuyExecution::orders

* Fixes

* Fixes

* Fixes

* Formatting

* Initial draft and make pallet-xcm build

* fix XCM tests

* Formatting

* Fixes

* Formatting

* spelling

* Fixes

* Fixes

* spelling

* tests for translation

* extra fields to XCM pallet

* Formatting

* Fixes

* spelling

* first integration test

* Another integration test

* Formatting

* fix tests

* all tests

* Fixes

* Fixes

* Formatting

* Fixes

* Fixes

* Formatting

* Bump

* Remove unneeded structuring

* add instruction

* Fixes

* spelling

* Fixes

* Fixes

* Formatting

* Fixes

* Fixes

* Formatting

* Introduce and use VersionedResponse

* Introduce versioning to dispatchables' params

* Fixes

* Formatting

* Rest of merge

* more work

* Formatting

* Basic logic

* Fixes

* Fixes

* Add test

* Fixes

* Formatting

* Fixes

* Fixes

* Fixes

* Nits

* Simplify

* Spelling

* Formatting

* Return weight of unexecuted instructions in case of error as surplus

* Formatting

* Fixes

* Test for instruction count limiting

* Formatting

* Docs
2021-08-26 12:41:16 +02:00
Martin Pugh e014e53f4a remove check_web_wasm job (#3654) 2021-08-17 20:02:34 +02:00
radupopa2010 fdb1653783 Simnet v8 move tests (#3647)
* Update to simnet v8

* rm duplicate

* add tags also

* revertme: add simnet image version for testing CI

* add info regarding the image names and values

* fix lint error

* one more try to fix the lint

* hardcode simnet version and revertme

* hardcode simnet

* more hardcodding

* I wonder how may silly mistakes I can make

* simnet image name needs to be hardcoded

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Update .gitlab-ci.yml

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* remove revertme changes

* rm unused script

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2021-08-17 16:49:52 +02:00
Bernhard Schuster ac338b6c28 collect better memory stats (#3612)
* add jemalloc memory statistics tracking

* chore: move Metronome in a separate file

* add meta flag spellcheck

* adjust metrics names

* account for new metrics in test
2021-08-13 12:40:45 +02:00
Gavin Wood ce80bc2d4c XCM v1 (#2815)
* MultiAsset TWO

* Draft next MultiAsset API.

* XCM core builds

* XCM Executor builds

* XCM Builder builds

* API changes making their way throughout

* Some TODOs

* Further build fixes

* Basic compile builds

* First test fixed

* All executor tests fixed

* Typo

* Optimize subsume_assets and add test

* Optimize checked_sub

* XCM Builder first test fixed

* Fix builder tests

* Fix doc test

* fix some doc tests

* spelling

* named fields for AllOf

* Update xcm/src/v0/multiasset.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update xcm/src/v0/multiasset.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update xcm/src/v0/multiasset.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update xcm/src/v0/multiasset.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Reformat

* Move to XCM version 1

* Spelling

* warnings

* Replace some more v0->v1s

* warnings

* format

* Add max_assets param

* building

* test fixes

* tests

* another test

* final test

* tests

* Rename Null -> Here

* Introduce

* More ergonomics

* More ergonomics

* test fix

* test fixes

* docs

* BuyExecution includes

* Fix XCM extrinsics

* fmt

* Make Vec<MultiAsset>/MultiAssets conversions safe

* More MultiAssets conversion safety

* spelling

* fix doc test

* Apply suggestions from code review

Co-authored-by: Amar Singh <asinghchrony@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Amar Singh <asinghchrony@protonmail.com>

* fmt

* Add v0, remove VersionedMultiAsset

* Remove VersionedMultiLocation

* Update xcm/src/v1/order.rs

Co-authored-by: Amar Singh <asinghchrony@protonmail.com>

* Update xcm/src/v1/mod.rs

Co-authored-by: Amar Singh <asinghchrony@protonmail.com>

* XCM v0 backwards compatibility

* Full compatibility

* fmt

* Update xcm/pallet-xcm/src/lib.rs

* Update xcm/src/v0/order.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Tweaks to versioning system

* Fixes

* fmt

* Update xcm/xcm-executor/src/assets.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update xcm/xcm-executor/src/assets.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Grumbles

* Update xcm/src/v1/multiasset.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* fmt

* Update xcm/src/v1/multiasset.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update xcm/src/v1/multiasset.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Fixes

* Formatting

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Amar Singh <asinghchrony@protonmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-08-06 18:25:01 +02:00
Andronik Ordian 33fe763188 some fixes to please cargo-spellcheck (#3550)
* some fixes to please cargo-spellcheck

* some (not all) fixes for the impl guide

* fix
2021-08-02 11:00:04 +00:00
Squirrel 4c5b73097c Companion for substrate #9319 (#3456)
* move client consensus code out of primitives

* merging crates

* import tweak

* Fixing build: ServiceFactory is a word...

* updating lock file

* Fixed typo in error message

(to bump build)

* update Substrate

Co-authored-by: parity-processbot <>
2021-07-30 14:02:43 +00:00
Denis Pisarev fc253e6e4d WIP: CI: add spellcheck (#3421)
* CI: add spellcheck

* revert me

* CI: explicit command for spellchecker

* spellcheck: edit misspells

* CI: run spellcheck on diff

* spellcheck: edits

* spellcheck: edit misspells

* spellcheck: add rules

* spellcheck: mv configs

* spellcheck: more edits

* spellcheck: chore

* spellcheck: one more thing

* spellcheck: and another one

* spellcheck: seems like it doesn't get to an end

* spellcheck: new words after rebase

* spellcheck: new words appearing out of nowhere

* chore

* review edits

* more review edits

* more edits

* wonky behavior

* wonky behavior 2

* wonky behavior 3

* change git behavior

* spellcheck: another bunch of new edits

* spellcheck: new words are koming out of nowhere

* CI: finding the master

* CI: fetching master implicitly

* CI: undebug

* new errors

* a bunch of new edits

* and some more

* Update node/core/approval-voting/src/approval_db/v1/mod.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update xcm/xcm-executor/src/assets.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <write@reusable.software>

* Suggestions from the code review

* CI: scan only changed files

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-07-14 19:22:58 +02:00
radupopa2010 e6881457b8 Make script usable from both pipeline and localhost (#3147)
* Make script usable from both pipeline and localhost

* use the new argument --simnet-version=

* dummy, to trigger pipeline one more time

* CI: optimize the script

* No arg for help flag

* CI: comment and unneded parameter

Co-authored-by: Denis P <denis.pisarev@parity.io>
2021-06-01 16:52:15 +02:00
radupopa2010 a7a0270a9f Update simnet tests to v4 (#3136)
* Update simnet tests to v4

* CI: revert me

* Revert "CI: revert me"

This reverts commit 2693b0640a0410bdf608ccdbb0422b831c5283e8.

Co-authored-by: Denis P <denis.pisarev@parity.io>
2021-05-31 14:37:24 +02:00
Denis Pisarev 6e5d90a972 WIP: CI: simnet to run pinned versions (#3051)
* CI: simnet to run pinned versions

* CI: explanation and tag

* dummy commit to trigger pipeline

* try with a new tag

* update new simnet stable version

Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
2021-05-27 15:02:30 +02:00
Robert Klotzner 601a3781c1 Always connect in collator protocol. (#2980)
* Always connect in collator protocol.

* Fix unused import.

* We always issue connection requests now.

* Fix stupid boolean logic with one variable.

* Fix CI.
2021-05-06 12:54:54 +02:00
Denis Pisarev 683211837b CI: optimizations (#2885)
* CI: do not publish unnecessary things to s3 and to release registry

* CI: remove the build-linux-release job

* CI: publish less

* CI: workspace = all

* CI: more logs for the pipeline trigger

* CI: chore

* CI: nicer

* CI: new bucket for bins

* CI: no need pushing every master image to product registry

* CI: default variables in image jobs

* CI: optimize the bin build
2021-04-22 10:48:15 +02:00
Denis Pisarev 00812ac9bb Simnet triggers and images (#2714)
* CI: new image for simnet

* CI: chore

* CI: separate image for collator

* dockerfile: it's bad upgrading in the images

* CI: correct links to Dockerfiles

* CI: right dotenv usage

* CI: workaround GitLab's bug that failed trigger status is not reported and 'job runs forever'

* CI: debug: need instead of rules to avoid the bug

* dockerfile: fix storage

* CI: workaround sending the status when the trigger is failed

* CI: fix needed job

* CI: edit comments

* CI: trigger simnet with API

* CI: triggering script

* CI: remove real-overseer

* CI: trigger real simnet
2021-04-14 11:31:46 +02:00
Robert Habermeier 57038b2e46 Remove real-overseer 🎉 (#2834)
* remove real-overseer

* overseer: only activate leaves which support parachains

* integrate HeadSupportsParachains into service

* remove unneeded line
2021-04-08 20:24:06 +02:00
Martin Pugh 82fa27d68a [CI] Require D*-audit labels for any runtime changes (#2617)
* update generate_release_text.rb

* add checks for audit labels for runtime changes

* fix check_labels.sh

* use has_runtime_changes in check_runtime.sh

* add D1-trivial label

* fix check_labels.sh
2021-03-23 13:15:08 +01:00
Gavin Wood 3c8b9271c4 Fix up CI for new labels (#2673)
* Fix up CI/CD for the new labels.

* Fix labels
2021-03-23 12:17:29 +01:00
Martin Pugh 62c5896592 [CI] Fix check-tx-version Gitlab job (#2449)
* set -e for tx job

* dont set -e while we investigate

* fetch latest release polkadot

* add jq to node:15 image

* Add logic for timing out if nodes never start listening

* remove trap
2021-02-17 16:25:44 +01:00
Martin Pugh 013b271359 [CI] Move check_labels to github actions (#2415)
* move lib.sh to common dir

* make check-labels a github action

* check out the repo

* fix secrets

* add labelled PR type

* remove check-labels job from gitlab

* trigger on unlabelling also
2021-02-10 17:50:49 +00:00
Bastian Köcher 1439558f14 Compress the PoV block before sending it over the network (#2288)
* Compress the PoV block before sending it over the network

This pr changes the way we send PoV blocks over the network. We now
compress the PoV block before it is send over the network. This should
reduce the size significant for PoVs which contain the runtime WASM for
example.

* Preallocate 1KB

* Try something..

* Switch to zstd and some renamings

* Make compression/decompression fail in browsers

* Use some sane maximum value

* Update roadmap/implementers-guide/src/types/network.md

Co-authored-by: Andronik Ordian <write@reusable.software>

* Fix and add test

* add

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2021-01-21 18:04:14 +00:00