pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-06-12 03:01:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin' into gav-xcm-v3

Browse Source
This commit is contained in:
Keith Yeung
2022-08-14 07:19:05 +08:00
parent 4453f082d5 16431dd1d9
commit 9c6296ca95
311 changed files with 18656 additions and 7266 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
+1 -1
View File
@@ -3,7 +3,7 @@
**/*.txt
**/*.md
/docker/
!/target/release/polkadot-collator
!/target/release/polkadot-parachain
# dotfiles in the repo root
/.*
.github/ISSUE_TEMPLATE/release-client.md
+20
View File
@@ -0,0 +1,20 @@
---
name: Release Checklist for Client
about: Release Checklist for Client
title: Release Checklist for Client {{ env.VERSION }}
---
# Release Checklist - Client
### Client Release
- [ ] build a new `polkadot-parachain` binary and publish it to S3
- [ ] new `polkadot-parachain` version has [run on the network](../../docs/release.md#burnin)
without issue for at least 12h
- [ ] a draft release has been created in the [Github Releases page](https://github.com/paritytech/cumulus/releases) with the relevant release-notes
- [ ] the [build artifacts](../../docs/release.md#build-artifacts) have been added to the
draft-release.
---
Read more about the [release documentation](../../docs/release.md).
.github/ISSUE_TEMPLATE/release-runtime.md
+46
View File
@@ -0,0 +1,46 @@
---
name: Release Checklist for Runtime
about: Release Checklist for Runtime
title: Release Checklist for Runtime {{ env.VERSION }}
---
# Release Checklist - Runtimes
## Runtimes Release
### Codebase
These checks should be performed on the codebase.
- [ ] the [`spec_version`](https://github.com/paritytech/cumulus/blob/master/docs/release.md#spec-version) has been incremented since the
last release for any native runtimes from any existing use on public (non-private/test) networks
- [ ] previously [completed migrations](https://github.com/paritytech/cumulus/blob/master/docs/release.md#old-migrations-removed) are
removed for any public (non-private/test) networks
- [ ] No migrations added in the last release that would need to be removed
- [ ] pallet and [extrinsic ordering](https://github.com/paritytech/cumulus/blob/master/docs/release.md#extrinsic-ordering--storage) as well as `SignedExtension`s have stayed
the same. Bump `transaction_version` otherwise
- [ ] the [benchmarks](https://github.com/paritytech/ci_cd/wiki/Benchmarks:-cumulus) ran
- [ ] the weights have been updated for any modified runtime logic
- [ ] the various pieces of XCM config are sane
### On the release branch
The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)
- [ ] Verify [new migrations](https://github.com/paritytech/cumulus/blob/master/docs/release.md#new-migrations) complete successfully, and the
runtime state is correctly updated for any public (non-private/test)
networks
- [ ] Run [integration tests](https://github.com/paritytech/cumulus/blob/master/docs/release.md#integration-tests), and make sure they pass.
- [ ] Push runtime upgrade to Westmint and verify network stability
### Github
- [ ] Check that a draft release has been created at the [Github Releases page](https://github.com/paritytech/cumulus/releases) with relevant [release
notes](https://github.com/paritytech/cumulus/blob/master/docs/release.md#release-notes)
- [ ] Check that [build artifacts](https://github.com/paritytech/cumulus/blob/master/docs/release.md#build-artifacts) have been added to the
draft-release.
---
Read more about the [release documentation](https://github.com/paritytech/cumulus/blob/master/docs/release.md).
.github/ISSUE_TEMPLATE/release.md
-156
View File
@@ -1,156 +0,0 @@
---
name: Release issue template
about: Tracking issue for new releases
title: Cumulus {{ env.VERSION }} Release checklist
---
# Release Checklist
### Runtime Releases
These checks should be performed on the codebase.
- [ ] Verify [`spec_version`](#spec-version) has been incremented since the
last release for any native runtimes from any existing use on public
(non-private/test) networks.
- [ ] Verify previously [completed migrations](#old-migrations-removed) are
removed for any public (non-private/test) networks.
- No migrations added in the last release that would need to be removed.
- [ ] Verify pallet and [extrinsic ordering](#extrinsic-ordering) as well as `SignedExtension`s have stayed
the same. Bump `transaction_version` if not.
- [ ] Verify new extrinsics have been correctly whitelisted/blacklisted for
[proxy filters](#proxy-filtering).
- No new extrinsics.
- [ ] Verify [benchmarks](#benchmarks) have been updated for any modified
runtime logic.
- [ ] Verify the weights are up-to-date.
- [ ] Verify that the various pieces of XCM config are sane.
The following checks can be performed after we have forked off to the release-
candidate branch or started an additional release candidate branch (rc-2, rc-3, etc)
- [ ] Verify [new migrations](#new-migrations) complete successfully, and the
runtime state is correctly updated for any public (non-private/test)
networks.
- [ ] Run integration tests.
- [ ] Teleport Relay -> Statemin* and back.
- [ ] Create asset (if applicable), mint and transfer
- [ ] Push runtime upgrade to Westmint and verify network stability.
### All Releases
- [ ] Check that the new polkadot-collator versions have [run on the network](#burn-in)
without issue.
- [ ] Check that a draft release has been created at
https://github.com/paritytech/cumulus/releases with relevant [release
notes](#release-notes).
- [ ] Check that [build artifacts](#build-artifacts) have been added to the
draft-release.
---
## Notes
### Burn In
Ensure that Parity DevOps has run the new release on Westmint and Statemine collators for 12h prior to publishing the release.
### Build Artifacts
Add any necessary assets to the release. They should include:
- Linux binary
- GPG signature of the Linux binary
- SHA256 of binary
- Source code
- Wasm binaries of any runtimes
### Release notes
The release notes should list:
- The priority of the release (i.e., how quickly users should upgrade) - this is
based on the max priority of any *client* changes.
- Which native runtimes and their versions are included
- The proposal hashes of the runtimes as built with
[srtool](https://github.com/paritytech/srtool)
- Any changes in this release that are still awaiting audit
The release notes may also list:
- Free text at the beginning of the notes mentioning anything important
regarding this release
- Notable changes separated into sections.
### Spec Version
A runtime upgrade must bump the spec number. This may follow a pattern with the
client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
runtime is not v11).
### Runtime version bump between RCs
The clients need to be aware of runtime changes. However, we do not want to bump the
`spec_version` for every single release candidate. Instead, we can bump the `impl` field of the version
to signal the change to the client.
### Old Migrations Removed
Previous `on_runtime_upgrade` functions from old upgrades should be removed.
### New Migrations
Ensure that any migrations that are required due to storage or logic changes
are included in the `on_runtime_upgrade` function of the appropriate pallets.
### Extrinsic Ordering & Storage
Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the `module index, call index` tuples map to the same set of functions. It also checks if there have been any changes in `storage`. In case of a breaking change, increase `transaction_version`.
To verify the order has not changed, manually start the following [Github Action](https://github.com/paritytech/cumulus/actions/workflows/extrinsic-ordering-check-from-bin.yml). It takes around a minute to run and will produce the report as artifact you need to manually check.
To run it, in the _Run Workflow_ dropdown:
1. **Use workflow from**: to ignore, leave `master` as default
2. **The WebSocket url of the reference node**:
- Statemine: `wss://kusama-statemine-rpc.paritytech.net`
- Westmint: `wss://westmint-rpc.polkadot.io`
3. **A url to a Linux binary for the node containing the runtime to test**: Paste the URL of the latest release-candidate binary from the draft-release on Github. The binary has to previously be uploaded to S3 (Github url link to the binary is constantly changing)
- https://releases.parity.io/cumulus/statemine-v6.0.0-rc1/polkadot-collator
4. **The name of the chain under test. Usually, you would pass a local chain**:
- Statemine: `statemine-local`
- Westmint: `westmint-local`
5. Click **Run workflow**
When the workflow is done, click on it and download the zip artifact, inside you'll find an `output.txt` file. The things to look for in the output are lines like:
- `[Identity] idx 28 -> 25 (calls 15)` - indicates the index for Identity has changed
- `[+] Society, Recovery` - indicates the new version includes 2 additional modules/pallets.
- If no indices have changed, every modules line should look something like `[Identity] idx 25 (calls 15)`
**Note**: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.
**Note**: Extrinsic function signatures changes (adding/removing & ordering arguments) are not caught by the job, so those changes should be reviewed "manually"
### Proxy Filtering
The runtime contains proxy filters that map proxy types to allowable calls. If
the new runtime contains any new calls, verify that the proxy filters are up to
date to include them.
### Benchmarks
Until #631 is done, running the benchmarks is a manual process:
1. Connect to the bechmarking machine
2. Make sure no one else is using the machine with `htop check`
3. Pull in the branch of Cumulus that has the version of Statemine you want to release
4. Recompile `cargo build --profile production --locked --features runtime-benchmarks`
5. From the root directory run `nohup ./scripts/benchmarks.sh &` (it will take quite a few hours)
6. Checkout in your local machine to the branch of cumulus that has the version of Statemine you want to release
7. `scp` from the host to your local machine the weights for Statemine, Westmint and Statemint you'll find in:
- `/polkadot-parachains/statemine/src/weights`
- `/polkadot-parachains/westmint/src/weights`
- `/polkadot-parachains/statemint/src/weights`
8. Commit the changes in your local and create a PR
.github/dependabot.yml
+5
View File
@@ -23,3 +23,8 @@ updates:
- dependency-name: "westend-*"
schedule:
interval: "daily"
- package-ecosystem: github-actions
directory: '/'
labels: ["A2-insubstantial", "B0-silent", "C1-low 📌", "E3-dependencies"]
schedule:
interval: daily
.github/pr-custom-review.yml
+12 -2
View File
@@ -6,7 +6,7 @@ action-review-team: ci
rules:
- name: Runtime files
check_type: changed_files
condition: ^polkadot-parachains/(statemine|statemint)/src/[^/]+\.rs$
condition: ^parachains/runtimes/assets/(statemine|statemint)/src/[^/]+\.rs$
all_distinct:
- min_approvals: 1
teams:
@@ -19,11 +19,21 @@ rules:
check_type: changed_files
condition:
include: .*
exclude: ^polkadot-parachains/(statemine|statemint)/src/[^/]+\.rs$
# excluding files from 'Runtime files' and 'CI files' rules
exclude: ^parachains/runtimes/assets/(statemine|statemint)/src/[^/]+\.rs$|^\.gitlab-ci\.yml|^scripts/ci/.*|^\.github/.*
min_approvals: 2
teams:
- core-devs
- name: CI files
check_type: changed_files
condition:
include: ^\.gitlab-ci\.yml|^scripts/ci/.*|^\.github/.*
min_approvals: 2
teams:
- ci
- release-engineering
prevent-review-request:
teams:
- core-devs
.github/workflows/check-labels.yml
+1 -1
View File
@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.ref }}
.github/workflows/docs.yml
+1 -1
View File
@@ -12,7 +12,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v2.3.4
uses: actions/checkout@v3
- name: Rust versions
run: rustup show
.github/workflows/pr-custom-review.yml
+21 -2
View File
@@ -11,13 +11,32 @@ on:
- synchronize
- review_requested
- review_request_removed
- ready_for_review
- converted_to_draft
pull_request_review:
jobs:
pr-custom-review:
runs-on: ubuntu-latest
steps:
- name: Skip if pull request is in Draft
# `if: github.event.pull_request.draft == true` should be kept here, at
# the step level, rather than at the job level. The latter is not
# recommended because when the PR is moved from "Draft" to "Ready to
# review" the workflow will immediately be passing (since it was skipped),
# even though it hasn't actually ran, since it takes a few seconds for
# the workflow to start. This is also disclosed in:
# https://github.community/t/dont-run-actions-on-draft-pull-requests/16817/17
# That scenario would open an opportunity for the check to be bypassed:
# 1. Get your PR approved
# 2. Move it to Draft
# 3. Push whatever commits you want
# 4. Move it to "Ready for review"; now the workflow is passing (it was
# skipped) and "Check reviews" is also passing (it won't be updated
# until the workflow is finished)
if: github.event.pull_request.draft == true
run: exit 1
- name: pr-custom-review
uses: paritytech/pr-custom-review@v2
uses: paritytech/pr-custom-review@action-v3
with:
token: ${{ secrets.PRCR_TOKEN }}
checks-reviews-api: http://pcr.parity-prod.parity.io/api/v1/check_reviews
.github/workflows/quick-check.yml
+2 -2
View File
@@ -23,7 +23,7 @@ jobs:
components: clippy, rustfmt
- name: Cache Dependencies & Build Outputs
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: |
~/.cargo/registry
@@ -31,7 +31,7 @@ jobs:
target
key: ${{ runner.os }}-${{ matrix.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Cargo fmt
uses: actions-rs/cargo@v1
.github/workflows/release-01_branch-check.yml
+22
View File
@@ -0,0 +1,22 @@
name: Release branch check
on:
push:
branches:
- release-**v[0-9]+.[0-9]+.[0-9]+ # client
- release-**v[0-9]+ # runtimes
- polkadot-v[0-9]+.[0-9]+.[0-9]+ # cumulus code
workflow_dispatch:
jobs:
check_branch:
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Run check
shell: bash
run: ./scripts/ci/github/check-rel-br
.github/workflows/release-01_rc-automation.yml → .github/workflows/release-10_rc-automation.yml
+30 -12
View File
@@ -1,16 +1,17 @@
name: Release-candidate automation
name: Release - RC automation
on:
push:
branches:
- release-**v[0-9]+.[0-9]+.[0-9]+
- release-v[0-9]+.[0-9]+.[0-9]+
- release-parachains-v[0-9]+
workflow_dispatch:
jobs:
tag_rc:
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0
- id: compute_tag
@@ -34,30 +35,47 @@ jobs:
echo "::set-output name=new_tag::$version-rc1"
echo "::set-output name=first_rc::true"
fi
- name: Apply new tag
uses: tvdias/github-tagger@v0.0.2
uses: tvdias/github-tagger@ed7350546e3e503b5e942dffd65bc8751a95e49d # v0.0.2
with:
# We can't use the normal GITHUB_TOKEN for the following reason:
# https://docs.github.com/en/actions/reference/events-that-trigger-workflows#triggering-new-workflows-using-a-personal-access-token
# RELEASE_BRANCH_TOKEN requires public_repo OAuth scope
repo-token: "${{ secrets.RELEASE_BRANCH_TOKEN }}"
tag: ${{ steps.compute_tag.outputs.new_tag }}
- id: create-issue
uses: JasonEtco/create-an-issue@v2
- id: create-issue-checklist-client
uses: JasonEtco/create-an-issue@9e6213aec58987fa7d2f4deb8b256b99e63107a2 # v2.6.0
# Only create the issue if it's the first release candidate
if: steps.compute_tag.outputs.first_rc == 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.compute_tag.outputs.version }}
with:
filename: .github/ISSUE_TEMPLATE/release.md
assignees: release-engineering
filename: .github/ISSUE_TEMPLATE/release-client.md
- id: create-issue-checklist-runtime
uses: JasonEtco/create-an-issue@9e6213aec58987fa7d2f4deb8b256b99e63107a2 # v2.6.0
# Only create the issue if it's the first release candidate
if: steps.compute_tag.outputs.first_rc == 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.compute_tag.outputs.version }}
with:
assignees: release-engineering
filename: .github/ISSUE_TEMPLATE/release-runtime.md
- name: Send Matrix message
uses: s3krit/matrix-message-action@v0.0.3
if: steps.create-issue.outputs.url != ''
uses: s3krit/matrix-message-action@70ad3fb812ee0e45ff8999d6af11cafad11a6ecf # v0.0.3
if: steps.create-issue-checklist-client.outputs.url != '' && steps.create-issue-checklist-runtime.outputs.url != ''
with:
room_id: ${{ secrets.INTERNAL_CUMULUS_MATRIX_ROOM_ID }}
access_token: ${{ secrets.MATRIX_ACCESS_TOKEN }}
server: "matrix.parity.io"
message: |
Release process for Cumulus ${{ steps.compute_tag.outputs.version }} has been started.
Tracking issue: ${{ steps.create-issue.outputs.url }}"
The Release Process for Cumulus ${{ steps.compute_tag.outputs.version }} has been started.
Tracking issues:
- client: ${{ steps.create-issue-checklist-client.outputs.url }}"
- runtime: ${{ steps.create-issue-checklist-runtime.outputs.url }}"
.github/workflows/extrinsic-ordering-check-from-bin.yml → .github/workflows/release-20_extrinsic-ordering-check-from-bin.yml
+15 -11
View File
@@ -1,6 +1,6 @@
# This workflow performs the Extrinsic Ordering Check on demand using a binary
name: Extrinsic Ordering Check from Binary
name: Release - Extrinsic Ordering Check from Binary
on:
workflow_dispatch:
inputs:
@@ -10,7 +10,7 @@ on:
required: true
binary_url:
description: A url to a Linux binary for the node containing the runtime to test
default: https://github.com/paritytech/cumulus/releases/download/statemine_v3/polkadot-collator
default: https://releases.parity.io/cumulus/polkadot-v0.9.21/polkadot-parachain
required: true
chain:
description: The name of the chain under test. Usually, you would pass a local chain
@@ -23,27 +23,31 @@ jobs:
runs-on: ubuntu-latest
env:
CHAIN: ${{github.event.inputs.chain}}
BIN: node-bin
BIN_PATH: ./tmp/$BIN
BIN_URL: ${{github.event.inputs.binary_url}}
REF_URL: ${{github.event.inputs.reference_url}}
steps:
- uses: actions/checkout@v2
steps:
- uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b # v3.0.2
- name: Fetch binary
run: |
echo Creating a temp dir to download and run binary
mkdir -p tmp
echo Fetching $BIN_URL
wget $BIN_URL
chmod a+x polkadot-collator
./polkadot-collator --version
wget $BIN_URL -O $BIN_PATH
chmod a+x $BIN_PATH
$BIN_PATH --version
- name: Start local node
run: |
echo Running on $CHAIN
./polkadot-collator --chain=$CHAIN -- --chain polkadot-local &
$BIN_PATH --chain=$CHAIN -- --chain polkadot-local &
- name: Prepare output
run: |
VERSION=$(./polkadot-collator --version)
VERSION=$($BIN_PATH --version)
echo "Metadata comparison:" >> output.txt
echo "Date: $(date)" >> output.txt
echo "Reference: $REF_URL" >> output.txt
@@ -71,11 +75,11 @@ jobs:
- name: Stop our local node
run: |
pkill polkadot-collator
pkill $BIN
continue-on-error: true
- name: Save output as artifact
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: ${{ env.CHAIN }}
path: |
.github/workflows/release-21_extrinsic-ordering-check-from-two.yml
+120
View File
@@ -0,0 +1,120 @@
# This workflow performs the Extrinsic Ordering Check on demand using two reference binaries
name: Release - Extrinsic API Check with reference bins
on:
workflow_dispatch:
inputs:
reference_binary_url:
description: A url to a Linux binary for the node containing the reference runtime to test against
default: https://releases.parity.io/cumulus/v0.9.230/polkadot-parachain
required: true
binary_url:
description: A url to a Linux binary for the node containing the runtime to test
default: https://releases.parity.io/cumulus/v0.9.270-rc7/polkadot-parachain
required: true
jobs:
check:
name: Run check
runs-on: ubuntu-latest
timeout-minutes: 10
env:
REF_URL: ${{github.event.inputs.reference_binary_url}}
BIN_REF: polkadot-parachain-ref
BIN_URL: ${{github.event.inputs.binary_url}}
BIN_BASE: polkadot-parachain
TMP: ./tmp
strategy:
fail-fast: false
matrix:
include:
- runtime: statemine
local: statemine-local
relay: polkadot-local
- runtime: statemint
local: statemint-local
relay: polkadot-local
- runtime: westmint
local: westmint-local
relay: polkadot-local
- runtime: contracts-rococo
local: contracts-rococo-local
relay: polkadot-local
steps:
- uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b # v3.0.2
- name: Create tmp dir
run: |
mkdir -p $TMP
pwd
- name: Fetch reference binary for ${{ matrix.runtime }}
run: |
echo Fetching $REF_URL
curl $REF_URL -o $TMP/$BIN_REF
chmod a+x $TMP/$BIN_REF
$TMP/$BIN_REF --version
- name: Fetch test binary for ${{ matrix.runtime }}
run: |
echo Fetching $BIN_URL
curl $BIN_URL -o $TMP/$BIN_BASE
chmod a+x $TMP/$BIN_BASE
$TMP/$BIN_BASE --version
- name: Start local reference node for ${{ matrix.runtime }}
run: |
echo Running reference on ${{ matrix.local }}
$TMP/$BIN_REF --chain=${{ matrix.local }} --ws-port=9954 --tmp -- --chain ${{ matrix.relay }} &
sleep 15
- name: Start local test node for ${{ matrix.runtime }}
run: |
echo Running test on ${{ matrix.local }}
$TMP/$BIN_BASE --chain=${{ matrix.local }} --ws-port=9944 --tmp -- --chain ${{ matrix.relay }} &
sleep 15
- name: Prepare output
run: |
REF_VERSION=$($TMP/$BIN_REF --version)
BIN_VERSION=$($TMP/$BIN_BASE --version)
echo "Metadata comparison:" >> output.txt
echo "Date: $(date)" >> output.txt
echo "Ref. binary: $REF_URL" >> output.txt
echo "Test binary: $BIN_URL" >> output.txt
echo "Ref. version: $REF_VERSION" >> output.txt
echo "Test version: $BIN_VERSION" >> output.txt
echo "Chain: ${{ matrix.local }}" >> output.txt
echo "Relay: ${{ matrix.relay }}" >> output.txt
echo "----------------------------------------------------------------------" >> output.txt
- name: Pull polkadot-js-tools image
run: docker pull jacogr/polkadot-js-tools
- name: Compare the metadata
run: |
CMD="docker run --pull always --network host jacogr/polkadot-js-tools metadata ws://localhost:9954 ws://localhost:9944"
echo -e "Running:\n$CMD"
$CMD >> output.txt
sed -z -i 's/\n\n/\n/g' output.txt
cat output.txt | egrep -n -i ''
SUMMARY=$(./scripts/ci/github/extrinsic-ordering-filter.sh output.txt)
echo -e $SUMMARY
echo -e $SUMMARY >> output.txt
- name: Show result
run: |
cat output.txt
- name: Save output as artifact
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.runtime }}
path: |
output.txt
- name: Stop our local nodes
run: |
pkill $BIN_REF || true
pkill $BIN_BASE || true
.github/workflows/release-02_create-draft.yml → .github/workflows/release-30_create-draft.yml
+52 -19
View File
@@ -5,11 +5,11 @@ on:
inputs:
ref1:
description: The 'from' tag to use for the diff
default: parachains-v7.0.0
default: parachains-v9.0.0
required: true
ref2:
description: The 'to' tag to use for the diff
default: release-parachains-v8.0.0
default: release-parachains-v10.0.0
required: true
release_type:
description: Pass "client" for client releases, leave empty otherwise
@@ -42,8 +42,24 @@ jobs:
build-runtimes:
runs-on: ubuntu-latest
strategy:
matrix:
runtime: ["shell", "statemine", "statemint", "westmint", "rococo-parachain", "canvas-kusama"]
matrix:
include:
- category: assets
runtime: statemine
- category: assets
runtime: statemint
- category: assets
runtime: westmint
- category: collectives
runtime: collectives-polkadot
- category: contracts
runtime: contracts-rococo
- category: starters
runtime: seedling
- category: starters
runtime: shell
- category: testing
runtime: rococo-parachain
steps:
- name: Checkout sources
uses: actions/checkout@v3
@@ -52,7 +68,7 @@ jobs:
- name: Cache target dir
if: ${{ github.event.inputs.release_type != 'client' }}
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: "${{ github.workspace }}/runtime/${{ matrix.runtime }}/target"
key: srtool-target-${{ matrix.runtime }}-${{ github.sha }}
@@ -63,11 +79,11 @@ jobs:
- name: Build ${{ matrix.runtime }} runtime
if: ${{ github.event.inputs.release_type != 'client' }}
id: srtool_build
uses: chevdor/srtool-actions@v0.3.0
uses: chevdor/srtool-actions@v0.5.0
with:
image: paritytech/srtool
chain: ${{ matrix.runtime }}
runtime_dir: polkadot-parachains/${{ matrix.runtime }}
runtime_dir: parachains/runtimes/${{ matrix.category }}/${{ matrix.runtime }}
- name: Store srtool digest to disk
if: ${{ github.event.inputs.release_type != 'client' }}
@@ -109,7 +125,7 @@ jobs:
ruby-version: 3.0.0
- name: Download srtool json output
uses: actions/download-artifact@v2
uses: actions/download-artifact@v3
- name: Prepare tooling
run: |
@@ -134,8 +150,9 @@ jobs:
WESTMINT_DIGEST: ${{ github.workspace}}/westmint-srtool-json/westmint-srtool-digest.json
STATEMINE_DIGEST: ${{ github.workspace}}/statemine-srtool-json/statemine-srtool-digest.json
STATEMINT_DIGEST: ${{ github.workspace}}/statemint-srtool-json/statemint-srtool-digest.json
COLLECTIVES_POLKADOT_DIGEST: ${{ github.workspace}}/collectives-polkadot-srtool-json/collectives-polkadot-srtool-digest.json
ROCOCO_PARA_DIGEST: ${{ github.workspace}}/rococo-parachain-srtool-json/rococo-parachain-srtool-digest.json
CANVAS_KUSAMA_DIGEST: ${{ github.workspace}}/canvas-kusama-srtool-json/canvas-kusama-srtool-digest.json
CANVAS_KUSAMA_DIGEST: ${{ github.workspace}}/contracts-rococo-srtool-json/contracts-rococo-srtool-digest.json
REF1: ${{ github.event.inputs.ref1 }}
REF2: ${{ github.event.inputs.ref2 }}
PRE_RELEASE: ${{ github.event.inputs.pre_release }}
@@ -143,11 +160,12 @@ jobs:
run: |
find ${{env.GITHUB_WORKSPACE}} -type f -name "*-srtool-digest.json"
if [ "$RELEASE_TYPE" == "client" ]; then
if [ "$RELEASE_TYPE" != "client" ]; then
ls -al $SHELL_DIGEST || true
ls -al $WESTMINT_DIGEST || true
ls -al $STATEMINE_DIGEST || true
ls -al $STATEMINT_DIGEST || true
ls -al $COLLECTIVES_POLKADOT_DIGEST || true
ls -al $ROCOCO_PARA_DIGEST || true
ls -al $CANVAS_KUSAMA_DIGEST || true
fi
@@ -170,7 +188,7 @@ jobs:
with:
name: release-notes-context
path: |
context.json
cumulus/scripts/ci/changelog/context.json
- name: Create draft release
id: create-release
@@ -188,10 +206,26 @@ jobs:
runs-on: ubuntu-latest
needs: ["publish-draft-release"]
env:
RUNTIME_DIR: polkadot-parachains
RUNTIME_DIR: parachains/runtimes
strategy:
matrix:
runtime: ["shell", "statemine", "statemint", "westmint", "rococo-parachain", "canvas-kusama"]
matrix:
include:
- category: assets
runtime: statemine
- category: assets
runtime: statemint
- category: assets
runtime: westmint
- category: collectives
runtime: collectives-polkadot
- category: contracts
runtime: contracts-rococo
- category: starters
runtime: seedling
- category: starters
runtime: shell
- category: testing
runtime: rococo-parachain
steps:
- name: Checkout sources
uses: actions/checkout@v3
@@ -199,7 +233,7 @@ jobs:
ref: ${{ github.event.inputs.ref2 }}
- name: Download artifacts
uses: actions/download-artifact@v2
uses: actions/download-artifact@v3
- uses: ruby/setup-ruby@v1
with:
@@ -209,10 +243,10 @@ jobs:
id: get-runtime-ver
run: |
echo "require './scripts/ci/github/runtime-version.rb'" > script.rb
echo "puts get_runtime(runtime: \"${{ matrix.runtime }}\", runtime_dir: \"$RUNTIME_DIR\")" >> script.rb
echo "puts get_runtime(runtime: \"${{ matrix.runtime }}\", runtime_dir: \"$RUNTIME_DIR/${{ matrix.category }}\")" >> script.rb
echo "Current folder: $PWD"
ls "$RUNTIME_DIR/${{ matrix.runtime }}"
ls "$RUNTIME_DIR/${{ matrix.category }}/${{ matrix.runtime }}"
runtime_ver=$(ruby script.rb)
echo "Found version: >$runtime_ver<"
echo "::set-output name=runtime_ver::$runtime_ver"
@@ -221,8 +255,7 @@ jobs:
id: fix-runtime-path
run: |
cd "${{ matrix.runtime }}-runtime/"
mv "$(sed 's/-parachain/_parachain/' <<< ${{ matrix.runtime }})_runtime.compact.compressed.wasm" "${{ matrix.runtime }}_runtime.compact.compressed.wasm" || true
mv "$(sed 's/-kusama/_kusama/' <<< ${{ matrix.runtime }})_runtime.compact.compressed.wasm" "${{ matrix.runtime }}_runtime.compact.compressed.wasm" || true
mv "$(sed -E 's/-(.*)/_\1/' <<< ${{ matrix.runtime }})_runtime.compact.compressed.wasm" "${{ matrix.runtime }}_runtime.compact.compressed.wasm" || true
- name: Upload compressed ${{ matrix.runtime }} wasm
uses: actions/upload-release-asset@v1
.github/workflows/release-10_docker-manual.yml → .github/workflows/release-50_docker-manual.yml
+18 -7
View File
@@ -9,27 +9,37 @@ on:
inputs:
tag:
description: release tag to build image for
default: polkadot-v0.9.17
default: v0.9.230
required: true
prerelease:
description: is prerelease
default: false
default: "false"
required: true
jobs:
docker_build_publish:
env:
BINARY: polkadot-collator
BINARY: polkadot-parachain
TMP: tmp
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: ${{ github.event.release.tag_name }}
- name: Prepare temp folder
run: |
mkdir ${TMP}
ls -al
- name: Fetch files from release
working-directory: ${{ env.TMP }}
run: |
echo "Repo: ${{ github.event.repository.full_name }}"
echo "Working from folder $(pwd)"
ls -al
for f in $BINARY $BINARY.asc $BINARY.sha256; do
URL="https://github.com/${{ github.event.repository.full_name }}/releases/download/${{ github.event.inputs.tag }}/$f"
@@ -40,8 +50,9 @@ jobs:
ls -al
- name: Check files
working-directory: ${{ env.TMP }}
run: |
ls -al *collator*
ls -al *$BINARY*
shasum -a 256 -c $BINARY.sha256
sha_result=$?
@@ -76,11 +87,11 @@ jobs:
run: |
export OWNER=$DOCKERHUB_ORG
mkdir -p target/release
cp -f $BINARY* target/release/
cp -f ${TMP}/$BINARY* target/release/
./docker/scripts/build-injected-image.sh
- name: Login to Dockerhub
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
.github/workflows/release-10_docker.yml → .github/workflows/release-50_docker.yml
+14 -5
View File
@@ -14,15 +14,23 @@ on:
jobs:
docker_build_publish:
env:
BINARY: polkadot-collator
BINARY: polkadot-parachain
TMP: tmp
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
ref: ${{ github.event.release.tag_name }}
- name: Prepare temp folder
run: |
mkdir ${TMP}
ls -al
- name: Fetch files from release
working-directory: ${{ env.TMP }}
run: |
echo "Repo: ${{ github.event.repository.full_name }}"
@@ -41,8 +49,9 @@ jobs:
ls -al
- name: Check files
working-directory: ${{ env.TMP }}
run: |
ls -al *collator*
ls -al *$BINARY*
shasum -a 256 -c $BINARY.sha256
sha_result=$?
@@ -77,11 +86,11 @@ jobs:
run: |
export OWNER=$DOCKERHUB_ORG
mkdir -p target/release
cp -f $BINARY* target/release/
cp -f ${TMP}/$BINARY* target/release/
./docker/scripts/build-injected-image.sh
- name: Login to Dockerhub
uses: docker/login-action@v1
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
.github/workflows/release-99_bot-announce.yml
+12 -2
View File
@@ -7,12 +7,23 @@ on:
jobs:
ping_matrix:
runs-on: ubuntu-latest
strategy:
matrix:
channel:
- name: 'Cumulus Release Coordination'
room: '!ZrLPsivsytpkdJfVaa:matrix.parity.io'
pre-releases: true
- name: 'Ledger <> Polkadot Coordination'
room: '!EoIhaKfGPmFOBrNSHT:web3.foundation'
pre-release: true
steps:
- name: send message
uses: s3krit/matrix-message-action@v0.0.3
with:
room_id: ${{ secrets.MATRIX_ROOM_ID }}
room_id: ${{ matrix.channel.room }}
access_token: ${{ secrets.MATRIX_ACCESS_TOKEN }}
server: "matrix.parity.io"
message: |
A (pre)release has been ${{github.event.action}} in **${{github.event.repository.full_name}}:**<br/>
Release version: [${{github.event.release.tag_name}}](${{github.event.release.html_url}})
@@ -20,4 +31,3 @@ jobs:
-----
${{github.event.release.body}}
server: "matrix.parity.io"
.github/workflows/srtool.yml
+48 -30
View File
@@ -1,22 +1,24 @@
name: Srtool build
env:
SUBWASM_VERSION: 0.15.0
SUBWASM_VERSION: 0.17.0
on:
push:
tags:
- "*"
paths-ignore:
- "docker"
- "docs"
- "scripts"
- "test"
# paths-ignore:
# - "docker"
# - "docs"
# - "scripts"
# - "test"
# - "client"
paths:
- parachains/runtimes/**/*
branches:
- "release*"
- "master"
schedule:
- cron: "00 02 * * 1" # 2AM weekly on monday
@@ -28,35 +30,51 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
chain: ["statemine", "westmint", "statemint", "rococo-parachain", "shell", "canvas-kusama"]
include:
- category: assets
runtime: statemine
- category: assets
runtime: statemint
- category: assets
runtime: westmint
- category: collectives
runtime: collectives-polkadot
- category: contracts
runtime: contracts-rococo
- category: starters
runtime: seedling
- category: starters
runtime: shell
- category: testing
runtime: rococo-parachain
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Srtool build
id: srtool_build
uses: chevdor/srtool-actions@v0.3.0
uses: chevdor/srtool-actions@v0.5.0
with:
chain: ${{ matrix.chain }}
runtime_dir: polkadot-parachains/${{ matrix.chain }}
chain: ${{ matrix.runtime }}
runtime_dir: parachains/runtimes/${{ matrix.category }}/${{ matrix.runtime }}
- name: Summary
run: |
echo '${{ steps.srtool_build.outputs.json }}' | jq > ${{ matrix.chain }}-srtool-digest.json
cat ${{ matrix.chain }}-srtool-digest.json
echo '${{ steps.srtool_build.outputs.json }}' | jq > ${{ matrix.runtime }}-srtool-digest.json
cat ${{ matrix.runtime }}-srtool-digest.json
echo "Compact Runtime: ${{ steps.srtool_build.outputs.wasm }}"
echo "Compressed Runtime: ${{ steps.srtool_build.outputs.wasm_compressed }}"
# it takes a while to build the runtime, so let's save the artifact as soon as we have it
- name: Archive Artifacts for ${{ matrix.chain }}
uses: actions/upload-artifact@v2
- name: Archive Artifacts for ${{ matrix.runtime }}
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.chain }}-runtime
name: ${{ matrix.runtime }}-runtime
path: |
${{ steps.srtool_build.outputs.wasm }}
${{ steps.srtool_build.outputs.wasm_compressed }}
${{ matrix.chain }}-srtool-digest.json
${{ matrix.runtime }}-srtool-digest.json
# We now get extra information thanks to subwasm
- name: Install subwasm
@@ -70,29 +88,29 @@ jobs:
run: |
subwasm info ${{ steps.srtool_build.outputs.wasm }}
subwasm info ${{ steps.srtool_build.outputs.wasm_compressed }}
subwasm --json info ${{ steps.srtool_build.outputs.wasm }} > ${{ matrix.chain }}-info.json
subwasm --json info ${{ steps.srtool_build.outputs.wasm_compressed }} > ${{ matrix.chain }}-compressed-info.json
subwasm --json info ${{ steps.srtool_build.outputs.wasm }} > ${{ matrix.runtime }}-info.json
subwasm --json info ${{ steps.srtool_build.outputs.wasm_compressed }} > ${{ matrix.runtime }}-compressed-info.json
- name: Extract the metadata
shell: bash
run: |
subwasm meta ${{ steps.srtool_build.outputs.wasm }}
subwasm --json meta ${{ steps.srtool_build.outputs.wasm }} > ${{ matrix.chain }}-metadata.json
subwasm --json meta ${{ steps.srtool_build.outputs.wasm }} > ${{ matrix.runtime }}-metadata.json
- name: Check the metadata diff
shell: bash
# the following subwasm call will error for chains that are not known and/or live, that includes shell for instance
run: |
subwasm diff ${{ steps.srtool_build.outputs.wasm }} --chain-b ${{ matrix.chain }} || \
echo "Subwasm call failed, check the logs. This is likely because ${{ matrix.chain }} is not known by subwasm" | \
tee ${{ matrix.chain }}-diff.txt
subwasm diff ${{ steps.srtool_build.outputs.wasm }} --chain-b ${{ matrix.runtime }} || \
echo "Subwasm call failed, check the logs. This is likely because ${{ matrix.runtime }} is not known by subwasm" | \
tee ${{ matrix.runtime }}-diff.txt
- name: Archive Subwasm results
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.chain }}-runtime
name: ${{ matrix.runtime }}-runtime
path: |
${{ matrix.chain }}-info.json
${{ matrix.chain }}-compressed-info.json
${{ matrix.chain }}-metadata.json
${{ matrix.chain }}-diff.txt
${{ matrix.runtime }}-info.json
${{ matrix.runtime }}-compressed-info.json
${{ matrix.runtime }}-metadata.json
${{ matrix.runtime }}-diff.txt
.gitignore
+2
View File
@@ -8,3 +8,5 @@ polkadot_argument_parsing
**/chains/
*.iml
.env
bin
**/._*
.gitlab-ci.yml
+266 -83
View File
@@ -7,9 +7,10 @@
stages:
- test
- publish
- build
- benchmarks-build
- benchmarks-run
- publish
default:
interruptible: true
@@ -43,30 +44,45 @@ variables:
- cargo +nightly --version
- bash --version
# collecting vars for pipeline stopper
# they will be used if the job fails
.pipeline-stopper-vars: &pipeline-stopper-vars
- echo "FAILED_JOB_URL=${CI_JOB_URL}" > pipeline-stopper.env
- echo "FAILED_JOB_NAME=${CI_JOB_NAME}" >> pipeline-stopper.env
- echo "FAILED_JOB_NAME=${CI_JOB_NAME}" >> pipeline-stopper.env
- echo "PR_NUM=${CI_COMMIT_REF_NAME}" >> pipeline-stopper.env
.pipeline-stopper-artifacts: &pipeline-stopper-artifacts
artifacts:
reports:
dotenv: pipeline-stopper.env
.common-refs: &common-refs
# these jobs run always*
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
- if: $CI_COMMIT_REF_NAME == "master"
- if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/ # PRs
- if: $CI_COMMIT_REF_NAME =~ /^v[0-9]+\.[0-9]+.*$/ # i.e. v1.0, v2.1rc1
- if: $CI_COMMIT_REF_NAME =~ /^release-parachains-v[0-9].*$/ # i.e. release-parachains-v1.0, release-parachains-v2.1rc1, release-parachains-v3000
- if: $CI_COMMIT_REF_NAME =~ /^polkadot-v[0-9]+\.[0-9]+.*$/ # i.e. polkadot-v1.0.99, polkadot-v2.1rc1
.publish-refs: &publish-refs
rules:
- if: $CI_PIPELINE_SOURCE == "web" &&
$CI_COMMIT_REF_NAME == "master" # run from web and on master branch
- if: $CI_PIPELINE_SOURCE == "web" &&
$CI_COMMIT_REF_NAME =~ /^v[0-9]+\.[0-9]+.*$/ # run from web and on version tag (i.e. v1.0, v2.1rc1)
- if: $CI_COMMIT_REF_NAME == "master"
- if: $CI_PIPELINE_SOURCE == "schedule"
- if: $CI_COMMIT_REF_NAME =~ /^v[0-9]+\.[0-9]+.*$/ # i.e. v1.0, v2.1rc1
# run benchmarks manually only on release-parachains-v* branch
.benchmarks-manual-refs: &benchmarks-manual-refs
rules:
- if: $CI_PIPELINE_SOURCE == "web" &&
$CI_COMMIT_REF_NAME =~ /^release-parachains-v[0-9]+\.[0-9]+.*$/ # run from web and on branch release-parachains-v* (i.e. 1.0, 2.1rc1)
when: manual
- if: $CI_COMMIT_REF_NAME =~ /^release-parachains-v[0-9]+\.[0-9]+.*$/ # i.e. release-parachains-v1.0, release-parachains-v2.1rc1
- if: $CI_COMMIT_REF_NAME =~ /^release-parachains-v[0-9].*$/ # i.e. release-parachains-v1.0, release-parachains-v2.1rc1, release-parachains-v3000
when: manual
# run benchmarks only on release-parachains-v* branch
.benchmarks-refs: &benchmarks-refs
rules:
- if: $CI_PIPELINE_SOURCE == "web" &&
$CI_COMMIT_REF_NAME =~ /^release-parachains-v[0-9]+\.[0-9]+.*$/ # run from web and on branch release-parachains-v* (i.e. 1.0, 2.1rc1)
- if: $CI_COMMIT_REF_NAME =~ /^release-parachains-v[0-9]+\.[0-9]+.*$/ # i.e. release-parachains-v1.0, release-parachains-v2.1rc1
- if: $CI_COMMIT_REF_NAME =~ /^release-parachains-v[0-9].*$/ # i.e. release-parachains-v1.0, release-parachains-v2.1rc1, release-parachains-v3000
.docker-env: &docker-env
image: "${CI_IMAGE}"
@@ -80,107 +96,186 @@ variables:
tags:
- kubernetes-parity-build
.collect-artifacts: &collect-artifacts
artifacts:
name: "${CI_JOB_NAME}_${CI_COMMIT_REF_NAME}"
when: on_success
expire_in: 28 days
paths:
- ./artifacts/
#### stage: test
test-linux-stable:
stage: test
<<: *docker-env
rules:
- if: $CI_COMMIT_REF_NAME == "master"
- if: $CI_COMMIT_REF_NAME == "tags"
- if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/ # PRs
# It doesn't make sense to build on every commit, so we build on tags
- if: $CI_COMMIT_REF_NAME =~ /^v[0-9]+\.[0-9]+.*$/ # i.e. v1.0, v2.1rc1
variables:
ARE_WE_RELEASING_YET: maybe!
# web and schedule triggers can be provided with the non-empty variable ARE_WE_RELEASING_YET
# to run building and publishing the binary.
- if: $CI_PIPELINE_SOURCE == "web"
- if: $CI_PIPELINE_SOURCE == "schedule"
<<: *collect-artifacts
<<: *common-refs
<<: *pipeline-stopper-artifacts
before_script:
- *rust-info-script
- *pipeline-stopper-vars
variables:
# Enable debug assertions since we are running optimized builds for testing
# but still want to have debug assertions.
RUSTFLAGS: "-Cdebug-assertions=y -Dwarnings"
script:
- time cargo test --all --release --locked -- --include-ignored
# It's almost free to produce a binary here, please refrain from using it in production since
# it goes with the debug assertions.
- if [ "${ARE_WE_RELEASING_YET}" ]; then
echo "___Building a binary___";
time cargo build --release --locked --bin polkadot-collator;
echo "___Packing the artifacts___";
mkdir -p ./artifacts;
mv ${CARGO_TARGET_DIR}/release/polkadot-collator ./artifacts/.;
echo "___The VERSION is either a tag name or the curent branch if triggered not by a tag___";
echo ${CI_COMMIT_REF_NAME} | tee ./artifacts/VERSION;
else
exit 0;
fi
- sccache -s
- time cargo nextest run --all --release --locked --run-ignored all
test-doc:
stage: test
<<: *docker-env
<<: *common-refs
variables:
# Enable debug assertions since we are running optimized builds for testing
# but still want to have debug assertions.
RUSTFLAGS: "-Cdebug-assertions=y -Dwarnings"
script:
- time cargo test --doc
check-runtime-benchmarks:
stage: test
<<: *docker-env
<<: *common-refs
script:
# Check that the node will compile with `runtime-benchmarks` feature flag.
- time cargo check --all --features runtime-benchmarks
# Check that parachain-template will compile with `runtime-benchmarks` feature flag.
- time cargo check -p parachain-template-node --features runtime-benchmarks
- sccache -s
cargo-check-try-runtime:
stage: test
<<: *docker-env
<<: *common-refs
# this is an artificial job dependency, for pipeline optimization using GitLab's DAGs
needs:
- job: check-runtime-benchmarks
artifacts: false
script:
# Check that the node will compile with `try-runtime` feature flag.
- time cargo check --all --features try-runtime
# Check that parachain-template will compile with `try-runtime` feature flag.
- time cargo check -p parachain-template-node --features try-runtime
- sccache -s
cargo-check-benches:
stage: test
<<: *docker-env
script:
- time cargo check --all --benches
- sccache -s
check-rustdoc:
stage: test
<<: *docker-env
<<: *common-refs
variables:
SKIP_WASM_BUILD: 1
RUSTDOCFLAGS: "-Dwarnings"
script:
- time cargo +nightly doc --workspace --all-features --verbose --no-deps
- sccache -s
cargo-check-benches:
stage: test
<<: *docker-env
<<: *common-refs
# this is an artificial job dependency, for pipeline optimization using GitLab's DAGs
needs:
- job: check-rustdoc
artifacts: false
script:
- time cargo check --all --benches
#### stage: build
build-linux-stable:
stage: build
<<: *docker-env
<<: *collect-artifacts
<<: *common-refs
variables:
# Enable debug assertions since we are running optimized builds for testing
# but still want to have debug assertions.
RUSTFLAGS: "-Cdebug-assertions=y -Dwarnings"
# this is an artificial job dependency, for pipeline optimization using GitLab's DAGs
needs:
- job: check-rustdoc
artifacts: false
script:
- echo "___Building a binary, please refrain from using it in production since it goes with the debug assertions.___"
- time cargo build --release --locked --bin polkadot-parachain
- echo "___Packing the artifacts___"
- mkdir -p ./artifacts
- mv ./target/release/polkadot-parachain ./artifacts/.
- echo "___The VERSION is either a tag name or the curent branch if triggered not by a tag___"
- echo ${CI_COMMIT_REF_NAME} | tee ./artifacts/VERSION
build-test-parachain:
stage: build
<<: *docker-env
<<: *collect-artifacts
<<: *common-refs
variables:
# Enable debug assertions since we are running optimized builds for testing
# but still want to have debug assertions.
RUSTFLAGS: "-Cdebug-assertions=y -Dwarnings"
# this is an artificial job dependency, for pipeline optimization using GitLab's DAGs
needs:
- job: check-rustdoc
artifacts: false
script:
- echo "___Building a binary, please refrain from using it in production since it goes with the debug assertions.___"
- time cargo +nightly build --release --locked --bin test-parachain
- echo "___Packing the artifacts___"
- mkdir -p ./artifacts
- mv ./target/release/test-parachain ./artifacts/.
#### stage: publish
.build-push-image: &build-push-image
image: quay.io/buildah/stable
variables:
DOCKERFILE: "" # docker/path-to.Dockerfile
IMAGE_NAME: "" # docker.io/paritypr/image_name
VERSION: "${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}"
script:
- test "$PARITYPR_USER" -a "$PARITYPR_PASS" ||
( echo "no docker credentials provided"; exit 1 )
- buildah bud
--format=docker
--build-arg VCS_REF="${CI_COMMIT_SHA}"
--build-arg BUILD_DATE="$(date -u '+%Y-%m-%dT%H:%M:%SZ')"
--build-arg IMAGE_NAME="${IMAGE_NAME}"
--tag "$IMAGE_NAME:$VERSION"
--file ${DOCKERFILE} .
- echo "$PARITYPR_PASS" |
buildah login --username "$PARITYPR_USER" --password-stdin docker.io
- buildah info
- buildah push --format=v2s2 "$IMAGE_NAME:$VERSION"
after_script:
- buildah logout --all
build-push-image-polkadot-parachain-debug:
stage: publish
<<: *kubernetes-env
<<: *common-refs
<<: *build-push-image
needs:
- job: build-linux-stable
artifacts: true
variables:
DOCKERFILE: "docker/polkadot-parachain-debug_unsigned_injected.Dockerfile"
IMAGE_NAME: "docker.io/paritypr/polkadot-parachain-debug"
VERSION: "${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}"
build-push-image-test-parachain:
stage: publish
<<: *kubernetes-env
<<: *common-refs
<<: *build-push-image
needs:
- job: build-test-parachain
artifacts: true
variables:
DOCKERFILE: "docker/test-parachain_injected.Dockerfile"
IMAGE_NAME: "docker.io/paritypr/test-parachain"
VERSION: "${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}"
publish-s3:
stage: publish
<<: *kubernetes-env
image: paritytech/awscli:latest
<<: *publish-refs
needs:
- job: build-linux-stable
artifacts: true
variables:
GIT_STRATEGY: none
BUCKET: "releases.parity.io"
PREFIX: "cumulus/${ARCH}-${DOCKER_OS}"
before_script:
# Job will fail if no artifacts were provided by test-linux-stable job. It's only possible for
# this test to fail if the pipeline was triggered by web or schedule trigger without supplying
# a nono-empty ARE_WE_RELEASING_YET variable.
- test -e ./artifacts/polkadot-collator ||
( echo "___No artifacts were provided by the previous job, please check the build there___"; exit 1 )
script:
- echo "___Publishing a binary with debug assertions!___"
- echo "___VERSION = $(cat ./artifacts/VERSION) ___"
@@ -201,37 +296,125 @@ benchmarks-build:
<<: *benchmarks-manual-refs
script:
- time cargo build --profile production --locked --features runtime-benchmarks
- mkdir artifacts
- cp $CARGO_TARGET_DIR/production/polkadot-collator ./artifacts/
- mkdir -p artifacts
- cp target/production/polkadot-parachain ./artifacts/
benchmarks:
.git-commit-push: &git-commit-push
- git status
# Set git config
- rm -rf .git/config
- git config --global user.email "${GITHUB_EMAIL}"
- git config --global user.name "${GITHUB_USER}"
- git config remote.origin.url "https://${GITHUB_USER}:${GITHUB_TOKEN}@github.com/paritytech/${CI_PROJECT_NAME}.git"
- git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
# push results to github
- git checkout -b $BRANCHNAME
- git add parachains/*
- git commit -m "[benchmarks] pr with weights"
- git push origin $BRANCHNAME
benchmarks-statemint:
stage: benchmarks-run
before_script:
- *rust-info-script
<<: *collect-artifacts
<<: *benchmarks-refs
timeout: 1d
script:
- ./scripts/benchmarks-ci.sh assets statemine ./artifacts
- ./scripts/benchmarks-ci.sh assets statemint ./artifacts
- ./scripts/benchmarks-ci.sh assets westmint ./artifacts
- export BRANCHNAME="weights-statemint-${CI_COMMIT_BRANCH}"
- *git-commit-push
# create PR
- curl -u ${GITHUB_USER}:${GITHUB_TOKEN}
-d '{"title":"[benchmarks] Update weights for statemine/t","body":"This PR is generated automatically by CI.","head":"'$BRANCHNAME'","base":"'${CI_COMMIT_BRANCH}'"}'
-X POST https://api.github.com/repos/paritytech/${CI_PROJECT_NAME}/pulls
after_script:
- rm -rf .git/config
tags:
- weights
benchmarks-collectives:
stage: benchmarks-run
before_script:
- *rust-info-script
<<: *collect-artifacts
<<: *benchmarks-refs
script:
- ./scripts/benchmarks-ci.sh statemine > ./artifacts/bench-statemine.log
- ./scripts/benchmarks-ci.sh statemint > ./artifacts/bench-statemint.log
- ./scripts/benchmarks-ci.sh westmint > ./artifacts/bench-westmint.log
- ./scripts/benchmarks-ci.sh collectives collectives-polkadot ./artifacts
- git status
- export BRANCHNAME="${CI_COMMIT_BRANCH}-weights"
# Set git config
- rm -rf .git/config
- git config --global user.email "${GITHUB_EMAIL}"
- git config --global user.name "${GITHUB_USER}"
- git config remote.origin.url "https://${GITHUB_USER}:${GITHUB_TOKEN}@github.com:/paritytech/${CI_PROJECT_NAME}.git"
- git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
# push results to github
- git checkout -b $BRANCHNAME
- git add polkadot-parachains/*
- git commit -m "[benchmarks] pr with wieghts"
- git push origin $BRANCHNAME
- export BRANCHNAME="weights-collectives-${CI_COMMIT_BRANCH}"
- *git-commit-push
# create PR
- curl -u ${GITHUB_USER}:${GITHUB_TOKEN}
-d '{"title":"[benchmarks] Update weights","body":"This PR is generated automatically by CI.","head":"'$BRANCHNAME'","base":"'${CI_COMMIT_BRANCH}'"}'
-d '{"title":"[benchmarks] Update weights for collectives","body":"This PR is generated automatically by CI.","head":"'$BRANCHNAME'","base":"'${CI_COMMIT_BRANCH}'"}'
-X POST https://api.github.com/repos/paritytech/${CI_PROJECT_NAME}/pulls
after_script:
- rm -rf .git/config
tags:
- weights
publish-benchmarks-statemint-s3: &publish-benchmarks
stage: publish
<<: *kubernetes-env
image: paritytech/awscli:latest
<<: *benchmarks-refs
needs:
- job: benchmarks-statemint
artifacts: true
variables:
GIT_STRATEGY: none
BUCKET: "releases.parity.io"
PREFIX: "cumulus/$CI_COMMIT_REF_NAME/benchmarks"
script:
- echo "___Removing binary from artifacts___"
- rm -f ./artifacts/polkadot-parachain
- echo "___Publishing benchmark results___"
- aws s3 sync ./artifacts/ s3://${BUCKET}/${PREFIX}/
after_script:
- aws s3 ls s3://${BUCKET}/${PREFIX}/ --recursive --human-readable --summarize
publish-benchmarks-collectives-s3:
<<: *publish-benchmarks
needs:
- job: benchmarks-collectives
artifacts: true
# Ref: https://github.com/paritytech/opstooling/issues/111
update-parachain-template:
stage: publish
extends: .kubernetes-env
rules:
- if: $CI_COMMIT_REF_NAME =~ /^polkadot-v[0-9]+\.[0-9]+.*$/ # i.e. polkadot-v1.0.99, polkadot-v2.1rc1
script:
- git clone --depth=1 --branch="$PIPELINE_SCRIPTS_TAG" https://github.com/paritytech/pipeline-scripts
- ./pipeline-scripts/update_substrate_template.sh
--repo-name "substrate-parachain-template"
--template-path "parachain-template"
--github-api-token "$GITHUB_TOKEN"
--polkadot-branch "$CI_COMMIT_REF_NAME"
#### stage: .post
# This job cancels the whole pipeline if any of provided jobs fail.
# In a DAG, every jobs chain is executed independently of others. The `fail_fast` principle suggests
# to fail the pipeline as soon as possible to shorten the feedback loop.
cancel-pipeline:
stage: .post
needs:
- job: test-linux-stable
rules:
- if: $CI_COMMIT_REF_NAME =~ /^[0-9]+$/ # PRs
when: on_failure
variables:
PROJECT_ID: "${CI_PROJECT_ID}"
PROJECT_NAME: "${CI_PROJECT_NAME}"
PIPELINE_ID: "${CI_PIPELINE_ID}"
FAILED_JOB_URL: "${FAILED_JOB_URL}"
FAILED_JOB_NAME: "${FAILED_JOB_NAME}"
PR_NUM: "${PR_NUM}"
trigger:
project: "parity/infrastructure/ci_cd/pipeline-stopper"
# remove branch, when pipeline-stopper for substrate and polakdot is updated to the same branch
branch: "as-improve"
Cargo.lock
Generated
+2832 -3042
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+13 -10
View File
@@ -24,16 +24,19 @@ members = [
"primitives/parachain-inherent",
"primitives/timestamp",
"primitives/utility",
"polkadot-parachains/",
"polkadot-parachains/pallets/parachain-info",
"polkadot-parachains/pallets/ping",
"polkadot-parachains/rococo-parachain",
"polkadot-parachains/shell",
"polkadot-parachains/parachains-common",
"polkadot-parachains/statemint",
"polkadot-parachains/statemine",
"polkadot-parachains/westmint",
"polkadot-parachains/canvas-kusama",
"polkadot-parachain/",
"parachains/common",
"parachains/pallets/parachain-info",
"parachains/pallets/ping",
"parachains/runtimes/testing/rococo-parachain",
"parachains/runtimes/starters/shell",
"parachains/runtimes/starters/seedling",
"parachains/runtimes/assets/statemint",
"parachains/runtimes/assets/statemine",
"parachains/runtimes/assets/westmint",
"parachains/runtimes/collectives/collectives-polkadot",
"parachains/runtimes/contracts/contracts-rococo",
"parachains/runtimes/testing/penpal",
"test/client",
"test/relay-sproof-builder",
"test/relay-validation-worker-provider",
README.md
+31 -105
View File
@@ -1,5 +1,7 @@
# Cumulus ☁️
[![Doc](https://img.shields.io/badge/cumulus%20docs-master-brightgreen)](https://paritytech.github.io/cumulus/)
This repository contains both the Cumulus SDK and also specific chains implemented
on top of this SDK.
@@ -33,7 +35,11 @@ and treat as best.
### Collator
A Polkadot [collator](https://wiki.polkadot.network/docs/en/learn-collator) for the parachain is
implemented by the `polkadot-collator` binary.
implemented by the `polkadot-parachain` binary (previously called `polkadot-collator`).
## Installation
Before building Cumulus SDK based nodes / runtimes prepare your environment by following Substrate [installation instructions](https://docs.substrate.io/main-docs/install/).
## Statemint 🪙
@@ -44,110 +50,24 @@ Statemint is a common good parachain providing an asset store for the Polkadot e
### Build & Launch a Node
To run a Statemine or Westmint node (Statemint is not deployed, yet) you will need to compile the
`polkadot-collator` binary:
`polkadot-parachain` binary:
```bash
cargo build --release --locked -p polkadot-collator
cargo build --release --locked -p polkadot-parachain
```
Once the executable is built, launch the parachain node via:
```bash
CHAIN=westmint # or statemine
./target/release/polkadot-collator --chain $CHAIN
./target/release/polkadot-parachain --chain $CHAIN
```
Refer to the [setup instructions below](#local-setup) to run a local network for development.
## Canvas 🧑‍🎨
## Contracts 📝
[![matrix][k1]][k2] [![discord][l1]][l2]
[k1]: https://img.shields.io/badge/matrix-chat-brightgreen.svg?style=flat
[k2]: https://riot.im/app/#/room/#ink:matrix.parity.io
[l1]: https://img.shields.io/discord/722223075629727774?style=flat-square&label=discord
[l2]: https://discord.com/invite/wGUDt2p
This is a node implementation of `Canvas`, a common good parachain for `pallet-contracts`
based wasm smart contracts. Right now this repository only contains the `canvas-kusama` runtime
which we plan to use for both Rococo and Kusama.
If you have any questions, feel free to talk to us on [Element][k2] or on [Discord][l2]
(in the [`ink_smart-contracts`](https://discord.com/channels/722223075629727774/765280480609828864) channel).
## Developing Smart Contracts for Canvas
![Canvas Overview](./docs/canvas-overview.svg)
This node contains Substrate's smart contracts module ‒ the
[`contracts`](https://github.com/paritytech/substrate/tree/master/frame/contracts) pallet.
This `contracts` pallet takes smart contracts as WebAssembly blobs and defines an API
for everything a smart contract needs (storage access, …).
As long as a programming language compiles to WebAssembly and there exists an implementation
of this API in it, you can write a smart contract for this pallet ‒ and thus for Canvas ‒ in
that language.
This is a list of languages you can currently choose from:
* [Parity's ink!](https://github.com/paritytech/ink) for Rust
* [ask!](https://github.com/patractlabs/ask) for Assembly Script
* The [Solang](https://github.com/hyperledger-labs/solang) compiler for Solidity
There are also different user interfaces and command-line tools you can use to deploy
or interact with contracts:
* [polkadot-js](https://polkadot.js.org/apps/)
* [Canvas UI](https://paritytech.github.io/canvas-ui/) (outdated)
If you are looking for a quickstart, we can recommend
[ink!'s Guided Tutorial for Beginners](https://docs.substrate.io/tutorials/v3/ink-workshop/pt1/).
### Build & Launch a Node
To run a Canvas node that connects to Rococo (Kusama and Polkadot parachains are not deployed, yet)
you will need to compile the `polkadot-collator` binary:
```bash
cargo build --release --locked -p polkadot-collator
```
Once the executable is built, launch the parachain node via:
```bash
./target/release/polkadot-collator --chain rocanvas
```
Refer to the [setup instructions below](#local-setup) to run a local network for development.
### Rococo Deployment
We have a live deployment of the Canvas parachain on [Rococo](https://wiki.polkadot.network/docs/build-pdk#rococo-testnet) ‒
a testnet for Polkadot and Kusama parachains.
You can interact with the network through Polkadot JS Apps,
[click here for a direct link to Canvas](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-canvas-rpc.polkadot.io#/explorer).
The Canvas parachain uses the Rococo relay chain's native token (ROC) instead of having its own token.
Due to this you'll need ROC in order to deploy contracts on Canvas.
As a first step, you should create an account. See [here](https://wiki.polkadot.network/docs/learn-account-generation)
for a detailed guide.
As a second step, you have to get ROC testnet tokens through the [Rococo Faucet](https://wiki.polkadot.network/docs/learn-DOT#getting-rococo-tokens).
This is a chat room in which you need to write:
```bash
!drip YOUR_SS_58_ADDRESS:1002
```
The number `1002` is the parachain id of Canvas on Rococo, by supplying it the faucet will teleport ROC
tokens directly to your account on the parachain.
If everything worked out, the teleported ROC tokens will show up under
[the "Accounts" tab for Canvas](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-canvas-rpc.polkadot.io#/accounts).
Once you have ROC on Canvas you can deploy a contract as you would normally.
If you're unsure about this, our [guided tutorial](https://docs.substrate.io/tutorials/v3/ink-workshop/pt1/)
will clarify that for you in no time.
See [the `contracts-rococo` readme](parachains/runtimes/contracts/contracts-rococo/README.md) for details.
## Rococo 👑
@@ -164,7 +84,7 @@ eventually be included by the relay chain for a parachain.
To run a Rococo collator you will need to compile the following binary:
```bash
cargo build --release --locked -p polkadot-collator
cargo build --release --locked -p polkadot-parachain
```
Otherwise you can compile it with
@@ -173,7 +93,7 @@ Otherwise you can compile it with
```bash
docker run --rm -it -w /shellhere/cumulus \
-v $(pwd):/shellhere/cumulus \
paritytech/ci-linux:production cargo build --release --locked -p polkadot-collator
paritytech/ci-linux:production cargo build --release --locked -p polkadot-parachain
sudo chown -R $(id -u):$(id -g) target/
```
@@ -184,13 +104,13 @@ Once the executable is built, launch collators for each parachain (repeat once e
`tick`, `trick`, `track`):
```bash
./target/release/polkadot-collator --chain $CHAIN --validator
./target/release/polkadot-parachain --chain $CHAIN --validator
```
### Parachains
* [Statemint](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-statemint-rpc.polkadot.io#/explorer)
* [Canvas on Rococo](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-canvas-rpc.polkadot.io#/explorer)
* [Contracts on Rococo](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-contracts-rpc.polkadot.io#/explorer)
* [RILT](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo.kilt.io#/explorer)
The network uses horizontal message passing (HRMP) to enable communication between parachains and
@@ -204,8 +124,11 @@ Launch a local setup including a Relay Chain and a Parachain.
#### Launch the Relay Chain
```bash
# Compile Polkadot with the real overseer feature
# Clone
git clone https://github.com/paritytech/polkadot
cd polkadot
# Compile Polkadot with the real overseer feature
cargo build --release
# Generate a raw chain spec
@@ -221,24 +144,27 @@ cargo build --release
#### Launch the Parachain
```bash
# Compile
# Clone
git clone https://github.com/paritytech/cumulus
cd cumulus
# Compile
cargo build --release
# Export genesis state
./target/release/polkadot-collator export-genesis-state > genesis-state
./target/release/polkadot-parachain export-genesis-state > genesis-state
# Export genesis wasm
./target/release/polkadot-collator export-genesis-wasm > genesis-wasm
./target/release/polkadot-parachain export-genesis-wasm > genesis-wasm
# Collator1
./target/release/polkadot-collator --collator --alice --force-authoring --tmp --port 40335 --ws-port 9946 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30335
./target/release/polkadot-parachain --collator --alice --force-authoring --tmp --port 40335 --ws-port 9946 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30335
# Collator2
./target/release/polkadot-collator --collator --bob --force-authoring --tmp --port 40336 --ws-port 9947 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30336
./target/release/polkadot-parachain --collator --bob --force-authoring --tmp --port 40336 --ws-port 9947 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30336
# Parachain Full Node 1
./target/release/polkadot-collator --tmp --port 40337 --ws-port 9948 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30337
./target/release/polkadot-parachain --tmp --port 40337 --ws-port 9948 -- --execution wasm --chain ../polkadot/rococo-local-cfde.json --port 30337
```
#### Register the parachain
@@ -247,7 +173,7 @@ cargo build --release
### Containerize
After building `polkadot-collator` with cargo or with Parity CI image as documented in [this chapter](#build--launch-rococo-collators),
After building `polkadot-parachain` with cargo or with Parity CI image as documented in [this chapter](#build--launch-rococo-collators),
the following will allow producing a new docker image where the compiled binary is injected:
```bash
@@ -257,7 +183,7 @@ the following will allow producing a new docker image where the compiled binary
Alternatively, you can build an image with a builder pattern:
```bash
docker build --tag $OWNER/$IMAGE_NAME --file ./docker/polkadot-collator_builder.Containerfile .
docker build --tag $OWNER/$IMAGE_NAME --file ./docker/polkadot-parachain_builder.Containerfile .
You may then run your new container:
client/cli/Cargo.toml
+6 -2
View File
@@ -5,9 +5,13 @@ authors = ["Parity Technologies <admin@parity.io>"]
edition = "2021"
[dependencies]
clap = { version = "3.1", features = ["derive"] }
clap = { version = "3.2.16", features = ["derive", "deprecated"] }
codec = { package = "parity-scale-codec", version = "3.0.0" }
url = "2.2.2"
# Substrate
sc-cli = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-chain-spec = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master" }
url = "2.2.2"
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-runtime = { git = "https://github.com/paritytech/substrate", branch = "master" }
client/cli/src/lib.rs
+183 -37
View File
@@ -18,20 +18,28 @@
#![warn(missing_docs)]
use clap::Parser;
use sc_service::{
config::{PrometheusConfig, TelemetryEndpoints},
BasePath, TransactionPoolOptions,
};
use std::{
fs,
io::{self, Write},
net::SocketAddr,
path::PathBuf,
};
use codec::Encode;
use sc_chain_spec::ChainSpec;
use sc_service::{
config::{PrometheusConfig, TelemetryEndpoints},
BasePath, TransactionPoolOptions,
};
use sp_core::hexdisplay::HexDisplay;
use sp_runtime::{
traits::{Block as BlockT, Hash as HashT, Header as HeaderT, Zero},
StateVersion,
};
use url::Url;
/// The `purge-chain` command used to remove the whole chain: the parachain and the relay chain.
#[derive(Debug, Parser)]
#[derive(Debug, clap::Parser)]
pub struct PurgeChainCmd {
/// The base struct of the purge-chain command.
#[clap(flatten)]
@@ -119,11 +127,145 @@ impl sc_cli::CliConfiguration for PurgeChainCmd {
}
}
fn validate_relay_chain_url(arg: &str) -> Result<(), String> {
/// Command for exporting the genesis state of the parachain
#[derive(Debug, clap::Parser)]
pub struct ExportGenesisStateCommand {
/// Output file name or stdout if unspecified.
#[clap(action)]
pub output: Option<PathBuf>,
/// Write output in binary. Default is to write in hex.
#[clap(short, long)]
pub raw: bool,
#[allow(missing_docs)]
#[clap(flatten)]
pub shared_params: sc_cli::SharedParams,
}
impl ExportGenesisStateCommand {
/// Run the export-genesis-state command
pub fn run<Block: BlockT>(
&self,
chain_spec: &dyn ChainSpec,
genesis_state_version: StateVersion,
) -> sc_cli::Result<()> {
let block: Block = generate_genesis_block(chain_spec, genesis_state_version)?;
let raw_header = block.header().encode();
let output_buf = if self.raw {
raw_header
} else {
format!("0x{:?}", HexDisplay::from(&block.header().encode())).into_bytes()
};
if let Some(output) = &self.output {
fs::write(output, output_buf)?;
} else {
io::stdout().write_all(&output_buf)?;
}
Ok(())
}
}
/// Generate the genesis block from a given ChainSpec.
pub fn generate_genesis_block<Block: BlockT>(
chain_spec: &dyn ChainSpec,
genesis_state_version: StateVersion,
) -> Result<Block, String> {
let storage = chain_spec.build_storage()?;
let child_roots = storage.children_default.iter().map(|(sk, child_content)| {
let state_root = <<<Block as BlockT>::Header as HeaderT>::Hashing as HashT>::trie_root(
child_content.data.clone().into_iter().collect(),
genesis_state_version,
);
(sk.clone(), state_root.encode())
});
let state_root = <<<Block as BlockT>::Header as HeaderT>::Hashing as HashT>::trie_root(
storage.top.clone().into_iter().chain(child_roots).collect(),
genesis_state_version,
);
let extrinsics_root = <<<Block as BlockT>::Header as HeaderT>::Hashing as HashT>::trie_root(
Vec::new(),
genesis_state_version,
);
Ok(Block::new(
<<Block as BlockT>::Header as HeaderT>::new(
Zero::zero(),
extrinsics_root,
state_root,
Default::default(),
Default::default(),
),
Default::default(),
))
}
impl sc_cli::CliConfiguration for ExportGenesisStateCommand {
fn shared_params(&self) -> &sc_cli::SharedParams {
&self.shared_params
}
}
/// Command for exporting the genesis wasm file.
#[derive(Debug, clap::Parser)]
pub struct ExportGenesisWasmCommand {
/// Output file name or stdout if unspecified.
#[clap(action)]
pub output: Option<PathBuf>,
/// Write output in binary. Default is to write in hex.
#[clap(short, long)]
pub raw: bool,
#[allow(missing_docs)]
#[clap(flatten)]
pub shared_params: sc_cli::SharedParams,
}
impl ExportGenesisWasmCommand {
/// Run the export-genesis-wasm command
pub fn run(&self, chain_spec: &dyn ChainSpec) -> sc_cli::Result<()> {
let raw_wasm_blob = extract_genesis_wasm(chain_spec)?;
let output_buf = if self.raw {
raw_wasm_blob
} else {
format!("0x{:?}", HexDisplay::from(&raw_wasm_blob)).into_bytes()
};
if let Some(output) = &self.output {
fs::write(output, output_buf)?;
} else {
io::stdout().write_all(&output_buf)?;
}
Ok(())
}
}
/// Extract the genesis code from a given ChainSpec.
pub fn extract_genesis_wasm(chain_spec: &dyn ChainSpec) -> sc_cli::Result<Vec<u8>> {
let mut storage = chain_spec.build_storage()?;
storage
.top
.remove(sp_core::storage::well_known_keys::CODE)
.ok_or_else(|| "Could not find wasm file in genesis state!".into())
}
impl sc_cli::CliConfiguration for ExportGenesisWasmCommand {
fn shared_params(&self) -> &sc_cli::SharedParams {
&self.shared_params
}
}
fn validate_relay_chain_url(arg: &str) -> Result<Url, String> {
let url = Url::parse(arg).map_err(|e| e.to_string())?;
if url.scheme() == "ws" {
Ok(())
Ok(url)
} else {
Err(format!(
"'{}' URL scheme not supported. Only websocket RPC is currently supported",
@@ -133,7 +275,7 @@ fn validate_relay_chain_url(arg: &str) -> Result<(), String> {
}
/// The `run` command used to run a node.
#[derive(Debug, Parser)]
#[derive(Debug, clap::Parser)]
pub struct RunCmd {
/// The cumulus RunCmd inherents from sc_cli's
#[clap(flatten)]
@@ -148,35 +290,12 @@ pub struct RunCmd {
/// EXPERIMENTAL: Specify an URL to a relay chain full node to communicate with.
#[clap(
long,
parse(try_from_str),
validator = validate_relay_chain_url,
conflicts_with = "collator",
conflicts_with = "validator",
conflicts_with = "alice",
conflicts_with = "bob",
conflicts_with = "charlie",
conflicts_with = "dave",
conflicts_with = "eve",
conflicts_with = "ferdie"
)]
value_parser = validate_relay_chain_url,
conflicts_with_all = &["alice", "bob", "charlie", "dave", "eve", "ferdie", "one", "two"] )
]
pub relay_chain_rpc_url: Option<Url>,
}
/// Options only relevant for collator nodes
#[derive(Clone, Debug)]
pub struct CollatorOptions {
/// Location of relay chain full node
pub relay_chain_rpc_url: Option<Url>,
}
/// A non-redundant version of the `RunCmd` that sets the `validator` field when the
/// original `RunCmd` had the `collator` field.
/// This is how we make `--collator` imply `--validator`.
pub struct NormalizedRunCmd {
/// The cumulus RunCmd inherents from sc_cli's
pub base: sc_cli::RunCmd,
}
impl RunCmd {
/// Create a [`NormalizedRunCmd`] which merges the `collator` cli argument into `validator` to have only one.
pub fn normalize(&self) -> NormalizedRunCmd {
@@ -193,6 +312,21 @@ impl RunCmd {
}
}
/// Options only relevant for collator nodes
#[derive(Clone, Debug)]
pub struct CollatorOptions {
/// Location of relay chain full node
pub relay_chain_rpc_url: Option<Url>,
}
/// A non-redundant version of the `RunCmd` that sets the `validator` field when the
/// original `RunCmd` had the `collator` field.
/// This is how we make `--collator` imply `--validator`.
pub struct NormalizedRunCmd {
/// The cumulus RunCmd inherents from sc_cli's
pub base: sc_cli::RunCmd,
}
impl sc_cli::CliConfiguration for NormalizedRunCmd {
fn shared_params(&self) -> &sc_cli::SharedParams {
self.base.shared_params()
@@ -277,12 +411,24 @@ impl sc_cli::CliConfiguration for NormalizedRunCmd {
self.base.rpc_max_payload()
}
fn rpc_max_request_size(&self) -> sc_cli::Result<Option<usize>> {
Ok(self.base.rpc_max_request_size)
}
fn rpc_max_response_size(&self) -> sc_cli::Result<Option<usize>> {
Ok(self.base.rpc_max_response_size)
}
fn rpc_max_subscriptions_per_connection(&self) -> sc_cli::Result<Option<usize>> {
Ok(self.base.rpc_max_subscriptions_per_connection)
}
fn ws_max_out_buffer_capacity(&self) -> sc_cli::Result<Option<usize>> {
self.base.ws_max_out_buffer_capacity()
}
fn transaction_pool(&self) -> sc_cli::Result<TransactionPoolOptions> {
self.base.transaction_pool()
fn transaction_pool(&self, is_dev: bool) -> sc_cli::Result<TransactionPoolOptions> {
self.base.transaction_pool(is_dev)
}
fn max_runtime_instances(&self) -> sc_cli::Result<Option<usize>> {
client/collator/Cargo.toml
+1 -1
View File
@@ -6,7 +6,7 @@ edition = "2021"
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0", features = [ "derive" ] }
futures = { version = "0.3.1", features = ["compat"] }
futures = "0.3.21"
parking_lot = "0.12.0"
tracing = "0.1.25"
client/collator/src/lib.rs
+6 -3
View File
@@ -359,6 +359,7 @@ pub async fn start_collator<Block, RA, BS, Spawner>(
#[cfg(test)]
mod tests {
use super::*;
use async_trait::async_trait;
use cumulus_client_consensus_common::ParachainCandidate;
use cumulus_test_client::{
Client, ClientBlockImportExt, DefaultTestClientBuilderExt, InitBlockBuilder,
@@ -374,8 +375,10 @@ mod tests {
use sp_state_machine::Backend;
struct AlwaysSupportsParachains;
#[async_trait]
impl HeadSupportsParachains for AlwaysSupportsParachains {
fn head_supports_parachains(&self, _head: &PHash) -> bool {
async fn head_supports_parachains(&self, _head: &PHash) -> bool {
true
}
}
@@ -478,8 +481,8 @@ mod tests {
.0
.into_memory_db();
let backend =
sp_state_machine::new_in_mem::<BlakeTwo256>().update_backend(*header.state_root(), db);
let backend = sp_state_machine::new_in_mem_hash_key::<BlakeTwo256>()
.update_backend(*header.state_root(), db);
// Should return an error, as it was not included while building the proof.
assert!(backend
client/consensus/aura/Cargo.toml
+3 -3
View File
@@ -6,10 +6,10 @@ authors = ["Parity Technologies <admin@parity.io>"]
edition = "2021"
[dependencies]
async-trait = "0.1.53"
async-trait = "0.1.57"
codec = { package = "parity-scale-codec", version = "3.0.0", features = [ "derive" ] }
futures = { version = "0.3.8", features = ["compat"] }
tracing = "0.1.33"
futures = "0.3.21"
tracing = "0.1.36"
# Substrate
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
client/consensus/aura/src/lib.rs
+20 -14
View File
@@ -31,7 +31,7 @@ use cumulus_primitives_core::{relay_chain::v2::Hash as PHash, PersistedValidatio
use futures::lock::Mutex;
use sc_client_api::{backend::AuxStore, BlockOf};
use sc_consensus::BlockImport;
use sc_consensus_slots::{BackoffAuthoringBlocksStrategy, SlotInfo};
use sc_consensus_slots::{BackoffAuthoringBlocksStrategy, SimpleSlotWorker, SlotInfo};
use sc_telemetry::TelemetryHandle;
use sp_api::ProvideRuntimeApi;
use sp_application_crypto::AppPublic;
@@ -42,7 +42,7 @@ use sp_core::crypto::Pair;
use sp_inherents::{CreateInherentDataProviders, InherentData, InherentDataProvider};
use sp_keystore::SyncCryptoStorePtr;
use sp_runtime::traits::{Block as BlockT, Header as HeaderT, Member, NumberFor};
use std::{convert::TryFrom, hash::Hash, sync::Arc};
use std::{convert::TryFrom, hash::Hash, marker::PhantomData, sync::Arc};
mod import_queue;
@@ -53,29 +53,25 @@ pub use sc_consensus_slots::InherentDataProviderExt;
const LOG_TARGET: &str = "aura::cumulus";
/// The implementation of the AURA consensus for parachains.
pub struct AuraConsensus<B, CIDP> {
pub struct AuraConsensus<B, CIDP, W> {
create_inherent_data_providers: Arc<CIDP>,
aura_worker: Arc<
Mutex<
dyn sc_consensus_slots::SlotWorker<B, <EnableProofRecording as ProofRecording>::Proof>
+ Send
+ 'static,
>,
>,
aura_worker: Arc<Mutex<W>>,
slot_duration: SlotDuration,
_phantom: PhantomData<B>,
}
impl<B, CIDP> Clone for AuraConsensus<B, CIDP> {
impl<B, CIDP, W> Clone for AuraConsensus<B, CIDP, W> {
fn clone(&self) -> Self {
Self {
create_inherent_data_providers: self.create_inherent_data_providers.clone(),
aura_worker: self.aura_worker.clone(),
slot_duration: self.slot_duration,
_phantom: PhantomData,
}
}
}
impl<B, CIDP> AuraConsensus<B, CIDP>
impl<B, CIDP> AuraConsensus<B, CIDP, ()>
where
B: BlockT,
CIDP: CreateInherentDataProviders<B, (PHash, PersistedValidationData)> + 'static,
@@ -134,13 +130,21 @@ where
},
);
Box::new(Self {
Box::new(AuraConsensus {
create_inherent_data_providers: Arc::new(create_inherent_data_providers),
aura_worker: Arc::new(Mutex::new(worker)),
slot_duration,
_phantom: PhantomData,
})
}
}
impl<B, CIDP, W> AuraConsensus<B, CIDP, W>
where
B: BlockT,
CIDP: CreateInherentDataProviders<B, (PHash, PersistedValidationData)> + 'static,
CIDP::InherentDataProviders: InherentDataProviderExt,
{
/// Create the inherent data.
///
/// Returns the created inherent data and the inherent data providers used.
@@ -178,11 +182,13 @@ where
}
#[async_trait::async_trait]
impl<B, CIDP> ParachainConsensus<B> for AuraConsensus<B, CIDP>
impl<B, CIDP, W> ParachainConsensus<B> for AuraConsensus<B, CIDP, W>
where
B: BlockT,
CIDP: CreateInherentDataProviders<B, (PHash, PersistedValidationData)> + Send + Sync + 'static,
CIDP::InherentDataProviders: InherentDataProviderExt + Send,
W: SimpleSlotWorker<B> + Send + Sync,
W::Proposer: Proposer<B, Proof = <EnableProofRecording as ProofRecording>::Proof>,
{
async fn produce_candidate(
&mut self,
client/consensus/common/Cargo.toml
+4 -4
View File
@@ -6,11 +6,11 @@ authors = ["Parity Technologies <admin@parity.io>"]
edition = "2021"
[dependencies]
futures = { version = "0.3.8", features = ["compat"] }
async-trait = "0.1.57"
codec = { package = "parity-scale-codec", version = "3.0.0", features = [ "derive" ] }
tracing = "0.1.33"
async-trait = "0.1.53"
dyn-clone = "1.0.5"
dyn-clone = "1.0.9"
futures = "0.3.21"
tracing = "0.1.36"
# Substrate
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
client/consensus/relay-chain/Cargo.toml
+4 -4
View File
@@ -6,10 +6,10 @@ authors = ["Parity Technologies <admin@parity.io>"]
edition = "2021"
[dependencies]
async-trait = "0.1.53"
futures = { version = "0.3.8", features = ["compat"] }
parking_lot = "0.12.0"
tracing = "0.1.33"
async-trait = "0.1.57"
futures = "0.3.21"
parking_lot = "0.12.1"
tracing = "0.1.36"
# Substrate
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
client/network/Cargo.toml
+5 -6
View File
@@ -6,13 +6,13 @@ description = "Cumulus-specific networking protocol"
edition = "2021"
[dependencies]
async-trait = "0.1.53"
async-trait = "0.1.57"
codec = { package = "parity-scale-codec", version = "3.0.0", features = [ "derive" ] }
derive_more = "0.99.2"
futures = { version = "0.3.1", features = ["compat"] }
futures = "0.3.21"
futures-timer = "3.0.2"
parking_lot = "0.12.0"
tracing = "0.1.33"
parking_lot = "0.12.1"
tracing = "0.1.36"
# Substrate
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
@@ -33,13 +33,12 @@ cumulus-relay-chain-interface = { path = "../relay-chain-interface" }
[dev-dependencies]
portpicker = "0.1.1"
tokio = { version = "1.17.0", features = ["macros"] }
tokio = { version = "1.19.2", features = ["macros"] }
url = "2.2.2"
# Substrate
sc-cli = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-consensus = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-keyring = { git = "https://github.com/paritytech/substrate", branch = "master" }
client/network/tests/sync_blocks_from_tip_without_connected_collator.rs
+1 -1
View File
@@ -18,7 +18,7 @@ use cumulus_primitives_core::ParaId;
use cumulus_test_service::{initial_head_data, run_relay_chain_validator_node, Keyring::*};
use futures::join;
#[substrate_test_utils::test]
#[substrate_test_utils::test(flavor = "multi_thread")]
#[ignore]
async fn sync_blocks_from_tip_without_being_connected_to_a_collator() {
let mut builder = sc_cli::LoggerBuilder::new("");
client/pov-recovery/Cargo.toml
+3 -4
View File
@@ -7,10 +7,10 @@ edition = "2021"
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0", features = [ "derive" ] }
futures = { version = "0.3.1", features = ["compat"] }
futures = "0.3.21"
futures-timer = "3.0.2"
rand = "0.8.5"
tracing = "0.1.33"
tracing = "0.1.36"
# Substrate
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
@@ -31,7 +31,7 @@ cumulus-primitives-core = { path = "../../primitives/core" }
cumulus-relay-chain-interface = {path = "../relay-chain-interface"}
[dev-dependencies]
tokio = { version = "1.17.0", features = ["macros"] }
tokio = { version = "1.19.2", features = ["macros"] }
# Cumulus
cumulus-test-service = { path = "../../test/service" }
@@ -39,5 +39,4 @@ cumulus-test-service = { path = "../../test/service" }
# Substrate
sc-cli = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master" }
substrate-test-utils = { git = "https://github.com/paritytech/substrate", branch = "master" }
client/pov-recovery/src/lib.rs
+1 -1
View File
@@ -218,7 +218,7 @@ where
/// Handle an imported block.
fn handle_block_imported(&mut self, hash: &Block::Hash) {
self.pending_candidates.remove(&hash);
self.pending_candidates.remove(hash);
}
/// Handle a finalized block with the given `block_number`.
client/pov-recovery/tests/pov_recovery.rs
+1 -1
View File
@@ -23,7 +23,7 @@ use std::sync::Arc;
/// If there is a block of the parachain included/backed by the relay chain that isn't circulated in
/// the parachain network, we need to recover the PoV from the relay chain. Using this PoV we can
/// recover the block, import it and share it with the other nodes of the parachain network.
#[substrate_test_utils::test]
#[substrate_test_utils::test(flavor = "multi_thread")]
#[ignore]
async fn pov_recovery() {
let mut builder = sc_cli::LoggerBuilder::new("");
client/relay-chain-inprocess-interface/Cargo.toml
+5 -4
View File
@@ -5,19 +5,19 @@ version = "0.1.0"
edition = "2021"
[dependencies]
async-trait = "0.1.53"
async-trait = "0.1.57"
futures = "0.3.21"
futures-timer = "3.0.2"
parking_lot = "0.12.0"
tracing = "0.1.33"
tracing = "0.1.36"
# Substrate
sc-cli = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-consensus-babe = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-network = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-telemetry = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-tracing = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-sysinfo = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-blockchain = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-consensus = { git = "https://github.com/paritytech/substrate", branch = "master" }
@@ -26,6 +26,7 @@ sp-runtime = { git = "https://github.com/paritytech/substrate", branch = "master
sp-state-machine = { git = "https://github.com/paritytech/substrate", branch = "master" }
# Polkadot
polkadot-cli = { git = "https://github.com/paritytech/polkadot", branch = "master", default-features = false }
polkadot-client = { git = "https://github.com/paritytech/polkadot", branch = "master" }
polkadot-service = { git = "https://github.com/paritytech/polkadot", branch = "master" }
client/relay-chain-inprocess-interface/src/lib.rs
+46 -45
View File
@@ -27,11 +27,11 @@ use cumulus_primitives_core::{
};
use cumulus_relay_chain_interface::{RelayChainError, RelayChainInterface, RelayChainResult};
use futures::{FutureExt, Stream, StreamExt};
use parking_lot::Mutex;
use polkadot_client::{ClientHandle, ExecuteWithClient, FullBackend};
use polkadot_service::{
AuxStore, BabeApi, CollatorPair, Configuration, Handle, NewFull, Role, TaskManager,
AuxStore, BabeApi, CollatorPair, Configuration, Handle, NewFull, TaskManager,
};
use sc_cli::SubstrateCli;
use sc_client_api::{
blockchain::BlockStatus, Backend, BlockchainEvents, HeaderBackend, ImportNotifications,
StorageProof, UsageProvider,
@@ -49,7 +49,7 @@ const TIMEOUT_IN_SECONDS: u64 = 6;
pub struct RelayChainInProcessInterface<Client> {
full_client: Arc<Client>,
backend: Arc<FullBackend>,
sync_oracle: Arc<Mutex<Box<dyn SyncOracle + Send + Sync>>>,
sync_oracle: Arc<dyn SyncOracle + Send + Sync>,
overseer_handle: Option<Handle>,
}
@@ -58,7 +58,7 @@ impl<Client> RelayChainInProcessInterface<Client> {
pub fn new(
full_client: Arc<Client>,
backend: Arc<FullBackend>,
sync_oracle: Arc<Mutex<Box<dyn SyncOracle + Send + Sync>>>,
sync_oracle: Arc<dyn SyncOracle + Send + Sync>,
overseer_handle: Option<Handle>,
) -> Self {
Self { full_client, backend, sync_oracle, overseer_handle }
@@ -168,8 +168,7 @@ where
}
async fn is_major_syncing(&self) -> RelayChainResult<bool> {
let mut network = self.sync_oracle.lock();
Ok(network.is_major_syncing())
Ok(self.sync_oracle.is_major_syncing())
}
fn overseer_handle(&self) -> RelayChainResult<Option<Handle>> {
@@ -288,7 +287,7 @@ where
struct RelayChainInProcessInterfaceBuilder {
polkadot_client: polkadot_client::Client,
backend: Arc<FullBackend>,
sync_oracle: Arc<Mutex<Box<dyn SyncOracle + Send + Sync>>>,
sync_oracle: Arc<dyn SyncOracle + Send + Sync>,
overseer_handle: Option<Handle>,
}
@@ -327,45 +326,54 @@ fn build_polkadot_full_node(
config: Configuration,
parachain_config: &Configuration,
telemetry_worker_handle: Option<TelemetryWorkerHandle>,
hwbench: Option<sc_sysinfo::HwBench>,
) -> Result<(NewFull<polkadot_client::Client>, Option<CollatorPair>), polkadot_service::Error> {
let is_light = matches!(config.role, Role::Light);
if is_light {
Err(polkadot_service::Error::Sub("Light client not supported.".into()))
let (is_collator, maybe_collator_key) = if parachain_config.role.is_authority() {
let collator_key = CollatorPair::generate().0;
(polkadot_service::IsCollator::Yes(collator_key.clone()), Some(collator_key))
} else {
let (is_collator, maybe_collator_key) = if parachain_config.role.is_authority() {
let collator_key = CollatorPair::generate().0;
(polkadot_service::IsCollator::Yes(collator_key.clone()), Some(collator_key))
} else {
(polkadot_service::IsCollator::No, None)
};
(polkadot_service::IsCollator::No, None)
};
let relay_chain_full_node = polkadot_service::build_full(
config,
is_collator,
None,
true,
None,
telemetry_worker_handle,
true,
polkadot_service::RealOverseerGen,
)?;
let relay_chain_full_node = polkadot_service::build_full(
config,
is_collator,
None,
// Disable BEEFY. It should not be required by the internal relay chain node.
false,
None,
telemetry_worker_handle,
true,
polkadot_service::RealOverseerGen,
None,
None,
hwbench,
)?;
Ok((relay_chain_full_node, maybe_collator_key))
}
Ok((relay_chain_full_node, maybe_collator_key))
}
/// Builds a relay chain interface by constructing a full relay chain node
pub fn build_inprocess_relay_chain(
polkadot_config: Configuration,
mut polkadot_config: Configuration,
parachain_config: &Configuration,
telemetry_worker_handle: Option<TelemetryWorkerHandle>,
task_manager: &mut TaskManager,
hwbench: Option<sc_sysinfo::HwBench>,
) -> RelayChainResult<(Arc<(dyn RelayChainInterface + 'static)>, Option<CollatorPair>)> {
let (full_node, collator_key) =
build_polkadot_full_node(polkadot_config, parachain_config, telemetry_worker_handle)?;
// This is essentially a hack, but we want to ensure that we send the correct node version
// to the telemetry.
polkadot_config.impl_version = polkadot_cli::Cli::impl_version();
polkadot_config.impl_name = polkadot_cli::Cli::impl_name();
let sync_oracle: Box<dyn SyncOracle + Send + Sync> = Box::new(full_node.network.clone());
let sync_oracle = Arc::new(Mutex::new(sync_oracle));
let (full_node, collator_key) = build_polkadot_full_node(
polkadot_config,
parachain_config,
telemetry_worker_handle,
hwbench,
)?;
let sync_oracle: Arc<dyn SyncOracle + Send + Sync> = Arc::new(full_node.network.clone());
let relay_chain_interface_builder = RelayChainInProcessInterfaceBuilder {
polkadot_client: full_node.client.clone(),
backend: full_node.backend.clone(),
@@ -380,8 +388,6 @@ pub fn build_inprocess_relay_chain(
#[cfg(test)]
mod tests {
use parking_lot::Mutex;
use super::*;
use polkadot_primitives::v2::Block as PBlock;
@@ -390,20 +396,20 @@ mod tests {
DefaultTestClientBuilderExt, ExecutionStrategy, InitPolkadotBlockBuilder,
TestClientBuilder, TestClientBuilderExt,
};
use sc_service::Arc;
use sp_consensus::{BlockOrigin, SyncOracle};
use sp_runtime::traits::Block as BlockT;
use std::sync::Arc;
use futures::{executor::block_on, poll, task::Poll};
struct DummyNetwork {}
impl SyncOracle for DummyNetwork {
fn is_major_syncing(&mut self) -> bool {
fn is_major_syncing(&self) -> bool {
unimplemented!("Not needed for test")
}
fn is_offline(&mut self) -> bool {
fn is_offline(&self) -> bool {
unimplemented!("Not needed for test")
}
}
@@ -417,17 +423,12 @@ mod tests {
let block_builder = client.init_polkadot_block_builder();
let block = block_builder.build().expect("Finalizes the block").block;
let dummy_network: Box<dyn SyncOracle + Sync + Send> = Box::new(DummyNetwork {});
let dummy_network: Arc<dyn SyncOracle + Sync + Send> = Arc::new(DummyNetwork {});
(
client.clone(),
block,
RelayChainInProcessInterface::new(
client,
backend.clone(),
Arc::new(Mutex::new(dummy_network)),
None,
),
RelayChainInProcessInterface::new(client, backend.clone(), dummy_network, None),
)
}
client/relay-chain-interface/Cargo.toml
+5 -6
View File
@@ -16,12 +16,11 @@ sp-runtime = { git = "https://github.com/paritytech/substrate", branch = "master
sp-blockchain = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-state-machine = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master" }
futures = "0.3.21"
parking_lot = "0.12.0"
parking_lot = "0.12.1"
derive_more = "0.99.2"
async-trait = "0.1.53"
thiserror = "1.0.30"
jsonrpsee-core = "0.10.1"
parity-scale-codec = "3.1.2"
async-trait = "0.1.57"
thiserror = "1.0.32"
jsonrpsee-core = "0.15.1"
parity-scale-codec = "3.1.5"
client/relay-chain-interface/src/lib.rs
+5 -3
View File
@@ -29,7 +29,7 @@ use sc_client_api::StorageProof;
use futures::Stream;
use async_trait::async_trait;
use jsonrpsee_core::Error as JsonRPSeeError;
use jsonrpsee_core::Error as JsonRpcError;
use parity_scale_codec::Error as CodecError;
use sp_api::ApiError;
use sp_state_machine::StorageValue;
@@ -51,9 +51,11 @@ pub enum RelayChainError {
#[error("State machine error occured: {0}")]
StateMachineError(Box<dyn sp_state_machine::Error>),
#[error("Unable to call RPC method '{0}' due to error: {1}")]
RPCCallError(String, JsonRPSeeError),
RpcCallError(String, JsonRpcError),
#[error("RPC Error: '{0}'")]
JsonRPCError(#[from] JsonRPSeeError),
JsonRpcError(#[from] JsonRpcError),
#[error("Unable to reach RpcStreamWorker: {0}")]
WorkerCommunicationError(String),
#[error("Scale codec deserialization error: {0}")]
DeserializationError(CodecError),
#[error("Scale codec deserialization error: {0}")]
client/relay-chain-rpc-interface/Cargo.toml
+6 -5
View File
@@ -18,13 +18,14 @@ sp-state-machine = { git = "https://github.com/paritytech/substrate", branch = "
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sp-storage = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-rpc-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
tokio = { version = "1.19.2", features = ["sync"] }
futures = "0.3.21"
futures-timer = "3.0.2"
parity-scale-codec = "3.1.2"
parking_lot = "0.12.0"
jsonrpsee = { version = "0.10.1", features = ["client"] }
tracing = "0.1.33"
async-trait = "0.1.53"
parity-scale-codec = "3.1.5"
parking_lot = "0.12.1"
jsonrpsee = { version = "0.15.1", features = ["ws-client"] }
tracing = "0.1.36"
async-trait = "0.1.57"
url = "2.2.2"
backoff = { version = "0.4.0", features = ["tokio"] }
client/relay-chain-rpc-interface/src/lib.rs
+17 -291
View File
@@ -15,7 +15,6 @@
// along with Cumulus. If not, see <http://www.gnu.org/licenses/>.
use async_trait::async_trait;
use backoff::{future::retry_notify, ExponentialBackoff};
use core::time::Duration;
use cumulus_primitives_core::{
relay_chain::{
@@ -26,274 +25,35 @@ use cumulus_primitives_core::{
};
use cumulus_relay_chain_interface::{RelayChainError, RelayChainInterface, RelayChainResult};
use futures::{FutureExt, Stream, StreamExt};
use jsonrpsee::{
core::{
client::{Client as JsonRPCClient, ClientT, Subscription, SubscriptionClientT},
Error as JsonRpseeError,
},
rpc_params,
types::ParamsSer,
ws_client::WsClientBuilder,
};
use parity_scale_codec::{Decode, Encode};
use polkadot_service::Handle;
use sc_client_api::{StorageData, StorageProof};
use sc_rpc_api::{state::ReadProof, system::Health};
use sc_client_api::StorageProof;
use sp_core::sp_std::collections::btree_map::BTreeMap;
use sp_runtime::DeserializeOwned;
use sp_state_machine::StorageValue;
use sp_storage::StorageKey;
use std::{pin::Pin, sync::Arc};
use std::pin::Pin;
pub use url::Url;
const LOG_TARGET: &str = "relay-chain-rpc-interface";
mod rpc_client;
pub use rpc_client::{create_client_and_start_worker, RelayChainRpcClient};
const TIMEOUT_IN_SECONDS: u64 = 6;
/// Client that maps RPC methods and deserializes results
#[derive(Clone)]
struct RelayChainRPCClient {
/// Websocket client to make calls
ws_client: Arc<JsonRPCClient>,
/// Retry strategy that should be used for requests and subscriptions
retry_strategy: ExponentialBackoff,
}
impl RelayChainRPCClient {
pub async fn new(url: Url) -> RelayChainResult<Self> {
tracing::info!(target: LOG_TARGET, url = %url.to_string(), "Initializing RPC Client");
let ws_client = WsClientBuilder::default().build(url.as_str()).await?;
Ok(RelayChainRPCClient {
ws_client: Arc::new(ws_client),
retry_strategy: ExponentialBackoff::default(),
})
}
/// Call a call to `state_call` rpc method.
async fn call_remote_runtime_function<R: Decode>(
&self,
method_name: &str,
hash: PHash,
payload: Option<impl Encode>,
) -> RelayChainResult<R> {
let payload_bytes =
payload.map_or(sp_core::Bytes(Vec::new()), |v| sp_core::Bytes(v.encode()));
let params = rpc_params! {
method_name,
payload_bytes,
hash
};
let res = self
.request_tracing::<sp_core::Bytes, _>("state_call", params, |err| {
tracing::trace!(
target: LOG_TARGET,
%method_name,
%hash,
error = %err,
"Error during call to 'state_call'.",
);
})
.await?;
Decode::decode(&mut &*res.0).map_err(Into::into)
}
/// Subscribe to a notification stream via RPC
async fn subscribe<'a, R>(
&self,
sub_name: &'a str,
unsub_name: &'a str,
params: Option<ParamsSer<'a>>,
) -> RelayChainResult<Subscription<R>>
where
R: DeserializeOwned,
{
self.ws_client
.subscribe::<R>(sub_name, params, unsub_name)
.await
.map_err(|err| RelayChainError::RPCCallError(sub_name.to_string(), err))
}
/// Perform RPC request
async fn request<'a, R>(
&self,
method: &'a str,
params: Option<ParamsSer<'a>>,
) -> Result<R, RelayChainError>
where
R: DeserializeOwned + std::fmt::Debug,
{
self.request_tracing(
method,
params,
|e| tracing::trace!(target:LOG_TARGET, error = %e, %method, "Unable to complete RPC request"),
)
.await
}
/// Perform RPC request
async fn request_tracing<'a, R, OR>(
&self,
method: &'a str,
params: Option<ParamsSer<'a>>,
trace_error: OR,
) -> Result<R, RelayChainError>
where
R: DeserializeOwned + std::fmt::Debug,
OR: Fn(&jsonrpsee::core::Error),
{
retry_notify(
self.retry_strategy.clone(),
|| async {
self.ws_client.request(method, params.clone()).await.map_err(|err| match err {
JsonRpseeError::Transport(_) =>
backoff::Error::Transient { err, retry_after: None },
_ => backoff::Error::Permanent(err),
})
},
|error, dur| tracing::trace!(target: LOG_TARGET, %error, ?dur, "Encountered transport error, retrying."),
)
.await
.map_err(|err| {
trace_error(&err);
RelayChainError::RPCCallError(method.to_string(), err)})
}
async fn system_health(&self) -> Result<Health, RelayChainError> {
self.request("system_health", None).await
}
async fn state_get_read_proof(
&self,
storage_keys: Vec<StorageKey>,
at: Option<PHash>,
) -> Result<ReadProof<PHash>, RelayChainError> {
let params = rpc_params!(storage_keys, at);
self.request("state_getReadProof", params).await
}
async fn state_get_storage(
&self,
storage_key: StorageKey,
at: Option<PHash>,
) -> Result<Option<StorageData>, RelayChainError> {
let params = rpc_params!(storage_key, at);
self.request("state_getStorage", params).await
}
async fn chain_get_head(&self) -> Result<PHash, RelayChainError> {
self.request("chain_getHead", None).await
}
async fn chain_get_header(
&self,
hash: Option<PHash>,
) -> Result<Option<PHeader>, RelayChainError> {
let params = rpc_params!(hash);
self.request("chain_getHeader", params).await
}
async fn parachain_host_candidate_pending_availability(
&self,
at: PHash,
para_id: ParaId,
) -> Result<Option<CommittedCandidateReceipt>, RelayChainError> {
self.call_remote_runtime_function(
"ParachainHost_candidate_pending_availability",
at,
Some(para_id),
)
.await
}
async fn parachain_host_session_index_for_child(
&self,
at: PHash,
) -> Result<SessionIndex, RelayChainError> {
self.call_remote_runtime_function("ParachainHost_session_index_for_child", at, None::<()>)
.await
}
async fn parachain_host_validators(
&self,
at: PHash,
) -> Result<Vec<ValidatorId>, RelayChainError> {
self.call_remote_runtime_function("ParachainHost_validators", at, None::<()>)
.await
}
async fn parachain_host_persisted_validation_data(
&self,
at: PHash,
para_id: ParaId,
occupied_core_assumption: OccupiedCoreAssumption,
) -> Result<Option<PersistedValidationData>, RelayChainError> {
self.call_remote_runtime_function(
"ParachainHost_persisted_validation_data",
at,
Some((para_id, occupied_core_assumption)),
)
.await
}
async fn parachain_host_inbound_hrmp_channels_contents(
&self,
para_id: ParaId,
at: PHash,
) -> Result<BTreeMap<ParaId, Vec<InboundHrmpMessage>>, RelayChainError> {
self.call_remote_runtime_function(
"ParachainHost_inbound_hrmp_channels_contents",
at,
Some(para_id),
)
.await
}
async fn parachain_host_dmq_contents(
&self,
para_id: ParaId,
at: PHash,
) -> Result<Vec<InboundDownwardMessage>, RelayChainError> {
self.call_remote_runtime_function("ParachainHost_dmq_contents", at, Some(para_id))
.await
}
async fn subscribe_all_heads(&self) -> Result<Subscription<PHeader>, RelayChainError> {
self.subscribe::<PHeader>("chain_subscribeAllHeads", "chain_unsubscribeAllHeads", None)
.await
}
async fn subscribe_new_best_heads(&self) -> Result<Subscription<PHeader>, RelayChainError> {
self.subscribe::<PHeader>("chain_subscribeNewHeads", "chain_unsubscribeNewHeads", None)
.await
}
async fn subscribe_finalized_heads(&self) -> Result<Subscription<PHeader>, RelayChainError> {
self.subscribe::<PHeader>(
"chain_subscribeFinalizedHeads",
"chain_unsubscribeFinalizedHeads",
None,
)
.await
}
}
/// RelayChainRPCInterface is used to interact with a full node that is running locally
/// RelayChainRpcInterface is used to interact with a full node that is running locally
/// in the same process.
#[derive(Clone)]
pub struct RelayChainRPCInterface {
rpc_client: RelayChainRPCClient,
pub struct RelayChainRpcInterface {
rpc_client: RelayChainRpcClient,
}
impl RelayChainRPCInterface {
pub async fn new(url: Url) -> RelayChainResult<Self> {
Ok(Self { rpc_client: RelayChainRPCClient::new(url).await? })
impl RelayChainRpcInterface {
pub fn new(rpc_client: RelayChainRpcClient) -> Self {
Self { rpc_client }
}
}
#[async_trait]
impl RelayChainInterface for RelayChainRPCInterface {
impl RelayChainInterface for RelayChainRpcInterface {
async fn retrieve_dmq_contents(
&self,
para_id: ParaId,
@@ -344,17 +104,7 @@ impl RelayChainInterface for RelayChainRPCInterface {
async fn import_notification_stream(
&self,
) -> RelayChainResult<Pin<Box<dyn Stream<Item = PHeader> + Send>>> {
let imported_headers_stream =
self.rpc_client.subscribe_all_heads().await?.filter_map(|item| async move {
item.map_err(|err| {
tracing::error!(
target: LOG_TARGET,
"Encountered error in import notification stream: {}",
err
)
})
.ok()
});
let imported_headers_stream = self.rpc_client.get_imported_heads_stream().await?;
Ok(imported_headers_stream.boxed())
}
@@ -362,20 +112,7 @@ impl RelayChainInterface for RelayChainRPCInterface {
async fn finality_notification_stream(
&self,
) -> RelayChainResult<Pin<Box<dyn Stream<Item = PHeader> + Send>>> {
let imported_headers_stream = self
.rpc_client
.subscribe_finalized_heads()
.await?
.filter_map(|item| async move {
item.map_err(|err| {
tracing::error!(
target: LOG_TARGET,
"Encountered error in finality notification stream: {}",
err
)
})
.ok()
});
let imported_headers_stream = self.rpc_client.get_finalized_heads_stream().await?;
Ok(imported_headers_stream.boxed())
}
@@ -430,7 +167,7 @@ impl RelayChainInterface for RelayChainRPCInterface {
/// 3. Wait for the block to be imported via subscription.
/// 4. If timeout is reached, we return an error.
async fn wait_for_block(&self, wait_for_hash: PHash) -> RelayChainResult<()> {
let mut head_stream = self.rpc_client.subscribe_all_heads().await?;
let mut head_stream = self.rpc_client.get_imported_heads_stream().await?;
if self.rpc_client.chain_get_header(Some(wait_for_hash)).await?.is_some() {
return Ok(())
@@ -442,7 +179,7 @@ impl RelayChainInterface for RelayChainRPCInterface {
futures::select! {
_ = timeout => return Err(RelayChainError::WaitTimeout(wait_for_hash)),
evt = head_stream.next().fuse() => match evt {
Some(Ok(evt)) if evt.hash() == wait_for_hash => return Ok(()),
Some(evt) if evt.hash() == wait_for_hash => return Ok(()),
// Not the event we waited on.
Some(_) => continue,
None => return Err(RelayChainError::ImportListenerClosed(wait_for_hash)),
@@ -454,18 +191,7 @@ impl RelayChainInterface for RelayChainRPCInterface {
async fn new_best_notification_stream(
&self,
) -> RelayChainResult<Pin<Box<dyn Stream<Item = PHeader> + Send>>> {
let imported_headers_stream =
self.rpc_client.subscribe_new_best_heads().await?.filter_map(|item| async move {
item.map_err(|err| {
tracing::error!(
target: LOG_TARGET,
"Error in best block notification stream: {}",
err
)
})
.ok()
});
let imported_headers_stream = self.rpc_client.get_best_heads_stream().await?;
Ok(imported_headers_stream.boxed())
}
}
client/relay-chain-rpc-interface/src/rpc_client.rs
+466
View File
@@ -0,0 +1,466 @@
// Copyright 2022 Parity Technologies (UK) Ltd.
// This file is part of Cumulus.
// Cumulus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Cumulus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Cumulus. If not, see <http://www.gnu.org/licenses/>.
use backoff::{future::retry_notify, ExponentialBackoff};
use cumulus_primitives_core::{
relay_chain::{
v2::{CommittedCandidateReceipt, OccupiedCoreAssumption, SessionIndex, ValidatorId},
Hash as PHash, Header as PHeader, InboundHrmpMessage,
},
InboundDownwardMessage, ParaId, PersistedValidationData,
};
use cumulus_relay_chain_interface::{RelayChainError, RelayChainResult};
use futures::{
channel::mpsc::{Receiver, Sender},
StreamExt,
};
use jsonrpsee::{
core::{
client::{Client as JsonRpcClient, ClientT, Subscription, SubscriptionClientT},
Error as JsonRpseeError,
},
rpc_params,
types::ParamsSer,
ws_client::WsClientBuilder,
};
use parity_scale_codec::{Decode, Encode};
use polkadot_service::TaskManager;
use sc_client_api::StorageData;
use sc_rpc_api::{state::ReadProof, system::Health};
use sp_core::sp_std::collections::btree_map::BTreeMap;
use sp_runtime::DeserializeOwned;
use sp_storage::StorageKey;
use std::sync::Arc;
use tokio::sync::mpsc::{
channel as tokio_channel, Receiver as TokioReceiver, Sender as TokioSender,
};
pub use url::Url;
const LOG_TARGET: &str = "relay-chain-rpc-client";
const NOTIFICATION_CHANNEL_SIZE_LIMIT: usize = 20;
/// Client that maps RPC methods and deserializes results
#[derive(Clone)]
pub struct RelayChainRpcClient {
/// Websocket client to make calls
ws_client: Arc<JsonRpcClient>,
/// Retry strategy that should be used for requests and subscriptions
retry_strategy: ExponentialBackoff,
/// Channel to communicate with the RPC worker
to_worker_channel: TokioSender<NotificationRegisterMessage>,
}
/// Worker messages to register new notification listeners
#[derive(Clone, Debug)]
pub enum NotificationRegisterMessage {
RegisterBestHeadListener(Sender<PHeader>),
RegisterImportListener(Sender<PHeader>),
RegisterFinalizationListener(Sender<PHeader>),
}
/// Worker that should be used in combination with [`RelayChainRpcClient`]. Must be polled to distribute header notifications to listeners.
struct RpcStreamWorker {
// Communication channel with the RPC client
client_receiver: TokioReceiver<NotificationRegisterMessage>,
// Senders to distribute incoming header notifications to
imported_header_listeners: Vec<Sender<PHeader>>,
finalized_header_listeners: Vec<Sender<PHeader>>,
best_header_listeners: Vec<Sender<PHeader>>,
// Incoming notification subscriptions
rpc_imported_header_subscription: Subscription<PHeader>,
rpc_finalized_header_subscription: Subscription<PHeader>,
rpc_best_header_subscription: Subscription<PHeader>,
}
/// Entry point to create [`RelayChainRpcClient`] and start a worker that distributes notifications.
pub async fn create_client_and_start_worker(
url: Url,
task_manager: &mut TaskManager,
) -> RelayChainResult<RelayChainRpcClient> {
tracing::info!(target: LOG_TARGET, url = %url.to_string(), "Initializing RPC Client");
let ws_client = WsClientBuilder::default().build(url.as_str()).await?;
let best_head_stream = RelayChainRpcClient::subscribe_new_best_heads(&ws_client).await?;
let finalized_head_stream = RelayChainRpcClient::subscribe_finalized_heads(&ws_client).await?;
let imported_head_stream = RelayChainRpcClient::subscribe_imported_heads(&ws_client).await?;
let (worker, sender) =
RpcStreamWorker::new(imported_head_stream, best_head_stream, finalized_head_stream);
let client = RelayChainRpcClient::new(ws_client, sender).await?;
task_manager
.spawn_essential_handle()
.spawn("relay-chain-rpc-worker", None, worker.run());
Ok(client)
}
fn handle_event_distribution(
event: Option<Result<PHeader, JsonRpseeError>>,
senders: &mut Vec<Sender<PHeader>>,
) -> Result<(), String> {
match event {
Some(Ok(header)) => {
senders.retain_mut(|e| {
match e.try_send(header.clone()) {
// Receiver has been dropped, remove Sender from list.
Err(error) if error.is_disconnected() => false,
// Channel is full. This should not happen.
// TODO: Improve error handling here
// https://github.com/paritytech/cumulus/issues/1482
Err(error) => {
tracing::error!(target: LOG_TARGET, ?error, "Event distribution channel has reached its limit. This can lead to missed notifications.");
true
},
_ => true,
}
});
Ok(())
},
None => Err("RPC Subscription closed.".to_string()),
Some(Err(err)) => Err(format!("Error in RPC subscription: {}", err)),
}
}
impl RpcStreamWorker {
/// Create new worker. Returns the worker and a channel to register new listeners.
fn new(
import_sub: Subscription<PHeader>,
best_sub: Subscription<PHeader>,
finalized_sub: Subscription<PHeader>,
) -> (RpcStreamWorker, TokioSender<NotificationRegisterMessage>) {
let (tx, rx) = tokio_channel(100);
let worker = RpcStreamWorker {
client_receiver: rx,
imported_header_listeners: Vec::new(),
finalized_header_listeners: Vec::new(),
best_header_listeners: Vec::new(),
rpc_imported_header_subscription: import_sub,
rpc_best_header_subscription: best_sub,
rpc_finalized_header_subscription: finalized_sub,
};
(worker, tx)
}
/// Run this worker to drive notification streams.
/// The worker does two things:
/// 1. Listen for `NotificationRegisterMessage` and register new listeners for the notification streams
/// 2. Distribute incoming import, best head and finalization notifications to registered listeners.
/// If an error occurs during sending, the receiver has been closed and we remove the sender from the list.
pub async fn run(mut self) {
let mut import_sub = self.rpc_imported_header_subscription.fuse();
let mut best_head_sub = self.rpc_best_header_subscription.fuse();
let mut finalized_sub = self.rpc_finalized_header_subscription.fuse();
loop {
tokio::select! {
evt = self.client_receiver.recv() => match evt {
Some(NotificationRegisterMessage::RegisterBestHeadListener(tx)) => {
self.best_header_listeners.push(tx);
},
Some(NotificationRegisterMessage::RegisterImportListener(tx)) => {
self.imported_header_listeners.push(tx)
},
Some(NotificationRegisterMessage::RegisterFinalizationListener(tx)) => {
self.finalized_header_listeners.push(tx)
},
None => {
tracing::error!(target: LOG_TARGET, "RPC client receiver closed. Stopping RPC Worker.");
return;
}
},
import_event = import_sub.next() => {
if let Err(err) = handle_event_distribution(import_event, &mut self.imported_header_listeners) {
tracing::error!(target: LOG_TARGET, err, "Encountered error while processing imported header notification. Stopping RPC Worker.");
return;
}
},
best_header_event = best_head_sub.next() => {
if let Err(err) = handle_event_distribution(best_header_event, &mut self.best_header_listeners) {
tracing::error!(target: LOG_TARGET, err, "Encountered error while processing best header notification. Stopping RPC Worker.");
return;
}
}
finalized_event = finalized_sub.next() => {
if let Err(err) = handle_event_distribution(finalized_event, &mut self.finalized_header_listeners) {
tracing::error!(target: LOG_TARGET, err, "Encountered error while processing finalized header notification. Stopping RPC Worker.");
return;
}
}
}
}
}
}
impl RelayChainRpcClient {
/// Initialize new RPC Client.
async fn new(
ws_client: JsonRpcClient,
sender: TokioSender<NotificationRegisterMessage>,
) -> RelayChainResult<Self> {
let client = RelayChainRpcClient {
to_worker_channel: sender,
ws_client: Arc::new(ws_client),
retry_strategy: ExponentialBackoff::default(),
};
Ok(client)
}
/// Call a call to `state_call` rpc method.
pub async fn call_remote_runtime_function<R: Decode>(
&self,
method_name: &str,
hash: PHash,
payload: Option<impl Encode>,
) -> RelayChainResult<R> {
let payload_bytes =
payload.map_or(sp_core::Bytes(Vec::new()), |v| sp_core::Bytes(v.encode()));
let params = rpc_params! {
method_name,
payload_bytes,
hash
};
let res = self
.request_tracing::<sp_core::Bytes, _>("state_call", params, |err| {
tracing::trace!(
target: LOG_TARGET,
%method_name,
%hash,
error = %err,
"Error during call to 'state_call'.",
);
})
.await?;
Decode::decode(&mut &*res.0).map_err(Into::into)
}
/// Subscribe to a notification stream via RPC
/// Perform RPC request
async fn request<'a, R>(
&self,
method: &'a str,
params: Option<ParamsSer<'a>>,
) -> Result<R, RelayChainError>
where
R: DeserializeOwned + std::fmt::Debug,
{
self.request_tracing(
method,
params,
|e| tracing::trace!(target:LOG_TARGET, error = %e, %method, "Unable to complete RPC request"),
)
.await
}
/// Perform RPC request
async fn request_tracing<'a, R, OR>(
&self,
method: &'a str,
params: Option<ParamsSer<'a>>,
trace_error: OR,
) -> Result<R, RelayChainError>
where
R: DeserializeOwned + std::fmt::Debug,
OR: Fn(&jsonrpsee::core::Error),
{
retry_notify(
self.retry_strategy.clone(),
|| async {
self.ws_client.request(method, params.clone()).await.map_err(|err| match err {
JsonRpseeError::Transport(_) =>
backoff::Error::Transient { err, retry_after: None },
_ => backoff::Error::Permanent(err),
})
},
|error, dur| tracing::trace!(target: LOG_TARGET, %error, ?dur, "Encountered transport error, retrying."),
)
.await
.map_err(|err| {
trace_error(&err);
RelayChainError::RpcCallError(method.to_string(), err)})
}
pub async fn system_health(&self) -> Result<Health, RelayChainError> {
self.request("system_health", None).await
}
pub async fn state_get_read_proof(
&self,
storage_keys: Vec<StorageKey>,
at: Option<PHash>,
) -> Result<ReadProof<PHash>, RelayChainError> {
let params = rpc_params!(storage_keys, at);
self.request("state_getReadProof", params).await
}
pub async fn state_get_storage(
&self,
storage_key: StorageKey,
at: Option<PHash>,
) -> Result<Option<StorageData>, RelayChainError> {
let params = rpc_params!(storage_key, at);
self.request("state_getStorage", params).await
}
pub async fn chain_get_head(&self) -> Result<PHash, RelayChainError> {
self.request("chain_getHead", None).await
}
pub async fn chain_get_header(
&self,
hash: Option<PHash>,
) -> Result<Option<PHeader>, RelayChainError> {
let params = rpc_params!(hash);
self.request("chain_getHeader", params).await
}
pub async fn parachain_host_candidate_pending_availability(
&self,
at: PHash,
para_id: ParaId,
) -> Result<Option<CommittedCandidateReceipt>, RelayChainError> {
self.call_remote_runtime_function(
"ParachainHost_candidate_pending_availability",
at,
Some(para_id),
)
.await
}
pub async fn parachain_host_session_index_for_child(
&self,
at: PHash,
) -> Result<SessionIndex, RelayChainError> {
self.call_remote_runtime_function("ParachainHost_session_index_for_child", at, None::<()>)
.await
}
pub async fn parachain_host_validators(
&self,
at: PHash,
) -> Result<Vec<ValidatorId>, RelayChainError> {
self.call_remote_runtime_function("ParachainHost_validators", at, None::<()>)
.await
}
pub async fn parachain_host_persisted_validation_data(
&self,
at: PHash,
para_id: ParaId,
occupied_core_assumption: OccupiedCoreAssumption,
) -> Result<Option<PersistedValidationData>, RelayChainError> {
self.call_remote_runtime_function(
"ParachainHost_persisted_validation_data",
at,
Some((para_id, occupied_core_assumption)),
)
.await
}
pub async fn parachain_host_inbound_hrmp_channels_contents(
&self,
para_id: ParaId,
at: PHash,
) -> Result<BTreeMap<ParaId, Vec<InboundHrmpMessage>>, RelayChainError> {
self.call_remote_runtime_function(
"ParachainHost_inbound_hrmp_channels_contents",
at,
Some(para_id),
)
.await
}
pub async fn parachain_host_dmq_contents(
&self,
para_id: ParaId,
at: PHash,
) -> Result<Vec<InboundDownwardMessage>, RelayChainError> {
self.call_remote_runtime_function("ParachainHost_dmq_contents", at, Some(para_id))
.await
}
fn send_register_message_to_worker(
&self,
message: NotificationRegisterMessage,
) -> Result<(), RelayChainError> {
self.to_worker_channel
.try_send(message)
.map_err(|e| RelayChainError::WorkerCommunicationError(e.to_string()))
}
pub async fn get_imported_heads_stream(&self) -> Result<Receiver<PHeader>, RelayChainError> {
let (tx, rx) = futures::channel::mpsc::channel::<PHeader>(NOTIFICATION_CHANNEL_SIZE_LIMIT);
self.send_register_message_to_worker(NotificationRegisterMessage::RegisterImportListener(
tx,
))?;
Ok(rx)
}
pub async fn get_best_heads_stream(&self) -> Result<Receiver<PHeader>, RelayChainError> {
let (tx, rx) = futures::channel::mpsc::channel::<PHeader>(NOTIFICATION_CHANNEL_SIZE_LIMIT);
self.send_register_message_to_worker(
NotificationRegisterMessage::RegisterBestHeadListener(tx),
)?;
Ok(rx)
}
pub async fn get_finalized_heads_stream(&self) -> Result<Receiver<PHeader>, RelayChainError> {
let (tx, rx) = futures::channel::mpsc::channel::<PHeader>(NOTIFICATION_CHANNEL_SIZE_LIMIT);
self.send_register_message_to_worker(
NotificationRegisterMessage::RegisterFinalizationListener(tx),
)?;
Ok(rx)
}
async fn subscribe_imported_heads(
ws_client: &JsonRpcClient,
) -> Result<Subscription<PHeader>, RelayChainError> {
Ok(ws_client
.subscribe::<PHeader>("chain_subscribeAllHeads", None, "chain_unsubscribeAllHeads")
.await?)
}
async fn subscribe_finalized_heads(
ws_client: &JsonRpcClient,
) -> Result<Subscription<PHeader>, RelayChainError> {
Ok(ws_client
.subscribe::<PHeader>(
"chain_subscribeFinalizedHeads",
None,
"chain_unsubscribeFinalizedHeads",
)
.await?)
}
async fn subscribe_new_best_heads(
ws_client: &JsonRpcClient,
) -> Result<Subscription<PHeader>, RelayChainError> {
Ok(ws_client
.subscribe::<PHeader>(
"chain_subscribeNewHeads",
None,
"chain_unsubscribeFinalizedHeads",
)
.await?)
}
}
client/service/Cargo.toml
+2 -4
View File
@@ -5,12 +5,10 @@ authors = ["Parity Technologies <admin@parity.io>"]
edition = "2021"
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0" }
parking_lot = "0.12.0"
tracing = "0.1.33"
parking_lot = "0.12.1"
tracing = "0.1.36"
# Substrate
sc-chain-spec = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-client-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-consensus = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-consensus-babe = { git = "https://github.com/paritytech/substrate", branch = "master" }
client/service/src/genesis.rs
-55
View File
@@ -1,55 +0,0 @@
// Copyright 2020-2021 Parity Technologies (UK) Ltd.
// This file is part of Cumulus.
// Cumulus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Cumulus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Cumulus. If not, see <http://www.gnu.org/licenses/>.
use codec::Encode;
use sc_chain_spec::ChainSpec;
use sp_runtime::traits::{Block as BlockT, Hash as HashT, Header as HeaderT, Zero};
/// Generate the genesis block from a given ChainSpec.
pub fn generate_genesis_block<Block: BlockT>(
chain_spec: &Box<dyn ChainSpec>,
genesis_state_version: sp_runtime::StateVersion,
) -> Result<Block, String> {
let storage = chain_spec.build_storage()?;
let child_roots = storage.children_default.iter().map(|(sk, child_content)| {
let state_root = <<<Block as BlockT>::Header as HeaderT>::Hashing as HashT>::trie_root(
child_content.data.clone().into_iter().collect(),
genesis_state_version,
);
(sk.clone(), state_root.encode())
});
let state_root = <<<Block as BlockT>::Header as HeaderT>::Hashing as HashT>::trie_root(
storage.top.clone().into_iter().chain(child_roots).collect(),
genesis_state_version,
);
let extrinsics_root = <<<Block as BlockT>::Header as HeaderT>::Hashing as HashT>::trie_root(
Vec::new(),
sp_runtime::StateVersion::V0,
);
Ok(Block::new(
<<Block as BlockT>::Header as HeaderT>::new(
Zero::zero(),
extrinsics_root,
state_root,
Default::default(),
Default::default(),
),
Default::default(),
))
}
client/service/src/lib.rs
+2 -4
View File
@@ -41,8 +41,6 @@ use sp_runtime::{
};
use std::{sync::Arc, time::Duration};
pub mod genesis;
/// Parameters given to [`start_collator`].
pub struct StartCollatorParams<'a, Block: BlockT, BS, Client, RCInterface, Spawner, IQ> {
pub block_status: Arc<BS>,
@@ -220,9 +218,9 @@ where
min: relay_chain_slot_duration * 25,
max: relay_chain_slot_duration * 50,
},
client.clone(),
client,
import_queue,
relay_chain_interface.clone(),
relay_chain_interface,
para_id,
);
docker/docker-compose.yml
+3 -3
View File
@@ -9,7 +9,7 @@ services:
volumes:
- "polkadot-data-alice:/data"
- type: bind
source: ./test/parachain/res/polkadot_chainspec.json
source: ./test/parachain/chain-specs/polkadot_chainspec.json
target: /chainspec.json
read_only: true
command: >
@@ -38,7 +38,7 @@ services:
volumes:
- "polkadot-data-bob:/data"
- type: bind
source: ./test/parachain/res/polkadot_chainspec.json
source: ./test/parachain/chain-specs/polkadot_chainspec.json
target: /chainspec.json
read_only: true
command: >
@@ -66,7 +66,7 @@ services:
volumes:
- "genesis-state:/data"
command: >
polkadot-collator
polkadot-parachain
export-genesis-state
/data/genesis-state
docker/injected.Dockerfile
+7 -7
View File
@@ -34,18 +34,18 @@ RUN apt-get update && \
ln -s /data /polkadot/.local/share/polkadot && \
mkdir -p /specs
# add polkadot-collator binary to the docker image
COPY ./target/release/polkadot-collator /usr/local/bin
COPY ./target/release/polkadot-collator.asc /usr/local/bin
COPY ./target/release/polkadot-collator.sha256 /usr/local/bin
COPY ./polkadot-parachains/res/*.json /specs/
# add polkadot-parachain binary to the docker image
COPY ./target/release/polkadot-parachain /usr/local/bin
COPY ./target/release/polkadot-parachain.asc /usr/local/bin
COPY ./target/release/polkadot-parachain.sha256 /usr/local/bin
COPY ./parachains/chain-specs/*.json /specs/
USER polkadot
# check if executable works in this container
RUN /usr/local/bin/polkadot-collator --version
RUN /usr/local/bin/polkadot-parachain --version
EXPOSE 30333 9933 9944
VOLUME ["/polkadot"]
ENTRYPOINT ["/usr/local/bin/polkadot-collator"]
ENTRYPOINT ["/usr/local/bin/polkadot-parachain"]
docker/polkadot-parachain-debug_unsigned_injected.Dockerfile
+49
View File
@@ -0,0 +1,49 @@
FROM docker.io/library/ubuntu:20.04
# metadata
ARG VCS_REF
ARG BUILD_DATE
ARG IMAGE_NAME
LABEL io.parity.image.authors="devops-team@parity.io" \
io.parity.image.vendor="Parity Technologies" \
io.parity.image.title="${IMAGE_NAME}" \
io.parity.image.description="Cumulus, the Polkadot collator." \
io.parity.image.source="https://github.com/paritytech/cumulus/blob/${VCS_REF}/scripts/docker/polkadot-parachain-debug_unsigned_injected.Dockerfile" \
io.parity.image.revision="${VCS_REF}" \
io.parity.image.created="${BUILD_DATE}" \
io.parity.image.documentation="https://github.com/paritytech/cumulus/"
# show backtraces
ENV RUST_BACKTRACE 1
# install tools and dependencies
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
libssl1.1 \
ca-certificates \
curl && \
# apt cleanup
apt-get autoremove -y && \
apt-get clean && \
find /var/lib/apt/lists/ -type f -not -name lock -delete; \
# add user and link ~/.local/share/polkadot-parachain to /data
useradd -m -u 10000 -U -s /bin/sh -d /polkadot-parachain polkadot-parachain && \
mkdir -p /data /polkadot-parachain/.local/share && \
chown -R polkadot-parachain:polkadot-parachain /data && \
ln -s /data /polkadot-parachain/.local/share/polkadot-parachain && \
mkdir -p /specs
# add polkadot-parachain binary to the docker image
COPY ./artifacts/polkadot-parachain /usr/local/bin
COPY ./parachains/chain-specs/*.json /specs/
USER polkadot-parachain
# check if executable works in this container
RUN /usr/local/bin/polkadot-parachain --version
EXPOSE 30333 9933 9944
VOLUME ["/polkadot-parachain"]
ENTRYPOINT ["/usr/local/bin/polkadot-parachain"]
docker/polkadot-collator_builder.Containerfile → docker/polkadot-parachain_builder.Containerfile
+11 -11
View File
@@ -1,11 +1,11 @@
# This file is sourced from https://github.com/paritytech/polkadot/blob/master/scripts/ci/dockerfiles/polkadot/polkadot_builder.Dockerfile
# This is the build stage for Polkadot-collator. Here we create the binary in a temporary image.
# This is the build stage for polkadot-parachain. Here we create the binary in a temporary image.
FROM docker.io/paritytech/ci-linux:production as builder
WORKDIR /cumulus
COPY . /cumulus
RUN cargo build --release --locked -p polkadot-collator
RUN cargo build --release --locked -p polkadot-parachain
# This is the 2nd stage: a very small image where we copy the Polkadot binary."
FROM docker.io/library/ubuntu:20.04
@@ -13,24 +13,24 @@ FROM docker.io/library/ubuntu:20.04
LABEL io.parity.image.type="builder" \
io.parity.image.authors="devops-team@parity.io" \
io.parity.image.vendor="Parity Technologies" \
io.parity.image.description="Multistage Docker image for Polkadot-collator" \
io.parity.image.source="https://github.com/paritytech/polkadot/blob/${VCS_REF}/scripts/ci/dockerfiles/polkadot/polkadot-collator_builder.Dockerfile" \
io.parity.image.description="Multistage Docker image for polkadot-parachain" \
io.parity.image.source="https://github.com/paritytech/polkadot/blob/${VCS_REF}/scripts/ci/dockerfiles/polkadot/polkadot-parachain_builder.Dockerfile" \
io.parity.image.documentation="https://github.com/paritytech/cumulus"
COPY --from=builder /cumulus/target/release/polkadot-collator /usr/local/bin
COPY --from=builder /cumulus/target/release/polkadot-parachain /usr/local/bin
RUN useradd -m -u 1000 -U -s /bin/sh -d /cumulus polkadot-collator && \
RUN useradd -m -u 1000 -U -s /bin/sh -d /cumulus polkadot-parachain && \
mkdir -p /data /cumulus/.local/share && \
chown -R polkadot-collator:polkadot-collator /data && \
ln -s /data /cumulus/.local/share/polkadot-collator && \
chown -R polkadot-parachain:polkadot-parachain /data && \
ln -s /data /cumulus/.local/share/polkadot-parachain && \
# unclutter and minimize the attack surface
rm -rf /usr/bin /usr/sbin && \
# check if executable works in this container
/usr/local/bin/polkadot-collator --version
/usr/local/bin/polkadot-parachain --version
USER polkadot-collator
USER polkadot-parachain
EXPOSE 30333 9933 9944 9615
VOLUME ["/data"]
ENTRYPOINT ["/usr/local/bin/polkadot-collator"]
ENTRYPOINT ["/usr/local/bin/polkadot-parachain"]
docker/scripts/build-injected-image.sh
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
OWNER=${OWNER:-parity}
IMAGE_NAME=${IMAGE_NAME:-polkadot-collator}
IMAGE_NAME=${IMAGE_NAME:-polkadot-parachain}
docker build --no-cache --build-arg IMAGE_NAME=$IMAGE_NAME -t $OWNER/$IMAGE_NAME -f ./docker/injected.Dockerfile .
docker images | grep $IMAGE_NAME
docker/scripts/inject_bootnodes.sh
+2 -2
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# this script runs the polkadot-collator after fetching
# this script runs the polkadot-parachain after fetching
# appropriate bootnode IDs
#
# this is _not_ a general-purpose script; it is closely tied to the
@@ -8,7 +8,7 @@
set -e -o pipefail
ctpc="/usr/bin/polkadot-collator"
ctpc="/usr/bin/polkadot-parachain"
if [ ! -x "$ctpc" ]; then
echo "FATAL: $ctpc does not exist or is not executable"
docker/test-parachain-collator.dockerfile
+4 -4
View File
@@ -4,7 +4,7 @@ FROM docker.io/paritytech/ci-linux:production as builder
WORKDIR /cumulus
COPY . /cumulus
RUN cargo build --release --locked -p polkadot-collator
RUN cargo build --release --locked -p polkadot-parachain
# the collator stage is normally built once, cached, and then ignored, but can
# be specified with the --target build flag. This adds some extra tooling to the
@@ -23,7 +23,7 @@ RUN apt-get update && apt-get install jq curl bash -y && \
npm install --global yarn && \
yarn global add @polkadot/api-cli@0.10.0-beta.14
COPY --from=builder \
/paritytech/cumulus/target/release/polkadot-collator /usr/bin
/paritytech/cumulus/target/release/polkadot-parachain /usr/bin
COPY ./docker/scripts/inject_bootnodes.sh /usr/bin
CMD ["/usr/bin/inject_bootnodes.sh"]
COPY ./docker/scripts/healthcheck.sh /usr/bin/
@@ -41,6 +41,6 @@ CMD ["cp", "-v", "/var/opt/cumulus_test_parachain_runtime.compact.wasm", "/runti
FROM debian:buster-slim
COPY --from=builder \
/paritytech/cumulus/target/release/polkadot-collator /usr/bin
/paritytech/cumulus/target/release/polkadot-parachain /usr/bin
CMD ["/usr/bin/polkadot-collator"]
CMD ["/usr/bin/polkadot-parachain"]
docker/test-parachain_injected.Dockerfile
+49
View File
@@ -0,0 +1,49 @@
FROM docker.io/library/ubuntu:20.04
# metadata
ARG VCS_REF
ARG BUILD_DATE
ARG IMAGE_NAME
LABEL io.parity.image.authors="devops-team@parity.io" \
io.parity.image.vendor="Parity Technologies" \
io.parity.image.title="${IMAGE_NAME}" \
io.parity.image.description="Test parachain for Zombienet" \
io.parity.image.source="https://github.com/paritytech/cumulus/blob/${VCS_REF}/docker/test-parachain_injected.Dockerfile" \
io.parity.image.revision="${VCS_REF}" \
io.parity.image.created="${BUILD_DATE}" \
io.parity.image.documentation="https://github.com/paritytech/cumulus/"
# show backtraces
ENV RUST_BACKTRACE 1
# install tools and dependencies
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
libssl1.1 \
ca-certificates \
curl && \
# apt cleanup
apt-get autoremove -y && \
apt-get clean && \
find /var/lib/apt/lists/ -type f -not -name lock -delete; \
# add user and link ~/.local/share/test-parachain to /data
useradd -m -u 10000 -U -s /bin/sh -d /test-parachain test-parachain && \
mkdir -p /data /test-parachain/.local/share && \
chown -R test-parachain:test-parachain /data && \
ln -s /data /test-parachain/.local/share/test-parachain && \
mkdir -p /specs
# add test-parachain binary to the docker image
COPY ./artifacts/test-parachain /usr/local/bin
COPY ./parachains/chain-specs/*.json /specs/
USER test-parachain
# check if executable works in this container
RUN /usr/local/bin/test-parachain --version
EXPOSE 30333 9933 9944
VOLUME ["/test-parachain"]
ENTRYPOINT ["/usr/local/bin/test-parachain"]
docs/overview.md
+1 -1
View File
@@ -53,7 +53,7 @@ the notification the so-called candidate message. This candidate message is issu
validator after approving a block. This proof of possible inclusion prevents spamming other collators
of the network with useless blocks.
The Collator joins the relay-chain network for two reasons. First, the Collator uses it to send the
Parachain blocks to the Parachain validators. Secondly, the Collator participates as light/full-node
Parachain blocks to the Parachain validators. Secondly, the Collator participates as a full-node
of the relay chain to be informed of new relay-chain blocks. This information will be used for the
consensus and the block production logic.
docs/release.md
+130
View File
@@ -0,0 +1,130 @@
# Releases
## Versioning
### Example #1
```
| Polkadot | v 0. 9.22 |
| Client | v 0. 9.22 0 |
| Runtime | v 9 22 0 | => 9220
| semver | 0. 9.22 0 |
```
### Example #2
```
| Polkadot | v 0.10.42 |
| Client | v 0.10.42 0 |
| Runtime | v 10.42 0 | => 10420
| semver | 0.10.42 0 |
```
### Example #3
```
| Polkadot | v 1. 2.18 |
| Client | v 1. 2.18 0 |
| Runtime | v 1 2 18 0 | => 102180
| semver | 1. 2.18 0 |
```
This document contains information related to the releasing process and describes a few of the steps and checks that are
performed during the release process.
## Client
### <a name="burnin"></a>Burn In
Ensure that Parity DevOps has run the new release on Westmint and Statemine collators for 12h prior to publishing the
release.
### Build Artifacts
Add any necessary assets to the release. They should include:
- Linux binaries
- GPG signature
- SHA256 checksum
- WASM binaries of the runtimes
- Source code
## Runtimes
### Spec Version
A new runtime release must bump the `spec_version`. This may follow a pattern with the client release (e.g. runtime
v9220 corresponds to v0.9.22).
### Runtime version bump between RCs
The clients need to be aware of runtime changes. However, we do not want to bump the `spec_version` for every single
release candidate. Instead, we can bump the `impl` field of the version to signal the change to the client. This applies
only to runtimes that have been deployed.
### Old Migrations Removed
Previous `on_runtime_upgrade` functions from old upgrades should be removed.
### New Migrations
Ensure that any migrations that are required due to storage or logic changes are included in the `on_runtime_upgrade`
function of the appropriate pallets.
### Extrinsic Ordering & Storage
Offline signing libraries depend on a consistent ordering of call indices and
functions. Compare the metadata of the current and new runtimes and ensure that
the `module index, call index` tuples map to the same set of functions. It also checks if there have been any changes in `storage`. In case of a breaking change, increase `transaction_version`.
To verify the order has not changed, manually start the following [Github Action](https://github.com/paritytech/cumulus/actions/workflows/extrinsic-ordering-check-from-bin.yml). It takes around a minute to run and will produce the report as artifact you need to manually check.
To run it, in the _Run Workflow_ dropdown:
1. **Use workflow from**: to ignore, leave `master` as default
2. **The WebSocket url of the reference node**:
- Statemint: `wss://statemint-rpc.polkadot.io`
- Statemine: `wss://statemine-rpc.polkadot.io`
- Westmint: `wss://westmint-rpc.polkadot.io`
3. **A url to a Linux binary for the node containing the runtime to test**: Paste the URL of the latest release-candidate binary from the draft-release on Github. The binary has to previously be uploaded to S3 (Github url link to the binary is constantly changing)
- E.g: https://releases.parity.io/cumulus/v0.9.270-rc3/polkadot-parachain
4. **The name of the chain under test. Usually, you would pass a local chain**:
- Statemint: `statemint-local`
- Statemine: `statemine-local`
- Westmint: `westmint-local`
5. Click **Run workflow**
When the workflow is done, click on it and download the zip artifact, inside you'll find an `output.txt` file. The things to look for in the output are lines like:
- `[Identity] idx 28 -> 25 (calls 15)` - indicates the index for Identity has changed
- `[+] Society, Recovery` - indicates the new version includes 2 additional modules/pallets.
- If no indices have changed, every modules line should look something like `[Identity] idx 25 (calls 15)`
**Note**: Adding new functions to the runtime does not constitute a breaking change
as long as the indexes did not change.
**Note**: Extrinsic function signatures changes (adding/removing & ordering arguments) are not caught by the job, so those changes should be reviewed "manually"
### Benchmarks
The Benchmarks can now be started from the CI. First find the CI pipeline from [here](https://gitlab.parity.io/parity/mirrors/cumulus/-/pipelines?page=1&scope=all&ref=release-parachains-v9220) and pick the latest.
[Guide](https://github.com/paritytech/ci_cd/wiki/Benchmarks:-cumulus)
### Integration Tests
Until https://github.com/paritytech/ci_cd/issues/499 is done, tests will have to be run manually.
1. Go to https://github.com/paritytech/parachains-integration-tests and checkout to the release branch.
E.g. https://github.com/paritytech/parachains-integration-tests/tree/release-v9270-v0.9.27
for `release-parachains-v0.9.270`
2. Clone `release-parachains-<version>` branch from Cumulus
3. `cargo build --release`
4. Copy `./target/polkadot-parachain` to `./bin`
5. Clone `it/release-<version>-fast-sudo` from Polkadot
In case the branch does not exists (it is a manual process): cherry pick paritytech/polkadot@791c8b8 and run
`find . -type f -name "*.toml" -print0 | xargs -0 sed -i '' -e 's/polkadot-vX.X.X/polkadot-v<version>/g'`
6. `cargo build --release features --fast-runtime`
7. Copy `./target/polkadot` into `./bin` (in Cumulus)
8. Run the tests:
- Statemint: `yarn zombienet-test -c ./examples/statemint/config.toml -t ./examples/statemint`
- Statemine: `yarn zombienet-test -c ./examples/statemine/config.toml -t ./examples/statemine`
pallets/aura-ext/Cargo.toml
+1 -1
View File
@@ -8,7 +8,7 @@ description = "AURA consensus extension pallet for parachains"
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false, features = ["derive"] }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
serde = { version = "1.0.132", optional = true, features = ["derive"] }
serde = { version = "1.0.143", optional = true, features = ["derive"] }
# Substrate
frame-executive = { git = "https://github.com/paritytech/substrate", default-features = false, branch = "master" }
pallets/aura-ext/src/lib.rs
+7 -8
View File
@@ -49,21 +49,19 @@ pub mod pallet {
use super::*;
use frame_support::pallet_prelude::*;
use frame_system::pallet_prelude::*;
use sp_std::vec::Vec;
/// The configuration trait.
#[pallet::config]
pub trait Config: pallet_aura::Config + frame_system::Config {}
#[pallet::pallet]
#[pallet::without_storage_info]
pub struct Pallet<T>(_);
#[pallet::hooks]
impl<T: Config> Hooks<BlockNumberFor<T>> for Pallet<T> {
fn on_finalize(_: BlockNumberFor<T>) {
// Update to the latest AuRa authorities.
Authorities::<T>::put(Aura::<T>::authorities().into_inner());
Authorities::<T>::put(Aura::<T>::authorities());
}
fn on_initialize(_: BlockNumberFor<T>) -> Weight {
@@ -74,16 +72,17 @@ pub mod pallet {
}
}
#[pallet::call]
impl<T: Config> Pallet<T> {}
/// Serves as cache for the authorities.
///
/// The authorities in AuRa are overwritten in `on_initialize` when we switch to a new session,
/// but we require the old authorities to verify the seal when validating a PoV. This will always
/// be updated to the latest AuRa authorities in `on_finalize`.
#[pallet::storage]
pub(crate) type Authorities<T: Config> = StorageValue<_, Vec<T::AuthorityId>, ValueQuery>;
pub(crate) type Authorities<T: Config> = StorageValue<
_,
BoundedVec<T::AuthorityId, <T as pallet_aura::Config>::MaxAuthorities>,
ValueQuery,
>;
#[pallet::genesis_config]
#[derive(Default)]
@@ -99,7 +98,7 @@ pub mod pallet {
"AuRa authorities empty, maybe wrong order in `construct_runtime!`?",
);
Authorities::<T>::put(authorities.into_inner());
Authorities::<T>::put(authorities);
}
}
}
pallets/collator-selection/Cargo.toml
+2 -2
View File
@@ -13,11 +13,11 @@ version = "3.0.0"
targets = ["x86_64-unknown-linux-gnu"]
[dependencies]
log = { version = "0.4.16", default-features = false }
log = { version = "0.4.17", default-features = false }
codec = { default-features = false, features = ["derive"], package = "parity-scale-codec", version = "3.0.0" }
rand = { version = "0.8.5", features = ["std_rng"], default-features = false }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
serde = { version = "1.0.132", default-features = false }
serde = { version = "1.0.143", default-features = false }
sp-std = { default-features = false, git = "https://github.com/paritytech/substrate", branch = "master" }
sp-runtime = { default-features = false, git = "https://github.com/paritytech/substrate", branch = "master" }
pallets/collator-selection/src/benchmarking.rs
+7 -7
View File
@@ -117,7 +117,7 @@ benchmarks! {
);
}
verify {
assert_last_event::<T>(Event::NewInvulnerables(new_invulnerables).into());
assert_last_event::<T>(Event::NewInvulnerables{invulnerables: new_invulnerables}.into());
}
set_desired_candidates {
@@ -129,19 +129,19 @@ benchmarks! {
);
}
verify {
assert_last_event::<T>(Event::NewDesiredCandidates(max).into());
assert_last_event::<T>(Event::NewDesiredCandidates{desired_candidates: max}.into());
}
set_candidacy_bond {
let bond: BalanceOf<T> = T::Currency::minimum_balance() * 10u32.into();
let bond_amount: BalanceOf<T> = T::Currency::minimum_balance() * 10u32.into();
let origin = T::UpdateOrigin::successful_origin();
}: {
assert_ok!(
<CollatorSelection<T>>::set_candidacy_bond(origin, bond.clone())
<CollatorSelection<T>>::set_candidacy_bond(origin, bond_amount.clone())
);
}
verify {
assert_last_event::<T>(Event::NewCandidacyBond(bond).into());
assert_last_event::<T>(Event::NewCandidacyBond{bond_amount}.into());
}
// worse case is when we have all the max-candidate slots filled except one, and we fill that
@@ -167,7 +167,7 @@ benchmarks! {
}: _(RawOrigin::Signed(caller.clone()))
verify {
assert_last_event::<T>(Event::CandidateAdded(caller, bond / 2u32.into()).into());
assert_last_event::<T>(Event::CandidateAdded{account_id: caller, deposit: bond / 2u32.into()}.into());
}
// worse case is the last candidate leaving.
@@ -183,7 +183,7 @@ benchmarks! {
whitelist!(leaving);
}: _(RawOrigin::Signed(leaving.clone()))
verify {
assert_last_event::<T>(Event::CandidateRemoved(leaving).into());
assert_last_event::<T>(Event::CandidateRemoved{account_id: leaving}.into());
}
// worse case is paying a non-existing candidate account.
pallets/collator-selection/src/lib.rs
+46 -41
View File
@@ -89,7 +89,7 @@ pub mod pallet {
ValidatorRegistration,
},
weights::DispatchClass,
PalletId,
BoundedVec, PalletId,
};
use frame_system::{pallet_prelude::*, Config as SystemConfig};
use pallet_session::SessionManager;
@@ -123,8 +123,7 @@ pub mod pallet {
/// Account Identifier from which the internal Pot is generated.
type PotId: Get<PalletId>;
/// Maximum number of candidates that we should have. This is used for benchmarking and is not
/// enforced.
/// Maximum number of candidates that we should have. This is enforced in code.
///
/// This does not take into account the invulnerables.
type MaxCandidates: Get<u32>;
@@ -134,9 +133,7 @@ pub mod pallet {
/// This does not take into account the invulnerables.
type MinCandidates: Get<u32>;
/// Maximum number of invulnerables.
///
/// Used only for benchmarking.
/// Maximum number of invulnerables. This is enforced in code.
type MaxInvulnerables: Get<u32>;
// Will be kicked if block is not produced in threshold.
@@ -158,7 +155,9 @@ pub mod pallet {
}
/// Basic information about a collation candidate.
#[derive(PartialEq, Eq, Clone, Encode, Decode, RuntimeDebug, scale_info::TypeInfo)]
#[derive(
PartialEq, Eq, Clone, Encode, Decode, RuntimeDebug, scale_info::TypeInfo, MaxEncodedLen,
)]
pub struct CandidateInfo<AccountId, Balance> {
/// Account identifier.
pub who: AccountId,
@@ -168,19 +167,22 @@ pub mod pallet {
#[pallet::pallet]
#[pallet::generate_store(pub(super) trait Store)]
#[pallet::without_storage_info]
pub struct Pallet<T>(_);
/// The invulnerable, fixed collators.
#[pallet::storage]
#[pallet::getter(fn invulnerables)]
pub type Invulnerables<T: Config> = StorageValue<_, Vec<T::AccountId>, ValueQuery>;
pub type Invulnerables<T: Config> =
StorageValue<_, BoundedVec<T::AccountId, T::MaxInvulnerables>, ValueQuery>;
/// The (community, limited) collation candidates.
#[pallet::storage]
#[pallet::getter(fn candidates)]
pub type Candidates<T: Config> =
StorageValue<_, Vec<CandidateInfo<T::AccountId, BalanceOf<T>>>, ValueQuery>;
pub type Candidates<T: Config> = StorageValue<
_,
BoundedVec<CandidateInfo<T::AccountId, BalanceOf<T>>, T::MaxCandidates>,
ValueQuery,
>;
/// Last block authored by collator.
#[pallet::storage]
@@ -230,10 +232,9 @@ pub mod pallet {
"duplicate invulnerables in genesis."
);
assert!(
T::MaxInvulnerables::get() >= (self.invulnerables.len() as u32),
"genesis invulnerables are more than T::MaxInvulnerables",
);
let bounded_invulnerables =
BoundedVec::<_, T::MaxInvulnerables>::try_from(self.invulnerables.clone())
.expect("genesis invulnerables are more than T::MaxInvulnerables");
assert!(
T::MaxCandidates::get() >= self.desired_candidates,
"genesis desired_candidates are more than T::MaxCandidates",
@@ -241,18 +242,18 @@ pub mod pallet {
<DesiredCandidates<T>>::put(&self.desired_candidates);
<CandidacyBond<T>>::put(&self.candidacy_bond);
<Invulnerables<T>>::put(&self.invulnerables);
<Invulnerables<T>>::put(bounded_invulnerables);
}
}
#[pallet::event]
#[pallet::generate_deposit(pub(super) fn deposit_event)]
pub enum Event<T: Config> {
NewInvulnerables(Vec<T::AccountId>),
NewDesiredCandidates(u32),
NewCandidacyBond(BalanceOf<T>),
CandidateAdded(T::AccountId, BalanceOf<T>),
CandidateRemoved(T::AccountId),
NewInvulnerables { invulnerables: Vec<T::AccountId> },
NewDesiredCandidates { desired_candidates: u32 },
NewCandidacyBond { bond_amount: BalanceOf<T> },
CandidateAdded { account_id: T::AccountId, deposit: BalanceOf<T> },
CandidateRemoved { account_id: T::AccountId },
}
// Errors inform users that something went wrong.
@@ -270,6 +271,8 @@ pub mod pallet {
AlreadyCandidate,
/// User is not a candidate
NotCandidate,
/// Too many invulnerables
TooManyInvulnerables,
/// User is already an Invulnerable
AlreadyInvulnerable,
/// Account has no associated validator ID
@@ -290,15 +293,11 @@ pub mod pallet {
new: Vec<T::AccountId>,
) -> DispatchResultWithPostInfo {
T::UpdateOrigin::ensure_origin(origin)?;
// we trust origin calls, this is just a for more accurate benchmarking
if (new.len() as u32) > T::MaxInvulnerables::get() {
log::warn!(
"invulnerables > T::MaxInvulnerables; you might need to run benchmarks again"
);
}
let bounded_invulnerables = BoundedVec::<_, T::MaxInvulnerables>::try_from(new)
.map_err(|_| Error::<T>::TooManyInvulnerables)?;
// check if the invulnerables have associated validator keys before they are set
for account_id in &new {
for account_id in bounded_invulnerables.iter() {
let validator_key = T::ValidatorIdOf::convert(account_id.clone())
.ok_or(Error::<T>::NoAssociatedValidatorId)?;
ensure!(
@@ -307,8 +306,10 @@ pub mod pallet {
);
}
<Invulnerables<T>>::put(&new);
Self::deposit_event(Event::NewInvulnerables(new));
<Invulnerables<T>>::put(&bounded_invulnerables);
Self::deposit_event(Event::NewInvulnerables {
invulnerables: bounded_invulnerables.to_vec(),
});
Ok(().into())
}
@@ -326,7 +327,7 @@ pub mod pallet {
log::warn!("max > T::MaxCandidates; you might need to run benchmarks again");
}
<DesiredCandidates<T>>::put(&max);
Self::deposit_event(Event::NewDesiredCandidates(max));
Self::deposit_event(Event::NewDesiredCandidates { desired_candidates: max });
Ok(().into())
}
@@ -338,7 +339,7 @@ pub mod pallet {
) -> DispatchResultWithPostInfo {
T::UpdateOrigin::ensure_origin(origin)?;
<CandidacyBond<T>>::put(&bond);
Self::deposit_event(Event::NewCandidacyBond(bond));
Self::deposit_event(Event::NewCandidacyBond { bond_amount: bond });
Ok(().into())
}
@@ -372,7 +373,7 @@ pub mod pallet {
Err(Error::<T>::AlreadyCandidate)?
} else {
T::Currency::reserve(&who, deposit)?;
candidates.push(incoming);
candidates.try_push(incoming).map_err(|_| Error::<T>::TooManyCandidates)?;
<LastAuthoredBlock<T>>::insert(
who.clone(),
frame_system::Pallet::<T>::block_number() + T::KickThreshold::get(),
@@ -381,7 +382,7 @@ pub mod pallet {
}
})?;
Self::deposit_event(Event::CandidateAdded(who, deposit));
Self::deposit_event(Event::CandidateAdded { account_id: who, deposit });
Ok(Some(T::WeightInfo::register_as_candidate(current_count as u32)).into())
}
@@ -407,7 +408,7 @@ pub mod pallet {
impl<T: Config> Pallet<T> {
/// Get a unique, inaccessible account id from the `PotId`.
pub fn account_id() -> T::AccountId {
T::PotId::get().into_account()
T::PotId::get().into_account_truncating()
}
/// Removes a candidate if they exist and sends them back their deposit
@@ -423,15 +424,17 @@ pub mod pallet {
<LastAuthoredBlock<T>>::remove(who.clone());
Ok(candidates.len())
})?;
Self::deposit_event(Event::CandidateRemoved(who.clone()));
Self::deposit_event(Event::CandidateRemoved { account_id: who.clone() });
Ok(current_count)
}
/// Assemble the current set of candidates and invulnerables into the next collator set.
///
/// This is done on the fly, as frequent as we are told to do so, as the session manager.
pub fn assemble_collators(candidates: Vec<T::AccountId>) -> Vec<T::AccountId> {
let mut collators = Self::invulnerables();
pub fn assemble_collators(
candidates: BoundedVec<T::AccountId, T::MaxCandidates>,
) -> Vec<T::AccountId> {
let mut collators = Self::invulnerables().to_vec();
collators.extend(candidates);
collators
}
@@ -439,8 +442,8 @@ pub mod pallet {
/// Kicks out candidates that did not produce a block in the kick threshold
/// and refund their deposits.
pub fn kick_stale_candidates(
candidates: Vec<CandidateInfo<T::AccountId, BalanceOf<T>>>,
) -> Vec<T::AccountId> {
candidates: BoundedVec<CandidateInfo<T::AccountId, BalanceOf<T>>, T::MaxCandidates>,
) -> BoundedVec<T::AccountId, T::MaxCandidates> {
let now = frame_system::Pallet::<T>::block_number();
let kick_threshold = T::KickThreshold::get();
candidates
@@ -461,7 +464,9 @@ pub mod pallet {
None
}
})
.collect()
.collect::<Vec<_>>()
.try_into()
.expect("filter_map operation can't result in a bounded vec larger than its original; qed")
}
}
pallets/dmp-queue/Cargo.toml
+1 -1
View File
@@ -6,7 +6,7 @@ edition = "2021"
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0", features = [ "derive" ], default-features = false }
log = { version = "0.4.16", default-features = false }
log = { version = "0.4.17", default-features = false }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
# Substrate
pallets/dmp-queue/src/lib.rs
+34 -30
View File
@@ -149,11 +149,11 @@ pub mod pallet {
T::ExecuteOverweightOrigin::ensure_origin(origin)?;
let (sent_at, data) = Overweight::<T>::get(index).ok_or(Error::<T>::Unknown)?;
let used = Self::try_service_message(weight_limit, sent_at, &data[..])
let weight_used = Self::try_service_message(weight_limit, sent_at, &data[..])
.map_err(|_| Error::<T>::OverLimit)?;
Overweight::<T>::remove(index);
Self::deposit_event(Event::OverweightServiced(index, used));
Ok(Some(used.saturating_add(1_000_000)).into())
Self::deposit_event(Event::OverweightServiced { overweight_index: index, weight_used });
Ok(Some(weight_used.saturating_add(1_000_000)).into())
}
}
@@ -161,23 +161,21 @@ pub mod pallet {
#[pallet::generate_deposit(pub(super) fn deposit_event)]
pub enum Event<T: Config> {
/// Downward message is invalid XCM.
/// \[ id \]
InvalidFormat(MessageId),
InvalidFormat { message_id: MessageId },
/// Downward message is unsupported version of XCM.
/// \[ id \]
UnsupportedVersion(MessageId),
UnsupportedVersion { message_id: MessageId },
/// Downward message executed with the given outcome.
/// \[ id, outcome \]
ExecutedDownward(MessageId, Outcome),
ExecutedDownward { message_id: MessageId, outcome: Outcome },
/// The weight limit for handling downward messages was reached.
/// \[ id, remaining, required \]
WeightExhausted(MessageId, Weight, Weight),
WeightExhausted { message_id: MessageId, remaining_weight: Weight, required_weight: Weight },
/// Downward message is overweight and was placed in the overweight queue.
/// \[ id, index, required \]
OverweightEnqueued(MessageId, OverweightIndex, Weight),
OverweightEnqueued {
message_id: MessageId,
overweight_index: OverweightIndex,
required_weight: Weight,
},
/// Downward message from the overweight queue was executed.
/// \[ index, used \]
OverweightServiced(OverweightIndex, Weight),
OverweightServiced { overweight_index: OverweightIndex, weight_used: Weight },
}
impl<T: Config> Pallet<T> {
@@ -225,7 +223,7 @@ pub mod pallet {
_sent_at: RelayBlockNumber,
mut data: &[u8],
) -> Result<Weight, (MessageId, Weight)> {
let id = sp_io::hashing::blake2_256(data);
let message_id = sp_io::hashing::blake2_256(data);
let maybe_msg = VersionedXcm::<T::Call>::decode_all_with_depth_limit(
MAX_XCM_DECODE_DEPTH,
&mut data,
@@ -233,21 +231,21 @@ pub mod pallet {
.map(Xcm::<T::Call>::try_from);
match maybe_msg {
Err(_) => {
Self::deposit_event(Event::InvalidFormat(id));
Self::deposit_event(Event::InvalidFormat { message_id });
Ok(0)
},
Ok(Err(())) => {
Self::deposit_event(Event::UnsupportedVersion(id));
Self::deposit_event(Event::UnsupportedVersion { message_id });
Ok(0)
},
Ok(Ok(x)) => {
let outcome = T::XcmExecutor::execute_xcm(Parent, x, id, limit);
match outcome {
Outcome::Error(XcmError::WeightLimitReached(required)) =>
Err((id, required)),
Err((message_id, required)),
outcome => {
let weight_used = outcome.weight_used();
Self::deposit_event(Event::ExecutedDownward(id, outcome));
Self::deposit_event(Event::ExecutedDownward { message_id, outcome });
Ok(weight_used)
},
}
@@ -283,18 +281,22 @@ pub mod pallet {
for (i, (sent_at, data)) in iter.enumerate() {
if maybe_enqueue_page.is_none() {
// We're not currently enqueuing - try to execute inline.
let remaining = limit.saturating_sub(used);
match Self::try_service_message(remaining, sent_at, &data[..]) {
let remaining_weight = limit.saturating_sub(used);
match Self::try_service_message(remaining_weight, sent_at, &data[..]) {
Ok(consumed) => used += consumed,
Err((id, required)) =>
Err((message_id, required_weight)) =>
// Too much weight required right now.
{
if required > config.max_individual {
if required_weight > config.max_individual {
// overweight - add to overweight queue and continue with
// message execution.
let index = page_index.overweight_count;
Overweight::<T>::insert(index, (sent_at, data));
Self::deposit_event(Event::OverweightEnqueued(id, index, required));
let overweight_index = page_index.overweight_count;
Overweight::<T>::insert(overweight_index, (sent_at, data));
Self::deposit_event(Event::OverweightEnqueued {
message_id,
overweight_index,
required_weight,
});
page_index.overweight_count += 1;
// Not needed for control flow, but only to ensure that the compiler
// understands that we won't attempt to re-use `data` later.
@@ -304,9 +306,11 @@ pub mod pallet {
// from here on.
let item_count_left = item_count.saturating_sub(i);
maybe_enqueue_page = Some(Vec::with_capacity(item_count_left));
Self::deposit_event(Event::WeightExhausted(
id, remaining, required,
));
Self::deposit_event(Event::WeightExhausted {
message_id,
remaining_weight,
required_weight,
});
}
},
}
pallets/parachain-system/Cargo.toml
+7 -2
View File
@@ -6,12 +6,13 @@ edition = "2021"
description = "Base pallet for cumulus-based parachains"
[dependencies]
bytes = { version = "1.2.1", default-features = false }
codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false, features = ["derive"] }
environmental = { version = "1.1.2", default-features = false }
impl-trait-for-tuples = "0.2.1"
log = { version = "0.4.16", default-features = false }
log = { version = "0.4.17", default-features = false }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
serde = { version = "1.0.132", optional = true, features = ["derive"] }
serde = { version = "1.0.143", optional = true, features = ["derive"] }
# Substrate
frame-support = { git = "https://github.com/paritytech/substrate", default-features = false, branch = "master" }
@@ -73,3 +74,7 @@ std = [
"sp-trie/std",
"xcm/std"
]
runtime-benchmarks = [
"sp-runtime/runtime-benchmarks"
]
pallets/parachain-system/proc-macro/Cargo.toml
+4 -4
View File
@@ -9,10 +9,10 @@ description = "Proc macros provided by the parachain-system pallet"
proc-macro = true
[dependencies]
syn = "1.0.91"
proc-macro2 = "1.0.37"
quote = "1.0.18"
proc-macro-crate = "1.1.3"
syn = "1.0.99"
proc-macro2 = "1.0.43"
quote = "1.0.21"
proc-macro-crate = "1.2.1"
[features]
default = [ "std" ]
pallets/parachain-system/src/lib.rs
+88 -12
View File
@@ -88,6 +88,51 @@ pub use relay_state_snapshot::{MessagingStateSnapshot, RelayChainStateProof};
pub use pallet::*;
/// Something that can check the associated relay block number.
///
/// Each Parachain block is built in the context of a relay chain block, this trait allows us
/// to validate the given relay chain block number. With async backing it is legal to build
/// multiple Parachain blocks per relay chain parent. With this trait it is possible for the
/// Parachain to ensure that still only one Parachain block is build per relay chain parent.
///
/// By default [`RelayNumberStrictlyIncreases`] and [`AnyRelayNumber`] are provided.
pub trait CheckAssociatedRelayNumber {
/// Check the current relay number versus the previous relay number.
///
/// The implementation should panic when there is something wrong.
fn check_associated_relay_number(
current: RelayChainBlockNumber,
previous: RelayChainBlockNumber,
);
}
/// Provides an implementation of [`CheckAssociatedRelayNumber`].
///
/// It will ensure that the associated relay block number strictly increases between Parachain
/// blocks. This should be used by production Parachains when in doubt.
pub struct RelayNumberStrictlyIncreases;
impl CheckAssociatedRelayNumber for RelayNumberStrictlyIncreases {
fn check_associated_relay_number(
current: RelayChainBlockNumber,
previous: RelayChainBlockNumber,
) {
if current <= previous {
panic!("Relay chain block number needs to strictly increase between Parachain blocks!")
}
}
}
/// Provides an implementation of [`CheckAssociatedRelayNumber`].
///
/// This will accept any relay chain block number combination. This is mainly useful for
/// test parachains.
pub struct AnyRelayNumber;
impl CheckAssociatedRelayNumber for AnyRelayNumber {
fn check_associated_relay_number(_: RelayChainBlockNumber, _: RelayChainBlockNumber) {}
}
#[frame_support::pallet]
pub mod pallet {
use super::*;
@@ -128,6 +173,9 @@ pub mod pallet {
/// The weight we reserve at the beginning of the block for processing XCMP messages.
type ReservedXcmpWeight: Get<Weight>;
/// Something that can check the associated relay parent block number.
type CheckAssociatedRelayNumber: CheckAssociatedRelayNumber;
}
#[pallet::hooks]
@@ -232,7 +280,7 @@ pub mod pallet {
}
// Remove the validation from the old block.
<ValidationData<T>>::kill();
ValidationData::<T>::kill();
ProcessedDownwardMessages::<T>::kill();
HrmpWatermark::<T>::kill();
UpwardMessages::<T>::kill();
@@ -307,6 +355,13 @@ pub mod pallet {
Self::validate_validation_data(&vfp);
// Check that the associated relay chain block number is as expected.
T::CheckAssociatedRelayNumber::check_associated_relay_number(
vfp.relay_parent_number,
LastRelayChainBlockNumber::<T>::get(),
);
LastRelayChainBlockNumber::<T>::put(vfp.relay_parent_number);
let relay_state_proof = RelayChainStateProof::new(
T::SelfParaId::get(),
vfp.relay_parent_storage_root,
@@ -330,7 +385,9 @@ pub mod pallet {
Self::put_parachain_code(&validation_code);
<T::OnSystemEvent as OnSystemEvent>::on_validation_code_applied();
Self::deposit_event(Event::ValidationFunctionApplied(vfp.relay_parent_number));
Self::deposit_event(Event::ValidationFunctionApplied {
relay_chain_block_num: vfp.relay_parent_number,
});
},
Some(relay_chain::v2::UpgradeGoAhead::Abort) => {
<PendingValidationCode<T>>::kill();
@@ -389,7 +446,7 @@ pub mod pallet {
AuthorizedUpgrade::<T>::put(&code_hash);
Self::deposit_event(Event::UpgradeAuthorized(code_hash));
Self::deposit_event(Event::UpgradeAuthorized { code_hash });
Ok(())
}
@@ -411,17 +468,15 @@ pub mod pallet {
/// The validation function has been scheduled to apply.
ValidationFunctionStored,
/// The validation function was applied as of the contained relay chain block number.
ValidationFunctionApplied(RelayChainBlockNumber),
ValidationFunctionApplied { relay_chain_block_num: RelayChainBlockNumber },
/// The relay-chain aborted the upgrade process.
ValidationFunctionDiscarded,
/// An upgrade has been authorized.
UpgradeAuthorized(T::Hash),
UpgradeAuthorized { code_hash: T::Hash },
/// Some downward messages have been received and will be processed.
/// \[ count \]
DownwardMessagesReceived(u32),
DownwardMessagesReceived { count: u32 },
/// Downward messages were processed using the given weight.
/// \[ weight_used, result_mqc_head \]
DownwardMessagesProcessed(Weight, relay_chain::Hash),
DownwardMessagesProcessed { weight_used: Weight, dmq_head: relay_chain::Hash },
}
#[pallet::error]
@@ -474,6 +529,11 @@ pub mod pallet {
#[pallet::storage]
pub(super) type DidSetValidationCode<T: Config> = StorageValue<_, bool, ValueQuery>;
/// The relay chain block number associated with the last parachain block.
#[pallet::storage]
pub(super) type LastRelayChainBlockNumber<T: Config> =
StorageValue<_, RelayChainBlockNumber, ValueQuery>;
/// An option which indicates if the relay-chain restricts signalling a validation code upgrade.
/// In other words, if this is `Some` and [`NewValidationCode`] is `Some` then the produced
/// candidate will be invalid.
@@ -750,7 +810,7 @@ impl<T: Config> Pallet<T> {
let mut weight_used = 0;
if dm_count != 0 {
Self::deposit_event(Event::DownwardMessagesReceived(dm_count));
Self::deposit_event(Event::DownwardMessagesReceived { count: dm_count });
let max_weight =
<ReservedDmpWeightOverride<T>>::get().unwrap_or_else(T::ReservedDmpWeight::get);
@@ -763,7 +823,10 @@ impl<T: Config> Pallet<T> {
weight_used += T::DmpMessageHandler::handle_dmp_messages(message_iter, max_weight);
<LastDmqMqcHead<T>>::put(&dmq_head);
Self::deposit_event(Event::DownwardMessagesProcessed(weight_used, dmq_head.head()));
Self::deposit_event(Event::DownwardMessagesProcessed {
weight_used,
dmq_head: dmq_head.head(),
});
}
// After hashing each message in the message queue chain submitted by the collator, we
@@ -835,7 +898,7 @@ impl<T: Config> Pallet<T> {
running_mqc_heads
.entry(sender)
.or_insert_with(|| last_mqc_heads.get(&sender).cloned().unwrap_or_default())
.or_insert_with(|| last_mqc_heads.get(sender).cloned().unwrap_or_default())
.extend_hrmp(horizontal_message);
}
}
@@ -1047,4 +1110,17 @@ impl<T: Config> BlockNumberProvider for RelaychainBlockNumberProvider<T> {
.map(|d| d.relay_parent_number)
.unwrap_or_default()
}
#[cfg(feature = "runtime-benchmarks")]
fn set_block_number(block: Self::BlockNumber) {
let mut validation_data = Pallet::<T>::validation_data().unwrap_or_else(||
// PersistedValidationData does not impl default in non-std
PersistedValidationData {
parent_head: vec![].into(),
relay_parent_number: Default::default(),
max_pov_size: Default::default(),
relay_parent_storage_root: Default::default(),
});
validation_data.relay_parent_number = block;
ValidationData::<T>::put(validation_data)
}
}
pallets/parachain-system/src/migration.rs
+2
View File
@@ -41,9 +41,11 @@ pub fn on_runtime_upgrade<T: Config>() -> Weight {
/// mechanism now uses signals instead of block offsets.
mod v1 {
use crate::{Config, Pallet};
#[allow(deprecated)]
use frame_support::{migration::remove_storage_prefix, pallet_prelude::*};
pub fn migrate<T: Config>() -> Weight {
#[allow(deprecated)]
remove_storage_prefix(<Pallet<T>>::name().as_bytes(), b"LastUpgrade", b"");
T::DbWeight::get().writes(1)
}
pallets/parachain-system/src/tests.rs
+19 -6
View File
@@ -107,6 +107,7 @@ impl Config for Test {
type ReservedDmpWeight = ReservedDmpWeight;
type XcmpMessageHandler = SaveIntoThreadLocal;
type ReservedXcmpWeight = ReservedXcmpWeight;
type CheckAssociatedRelayNumber = RelayNumberStrictlyIncreases;
}
pub struct FromThreadLocal;
@@ -226,8 +227,7 @@ struct BlockTests {
ran: bool,
relay_sproof_builder_hook:
Option<Box<dyn Fn(&BlockTests, RelayChainBlockNumber, &mut RelayStateSproofBuilder)>>,
persisted_validation_data_hook:
Option<Box<dyn Fn(&BlockTests, RelayChainBlockNumber, &mut PersistedValidationData)>>,
persisted_validation_data_hook: Option<Box<dyn Fn(&BlockTests, &mut PersistedValidationData)>>,
inherent_data_hook:
Option<Box<dyn Fn(&BlockTests, RelayChainBlockNumber, &mut ParachainInherentData)>>,
}
@@ -274,10 +274,9 @@ impl BlockTests {
self
}
#[allow(dead_code)] // might come in handy in future. If now is future and it still hasn't - feel free.
fn with_validation_data<F>(mut self, f: F) -> Self
where
F: 'static + Fn(&BlockTests, RelayChainBlockNumber, &mut PersistedValidationData),
F: 'static + Fn(&BlockTests, &mut PersistedValidationData),
{
self.persisted_validation_data_hook = Some(Box::new(f));
self
@@ -319,7 +318,7 @@ impl BlockTests {
..Default::default()
};
if let Some(ref hook) = self.persisted_validation_data_hook {
hook(self, *n as RelayChainBlockNumber, &mut vfp);
hook(self, &mut vfp);
}
<ValidationData<Test>>::put(&vfp);
@@ -415,7 +414,10 @@ fn events() {
let events = System::events();
assert_eq!(
events[0].event,
Event::ParachainSystem(crate::Event::ValidationFunctionApplied(1234).into())
Event::ParachainSystem(
crate::Event::ValidationFunctionApplied { relay_chain_block_num: 1234 }
.into()
)
);
},
);
@@ -961,3 +963,14 @@ fn receive_hrmp_after_pause() {
});
});
}
#[test]
#[should_panic = "Relay chain block number needs to strictly increase between Parachain blocks!"]
fn test() {
BlockTests::new()
.with_validation_data(|_, data| {
data.relay_parent_number = 1;
})
.add(1, || {})
.add(2, || {});
}
pallets/parachain-system/src/validate_block/implementation.rs
+7 -25
View File
@@ -67,8 +67,8 @@ where
assert!(parent_head.hash() == *block.header().parent_hash(), "Invalid parent hash",);
// Create the db
let (db, _root) = match storage_proof.to_memory_db(Some(parent_head.state_root())) {
Ok((db, root)) => (db, root),
let db = match storage_proof.to_memory_db(Some(parent_head.state_root())) {
Ok((db, _)) => db,
Err(_) => panic!("Compact proof decoding failure."),
};
@@ -205,8 +205,8 @@ fn host_storage_set(key: &[u8], value: &[u8]) {
with_externalities(|ext| ext.place_storage(key.to_vec(), Some(value.to_vec())))
}
fn host_storage_get(key: &[u8]) -> Option<Vec<u8>> {
with_externalities(|ext| ext.storage(key).clone())
fn host_storage_get(key: &[u8]) -> Option<bytes::Bytes> {
with_externalities(|ext| ext.storage(key).map(|value| value.into()))
}
fn host_storage_exists(key: &[u8]) -> bool {
@@ -222,13 +222,7 @@ fn host_storage_root(version: StateVersion) -> Vec<u8> {
}
fn host_storage_clear_prefix(prefix: &[u8], limit: Option<u32>) -> KillStorageResult {
with_externalities(|ext| {
let (all_removed, num_removed) = ext.clear_prefix(prefix, limit);
match all_removed {
true => KillStorageResult::AllRemoved(num_removed),
false => KillStorageResult::SomeRemaining(num_removed),
}
})
with_externalities(|ext| ext.clear_prefix(prefix, limit, None).into())
}
fn host_storage_append(key: &[u8], value: Vec<u8>) {
@@ -294,13 +288,7 @@ fn host_default_child_storage_storage_kill(
limit: Option<u32>,
) -> KillStorageResult {
let child_info = ChildInfo::new_default(storage_key);
with_externalities(|ext| {
let (all_removed, num_removed) = ext.kill_child_storage(&child_info, limit);
match all_removed {
true => KillStorageResult::AllRemoved(num_removed),
false => KillStorageResult::SomeRemaining(num_removed),
}
})
with_externalities(|ext| ext.kill_child_storage(&child_info, limit, None).into())
}
fn host_default_child_storage_exists(storage_key: &[u8], key: &[u8]) -> bool {
@@ -314,13 +302,7 @@ fn host_default_child_storage_clear_prefix(
limit: Option<u32>,
) -> KillStorageResult {
let child_info = ChildInfo::new_default(storage_key);
with_externalities(|ext| {
let (all_removed, num_removed) = ext.clear_child_prefix(&child_info, prefix, limit);
match all_removed {
true => KillStorageResult::AllRemoved(num_removed),
false => KillStorageResult::SomeRemaining(num_removed),
}
})
with_externalities(|ext| ext.clear_child_prefix(&child_info, prefix, limit, None).into())
}
fn host_default_child_storage_root(storage_key: &[u8], version: StateVersion) -> Vec<u8> {
pallets/session-benchmarking/Cargo.toml
+1 -1
View File
@@ -13,7 +13,7 @@ readme = "README.md"
targets = ["x86_64-unknown-linux-gnu"]
[dependencies]
parity-scale-codec = { version = "3.1.2", default-features = false }
parity-scale-codec = { version = "3.1.5", default-features = false }
sp-std = { default-features = false, git = "https://github.com/paritytech/substrate", branch = "master" }
sp-runtime = { default-features = false, git = "https://github.com/paritytech/substrate", branch = "master" }
frame-support = { default-features = false, git = "https://github.com/paritytech/substrate", branch = "master" }
pallets/xcm/Cargo.toml
+1 -1
View File
@@ -7,7 +7,7 @@ version = "0.1.0"
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false, features = ["derive"] }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
serde = { version = "1.0.132", optional = true, features = ["derive"] }
serde = { version = "1.0.143", optional = true, features = ["derive"] }
sp-std = { git = "https://github.com/paritytech/substrate", default-features = false, branch = "master" }
sp-io = { git = "https://github.com/paritytech/substrate", default-features = false, branch = "master" }
pallets/xcm/src/lib.rs
+2 -3
View File
@@ -27,7 +27,7 @@ use cumulus_primitives_core::{
use frame_support::dispatch::Weight;
pub use pallet::*;
use scale_info::TypeInfo;
use sp_runtime::traits::BadOrigin;
use sp_runtime::{traits::BadOrigin, RuntimeDebug};
use sp_std::{convert::TryFrom, prelude::*};
use xcm::{
latest::{ExecuteXcm, Outcome, Parent, Xcm},
@@ -77,8 +77,7 @@ pub mod pallet {
}
/// Origin for the parachains module.
#[derive(PartialEq, Eq, Clone, Encode, Decode, TypeInfo)]
#[cfg_attr(feature = "std", derive(Debug))]
#[derive(PartialEq, Eq, Clone, Encode, Decode, TypeInfo, RuntimeDebug, MaxEncodedLen)]
#[pallet::origin]
pub enum Origin {
/// It comes from the (parent) relay chain.
pallets/xcmp-queue/Cargo.toml
+1 -1
View File
@@ -6,7 +6,7 @@ edition = "2021"
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0", features = [ "derive" ], default-features = false }
log = { version = "0.4.16", default-features = false }
log = { version = "0.4.17", default-features = false }
rand_chacha = { version = "0.3.0", default-features = false }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
pallets/xcmp-queue/src/lib.rs
+26 -15
View File
@@ -152,7 +152,7 @@ pub mod pallet {
let used = Self::handle_xcm_message(sender, sent_at, xcm, weight_limit)
.map_err(|_| Error::<T>::WeightOverLimit)?;
Overweight::<T>::remove(index);
Self::deposit_event(Event::OverweightServiced(index, used));
Self::deposit_event(Event::OverweightServiced { index, used });
Ok(Some(used.saturating_add(1_000_000)).into())
}
@@ -267,21 +267,26 @@ pub mod pallet {
#[pallet::generate_deposit(pub(super) fn deposit_event)]
pub enum Event<T: Config> {
/// Some XCM was executed ok.
Success(Option<XcmHash>),
Success { message_hash: Option<XcmHash>, weight: Weight },
/// Some XCM failed.
Fail(Option<XcmHash>, XcmError),
Fail { message_hash: Option<XcmHash>, error: XcmError, weight: Weight },
/// Bad XCM version used.
BadVersion(Option<XcmHash>),
BadVersion { message_hash: Option<XcmHash> },
/// Bad XCM format used.
BadFormat(Option<XcmHash>),
BadFormat { message_hash: Option<XcmHash> },
/// An upward message was sent to the relay chain.
UpwardMessageSent(Option<XcmHash>),
UpwardMessageSent { message_hash: Option<XcmHash> },
/// An HRMP message was sent to a sibling parachain.
XcmpMessageSent(Option<XcmHash>),
XcmpMessageSent { message_hash: Option<XcmHash> },
/// An XCM exceeded the individual message weight budget.
OverweightEnqueued(ParaId, RelayBlockNumber, OverweightIndex, Weight),
OverweightEnqueued {
sender: ParaId,
sent_at: RelayBlockNumber,
index: OverweightIndex,
required: Weight,
},
/// An XCM from the overweight queue was executed with the given actual weight used.
OverweightServiced(OverweightIndex, Weight),
OverweightServiced { index: OverweightIndex, used: Weight },
}
#[pallet::error]
@@ -603,15 +608,20 @@ impl<T: Config> Pallet<T> {
let (result, event) = match Xcm::<T::Call>::try_from(xcm) {
Ok(xcm) => {
let location = (Parent, Parachain(sender.into()));
match T::XcmExecutor::execute_xcm(location, xcm, hash, max_weight) {
Outcome::Error(e) => (Err(e), Event::Fail(Some(hash), e)),
Outcome::Complete(w) => (Ok(w), Event::Success(Some(hash))),
Outcome::Error(e) =>
(Err(e), Event::Fail { message_hash: Some(hash), error: e, weight: 0 }),
Outcome::Complete(w) =>
(Ok(w), Event::Success { message_hash: Some(hash), weight: w }),
// As far as the caller is concerned, this was dispatched without error, so
// we just report the weight used.
Outcome::Incomplete(w, e) => (Ok(w), Event::Fail(Some(hash), e)),
Outcome::Incomplete(w, e) =>
(Ok(w), Event::Fail { message_hash: Some(hash), error: e, weight: w }),
}
},
Err(()) => (Err(XcmError::UnhandledXcmVersion), Event::BadVersion(Some(hash))),
Err(()) =>
(Err(XcmError::UnhandledXcmVersion), Event::BadVersion { message_hash: Some(hash) }),
};
Self::deposit_event(event);
result
@@ -648,7 +658,8 @@ impl<T: Config> Pallet<T> {
.saturating_sub(remaining_fragments.len());
let overweight_xcm = last_remaining_fragments[..msg_len].to_vec();
let index = Self::stash_overweight(sender, sent_at, overweight_xcm);
let e = Event::OverweightEnqueued(sender, sent_at, index, required);
let e =
Event::OverweightEnqueued { sender, sent_at, index, required };
Self::deposit_event(e);
},
Err(XcmError::WeightLimitReached(required))
@@ -1134,7 +1145,7 @@ impl<T: Config> SendXcm for Pallet<T> {
match Self::send_fragment(id, XcmpMessageFormat::ConcatenatedVersionedXcm, xcm) {
Ok(_) => {
Self::deposit_event(Event::XcmpMessageSent(Some(hash)));
Self::deposit_event(Event::XcmpMessageSent { message_hash: Some(hash) });
Ok(hash)
},
Err(e) => Err(SendError::Transport(<&'static str>::from(e))),
pallets/xcmp-queue/src/mock.rs
+2
View File
@@ -16,6 +16,7 @@
use super::*;
use crate as xcmp_queue;
use core::marker::PhantomData;
use cumulus_pallet_parachain_system::AnyRelayNumber;
use cumulus_primitives_core::{IsSystem, ParaId};
use frame_support::{
parameter_types,
@@ -110,6 +111,7 @@ impl cumulus_pallet_parachain_system::Config for Test {
type ReservedDmpWeight = ();
type XcmpMessageHandler = XcmpQueue;
type ReservedXcmpWeight = ();
type CheckAssociatedRelayNumber = AnyRelayNumber;
}
parameter_types! {
parachain-template/node/Cargo.toml
+6 -8
View File
@@ -9,18 +9,14 @@ repository = "https://github.com/paritytech/cumulus/"
edition = "2021"
build = "build.rs"
[[bin]]
name = "parachain-collator"
path = "src/main.rs"
[dependencies]
clap = { version = "3.1", features = ["derive"] }
clap = { version = "3.2.16", features = ["derive"] }
derive_more = "0.99.2"
log = "0.4.16"
log = "0.4.17"
codec = { package = "parity-scale-codec", version = "3.0.0" }
serde = { version = "1.0.132", features = ["derive"] }
serde = { version = "1.0.143", features = ["derive"] }
hex-literal = "0.3.4"
jsonrpc-core = "18.0.0"
jsonrpsee = { version = "0.15.1", features = ["server"] }
# Local
parachain-template-runtime = { path = "../runtime" }
@@ -37,9 +33,11 @@ sc-consensus = { git = "https://github.com/paritytech/substrate", branch = "mast
sc-executor = { git = "https://github.com/paritytech/substrate", branch = "master", features = ["wasmtime"] }
sc-keystore = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-network = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-network-common = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-rpc = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-rpc-api = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-service = { git = "https://github.com/paritytech/substrate", branch = "master", features = ["wasmtime"] }
sc-sysinfo = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-telemetry = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-tracing = { git = "https://github.com/paritytech/substrate", branch = "master" }
sc-transaction-pool = { git = "https://github.com/paritytech/substrate", branch = "master" }
parachain-template/node/src/cli.rs
+21 -47
View File
@@ -1,18 +1,8 @@
use crate::chain_spec;
use clap::Parser;
use std::path::PathBuf;
/// Sub-commands supported by the collator.
#[derive(Debug, clap::Subcommand)]
pub enum Subcommand {
/// Export the genesis state of the parachain.
#[clap(name = "export-genesis-state")]
ExportGenesisState(ExportGenesisStateCommand),
/// Export the genesis wasm of the parachain.
#[clap(name = "export-genesis-wasm")]
ExportGenesisWasm(ExportGenesisWasmCommand),
/// Build a chain specification.
BuildSpec(sc_cli::BuildSpecCmd),
@@ -28,11 +18,17 @@ pub enum Subcommand {
/// Import blocks.
ImportBlocks(sc_cli::ImportBlocksCmd),
/// Revert the chain to a previous state.
Revert(sc_cli::RevertCmd),
/// Remove the whole chain.
PurgeChain(cumulus_client_cli::PurgeChainCmd),
/// Revert the chain to a previous state.
Revert(sc_cli::RevertCmd),
/// Export the genesis state of the parachain.
ExportGenesisState(cumulus_client_cli::ExportGenesisStateCommand),
/// Export the genesis wasm of the parachain.
ExportGenesisWasm(cumulus_client_cli::ExportGenesisWasmCommand),
/// Sub-commands concerned with benchmarking.
/// The pallet benchmarking moved to the `pallet` sub-command.
@@ -43,39 +39,7 @@ pub enum Subcommand {
TryRuntime(try_runtime_cli::TryRuntimeCmd),
}
/// Command for exporting the genesis state of the parachain
#[derive(Debug, Parser)]
pub struct ExportGenesisStateCommand {
/// Output file name or stdout if unspecified.
#[clap(parse(from_os_str))]
pub output: Option<PathBuf>,
/// Write output in binary. Default is to write in hex.
#[clap(short, long)]
pub raw: bool,
/// The name of the chain for that the genesis state should be exported.
#[clap(long)]
pub chain: Option<String>,
}
/// Command for exporting the genesis wasm file.
#[derive(Debug, Parser)]
pub struct ExportGenesisWasmCommand {
/// Output file name or stdout if unspecified.
#[clap(parse(from_os_str))]
pub output: Option<PathBuf>,
/// Write output in binary. Default is to write in hex.
#[clap(short, long)]
pub raw: bool,
/// The name of the chain for that the genesis wasm file should be exported.
#[clap(long)]
pub chain: Option<String>,
}
#[derive(Debug, Parser)]
#[derive(Debug, clap::Parser)]
#[clap(
propagate_version = true,
args_conflicts_with_subcommands = true,
@@ -88,6 +52,16 @@ pub struct Cli {
#[clap(flatten)]
pub run: cumulus_client_cli::RunCmd,
/// Disable automatic hardware benchmarks.
///
/// By default these benchmarks are automatically ran at startup and measure
/// the CPU speed, the memory bandwidth and the disk speed.
///
/// The results are then printed out in the logs, and also sent as part of
/// telemetry, if telemetry is enabled.
#[clap(long)]
pub no_hardware_benchmarks: bool,
/// Relay chain arguments
#[clap(raw = true)]
pub relay_chain_args: Vec<String>,
@@ -111,9 +85,9 @@ impl RelayChainCli {
para_config: &sc_service::Configuration,
relay_chain_args: impl Iterator<Item = &'a String>,
) -> Self {
let extension = chain_spec::Extensions::try_get(&*para_config.chain_spec);
let extension = crate::chain_spec::Extensions::try_get(&*para_config.chain_spec);
let chain_id = extension.map(|e| e.relay_chain.clone());
let base_path = para_config.base_path.as_ref().map(|x| x.path().join("polkadot"));
Self { base_path, chain_id, base: polkadot_cli::RunCmd::parse_from(relay_chain_args) }
Self { base_path, chain_id, base: clap::Parser::parse_from(relay_chain_args) }
}
}
parachain-template/node/src/command.rs
+69 -84
View File
@@ -1,15 +1,11 @@
use crate::{
chain_spec,
cli::{Cli, RelayChainCli, Subcommand},
service::{new_partial, TemplateRuntimeExecutor},
};
use std::net::SocketAddr;
use codec::Encode;
use cumulus_client_service::genesis::generate_genesis_block;
use cumulus_client_cli::generate_genesis_block;
use cumulus_primitives_core::ParaId;
use frame_benchmarking_cli::BenchmarkCmd;
use frame_benchmarking_cli::{BenchmarkCmd, SUBSTRATE_REFERENCE_HARDWARE};
use log::info;
use parachain_template_runtime::{Block, RuntimeApi};
use polkadot_parachain::primitives::AccountIdConversion;
use sc_cli::{
ChainSpec, CliConfiguration, DefaultConfigurationValues, ImportParams, KeystoreParams,
NetworkParams, Result, RuntimeVersion, SharedParams, SubstrateCli,
@@ -19,10 +15,15 @@ use sc_service::{
TaskManager,
};
use sp_core::hexdisplay::HexDisplay;
use sp_runtime::traits::Block as BlockT;
use std::{io::Write, net::SocketAddr};
use sp_runtime::traits::{AccountIdConversion, Block as BlockT};
fn load_spec(id: &str) -> std::result::Result<Box<dyn sc_service::ChainSpec>, String> {
use crate::{
chain_spec,
cli::{Cli, RelayChainCli, Subcommand},
service::{new_partial, TemplateRuntimeExecutor},
};
fn load_spec(id: &str) -> std::result::Result<Box<dyn ChainSpec>, String> {
Ok(match id {
"dev" => Box::new(chain_spec::development_config()),
"template-rococo" => Box::new(chain_spec::local_testnet_config()),
@@ -41,11 +42,13 @@ impl SubstrateCli for Cli {
}
fn description() -> String {
"Parachain Collator Template\n\nThe command-line arguments provided first will be \
format!(
"Parachain Collator Template\n\nThe command-line arguments provided first will be \
passed to the parachain node, while the arguments provided after -- will be passed \
to the relay chain node.\n\n\
parachain-collator <parachain-args> -- <relay-chain-args>"
.into()
{} <parachain-args> -- <relay-chain-args>",
Self::executable_name()
)
}
fn author() -> String {
@@ -79,11 +82,13 @@ impl SubstrateCli for RelayChainCli {
}
fn description() -> String {
"Parachain Collator Template\n\nThe command-line arguments provided first will be \
format!(
"Parachain Collator Template\n\nThe command-line arguments provided first will be \
passed to the parachain node, while the arguments provided after -- will be passed \
to the relay chain node.\n\n\
parachain-collator <parachain-args> -- <relay-chain-args>"
.into()
{} <parachain-args> -- <relay-chain-args>",
Self::executable_name()
)
}
fn author() -> String {
@@ -107,16 +112,6 @@ impl SubstrateCli for RelayChainCli {
}
}
#[allow(clippy::borrowed_box)]
fn extract_genesis_wasm(chain_spec: &Box<dyn sc_service::ChainSpec>) -> Result<Vec<u8>> {
let mut storage = chain_spec.build_storage()?;
storage
.top
.remove(sp_core::storage::well_known_keys::CODE)
.ok_or_else(|| "Could not find wasm file in genesis state!".into())
}
macro_rules! construct_async_run {
(|$components:ident, $cli:ident, $cmd:ident, $config:ident| $( $code:tt )* ) => {{
let runner = $cli.create_runner($cmd)?;
@@ -164,6 +159,11 @@ pub fn run() -> Result<()> {
Ok(cmd.run(components.client, components.import_queue))
})
},
Some(Subcommand::Revert(cmd)) => {
construct_async_run!(|components, cli, cmd, config| {
Ok(cmd.run(components.client, components.backend, None))
})
},
Some(Subcommand::PurgeChain(cmd)) => {
let runner = cli.create_runner(cmd)?;
@@ -183,54 +183,20 @@ pub fn run() -> Result<()> {
cmd.run(config, polkadot_config)
})
},
Some(Subcommand::Revert(cmd)) => {
construct_async_run!(|components, cli, cmd, config| {
Ok(cmd.run(components.client, components.backend, None))
Some(Subcommand::ExportGenesisState(cmd)) => {
let runner = cli.create_runner(cmd)?;
runner.sync_run(|_config| {
let spec = cli.load_spec(&cmd.shared_params.chain.clone().unwrap_or_default())?;
let state_version = Cli::native_runtime_version(&spec).state_version();
cmd.run::<Block>(&*spec, state_version)
})
},
Some(Subcommand::ExportGenesisState(params)) => {
let mut builder = sc_cli::LoggerBuilder::new("");
builder.with_profiling(sc_tracing::TracingReceiver::Log, "");
let _ = builder.init();
let spec = load_spec(&params.chain.clone().unwrap_or_default())?;
let state_version = Cli::native_runtime_version(&spec).state_version();
let block: Block = generate_genesis_block(&spec, state_version)?;
let raw_header = block.header().encode();
let output_buf = if params.raw {
raw_header
} else {
format!("0x{:?}", HexDisplay::from(&block.header().encode())).into_bytes()
};
if let Some(output) = &params.output {
std::fs::write(output, output_buf)?;
} else {
std::io::stdout().write_all(&output_buf)?;
}
Ok(())
},
Some(Subcommand::ExportGenesisWasm(params)) => {
let mut builder = sc_cli::LoggerBuilder::new("");
builder.with_profiling(sc_tracing::TracingReceiver::Log, "");
let _ = builder.init();
let raw_wasm_blob =
extract_genesis_wasm(&cli.load_spec(&params.chain.clone().unwrap_or_default())?)?;
let output_buf = if params.raw {
raw_wasm_blob
} else {
format!("0x{:?}", HexDisplay::from(&raw_wasm_blob)).into_bytes()
};
if let Some(output) = &params.output {
std::fs::write(output, output_buf)?;
} else {
std::io::stdout().write_all(&output_buf)?;
}
Ok(())
Some(Subcommand::ExportGenesisWasm(cmd)) => {
let runner = cli.create_runner(cmd)?;
runner.sync_run(|_config| {
let spec = cli.load_spec(&cmd.shared_params.chain.clone().unwrap_or_default())?;
cmd.run(&*spec)
})
},
Some(Subcommand::Benchmark(cmd)) => {
let runner = cli.create_runner(cmd)?;
@@ -261,7 +227,12 @@ pub fn run() -> Result<()> {
cmd.run(config, partials.client.clone(), db, storage)
}),
BenchmarkCmd::Overhead(_) => Err("Unsupported benchmarking command".into()),
BenchmarkCmd::Machine(cmd) =>
runner.sync_run(|config| cmd.run(&config, SUBSTRATE_REFERENCE_HARDWARE.clone())),
// NOTE: this allows the Client to leniently implement
// new benchmark commands without requiring a companion MR.
#[allow(unreachable_patterns)]
_ => Err("Benchmarking sub-command unsupported".into()),
}
},
Some(Subcommand::TryRuntime(cmd)) => {
@@ -286,6 +257,15 @@ pub fn run() -> Result<()> {
let collator_options = cli.run.collator_options();
runner.run_node_until_exit(|config| async move {
let hwbench = if !cli.no_hardware_benchmarks {
config.database.path().map(|database_path| {
let _ = std::fs::create_dir_all(&database_path);
sc_sysinfo::gather_hwbench(Some(database_path))
})
} else {
None
};
let para_id = chain_spec::Extensions::try_get(&*config.chain_spec)
.map(|e| e.para_id)
.ok_or_else(|| "Could not find parachain ID in chain-spec.")?;
@@ -298,11 +278,10 @@ pub fn run() -> Result<()> {
let id = ParaId::from(para_id);
let parachain_account =
AccountIdConversion::<polkadot_primitives::v2::AccountId>::into_account(&id);
AccountIdConversion::<polkadot_primitives::v2::AccountId>::into_account_truncating(&id);
let state_version =
RelayChainCli::native_runtime_version(&config.chain_spec).state_version();
let block: Block = generate_genesis_block(&config.chain_spec, state_version)
let state_version = Cli::native_runtime_version(&config.chain_spec).state_version();
let block: Block = generate_genesis_block(&*config.chain_spec, state_version)
.map_err(|e| format!("{:?}", e))?;
let genesis_state = format!("0x{:?}", HexDisplay::from(&block.header().encode()));
@@ -316,10 +295,16 @@ pub fn run() -> Result<()> {
info!("Parachain genesis state: {}", genesis_state);
info!("Is collating: {}", if config.role.is_authority() { "yes" } else { "no" });
crate::service::start_parachain_node(config, polkadot_config, collator_options, id)
.await
.map(|r| r.0)
.map_err(Into::into)
crate::service::start_parachain_node(
config,
polkadot_config,
collator_options,
id,
hwbench,
)
.await
.map(|r| r.0)
.map_err(Into::into)
})
},
}
@@ -410,8 +395,8 @@ impl CliConfiguration<Self> for RelayChainCli {
self.base.base.role(is_dev)
}
fn transaction_pool(&self) -> Result<sc_service::config::TransactionPoolOptions> {
self.base.base.transaction_pool()
fn transaction_pool(&self, is_dev: bool) -> Result<sc_service::config::TransactionPoolOptions> {
self.base.base.transaction_pool(is_dev)
}
fn state_cache_child_ratio(&self) -> Result<Option<usize>> {
parachain-template/node/src/rpc.rs
+10 -9
View File
@@ -17,7 +17,7 @@ use sp_block_builder::BlockBuilder;
use sp_blockchain::{Error as BlockChainError, HeaderBackend, HeaderMetadata};
/// A type representing all RPC extensions.
pub type RpcExtension = jsonrpc_core::IoHandler<sc_rpc::Metadata>;
pub type RpcExtension = jsonrpsee::RpcModule<()>;
/// Full client dependencies
pub struct FullDeps<C, P> {
@@ -30,7 +30,9 @@ pub struct FullDeps<C, P> {
}
/// Instantiate all RPC extensions.
pub fn create_full<C, P>(deps: FullDeps<C, P>) -> RpcExtension
pub fn create_full<C, P>(
deps: FullDeps<C, P>,
) -> Result<RpcExtension, Box<dyn std::error::Error + Send + Sync>>
where
C: ProvideRuntimeApi<Block>
+ HeaderBackend<Block>
@@ -44,14 +46,13 @@ where
C::Api: BlockBuilder<Block>,
P: TransactionPool + Sync + Send + 'static,
{
use pallet_transaction_payment_rpc::{TransactionPayment, TransactionPaymentApi};
use substrate_frame_rpc_system::{FullSystem, SystemApi};
use pallet_transaction_payment_rpc::{TransactionPayment, TransactionPaymentApiServer};
use substrate_frame_rpc_system::{System, SystemApiServer};
let mut io = jsonrpc_core::IoHandler::default();
let mut module = RpcExtension::new(());
let FullDeps { client, pool, deny_unsafe } = deps;
io.extend_with(SystemApi::to_delegate(FullSystem::new(client.clone(), pool, deny_unsafe)));
io.extend_with(TransactionPaymentApi::to_delegate(TransactionPayment::new(client)));
io
module.merge(System::new(client.clone(), pool, deny_unsafe).into_rpc())?;
module.merge(TransactionPayment::new(client).into_rpc())?;
Ok(module)
}
parachain-template/node/src/service.rs
+34 -13
View File
@@ -3,6 +3,9 @@
// std
use std::{sync::Arc, time::Duration};
// rpc
use jsonrpsee::RpcModule;
use cumulus_client_cli::CollatorOptions;
// Local Runtime Types
use parachain_template_runtime::{
@@ -19,13 +22,14 @@ use cumulus_client_service::{
use cumulus_primitives_core::ParaId;
use cumulus_relay_chain_inprocess_interface::build_inprocess_relay_chain;
use cumulus_relay_chain_interface::{RelayChainError, RelayChainInterface, RelayChainResult};
use cumulus_relay_chain_rpc_interface::RelayChainRPCInterface;
use cumulus_relay_chain_rpc_interface::{create_client_and_start_worker, RelayChainRpcInterface};
// Substrate Imports
use sc_client_api::ExecutorProvider;
use sc_executor::NativeElseWasmExecutor;
use sc_network::NetworkService;
use sc_service::{Configuration, PartialComponents, Role, TFullBackend, TFullClient, TaskManager};
use sc_network_common::service::NetworkBlock;
use sc_service::{Configuration, PartialComponents, TFullBackend, TFullClient, TaskManager};
use sc_telemetry::{Telemetry, TelemetryHandle, TelemetryWorker, TelemetryWorkerHandle};
use sp_api::ConstructRuntimeApi;
use sp_keystore::SyncCryptoStorePtr;
@@ -170,15 +174,19 @@ async fn build_relay_chain_interface(
telemetry_worker_handle: Option<TelemetryWorkerHandle>,
task_manager: &mut TaskManager,
collator_options: CollatorOptions,
hwbench: Option<sc_sysinfo::HwBench>,
) -> RelayChainResult<(Arc<(dyn RelayChainInterface + 'static)>, Option<CollatorPair>)> {
match collator_options.relay_chain_rpc_url {
Some(relay_chain_url) =>
Ok((Arc::new(RelayChainRPCInterface::new(relay_chain_url).await?) as Arc<_>, None)),
Some(relay_chain_url) => {
let client = create_client_and_start_worker(relay_chain_url, task_manager).await?;
Ok((Arc::new(RelayChainRpcInterface::new(client)) as Arc<_>, None))
},
None => build_inprocess_relay_chain(
polkadot_config,
parachain_config,
telemetry_worker_handle,
task_manager,
hwbench,
),
}
}
@@ -195,6 +203,7 @@ async fn start_node_impl<RuntimeApi, Executor, RB, BIQ, BIC>(
_rpc_ext_builder: RB,
build_import_queue: BIQ,
build_consensus: BIC,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(
TaskManager,
Arc<TFullClient<Block, RuntimeApi, NativeElseWasmExecutor<Executor>>>,
@@ -219,7 +228,7 @@ where
Executor: sc_executor::NativeExecutionDispatch + 'static,
RB: Fn(
Arc<TFullClient<Block, RuntimeApi, Executor>>,
) -> Result<jsonrpc_core::IoHandler<sc_rpc::Metadata>, sc_service::Error>
) -> Result<RpcModule<()>, sc_service::Error>
+ Send
+ 'static,
BIQ: FnOnce(
@@ -251,10 +260,6 @@ where
bool,
) -> Result<Box<dyn ParachainConsensus<Block>>, sc_service::Error>,
{
if matches!(parachain_config.role, Role::Light) {
return Err("Light client not supported!".into())
}
let parachain_config = prepare_node_config(parachain_config);
let params = new_partial::<RuntimeApi, Executor, BIQ>(&parachain_config, build_import_queue)?;
@@ -270,6 +275,7 @@ where
telemetry_worker_handle,
&mut task_manager,
collator_options.clone(),
hwbench.clone(),
)
.await
.map_err(|e| match e {
@@ -297,7 +303,7 @@ where
warp_sync: None,
})?;
let rpc_extensions_builder = {
let rpc_builder = {
let client = client.clone();
let transaction_pool = transaction_pool.clone();
@@ -308,12 +314,12 @@ where
deny_unsafe,
};
Ok(crate::rpc::create_full(deps))
crate::rpc::create_full(deps).map_err(Into::into)
})
};
sc_service::spawn_tasks(sc_service::SpawnTasksParams {
rpc_extensions_builder,
rpc_builder,
client: client.clone(),
transaction_pool: transaction_pool.clone(),
task_manager: &mut task_manager,
@@ -325,6 +331,19 @@ where
telemetry: telemetry.as_mut(),
})?;
if let Some(hwbench) = hwbench {
sc_sysinfo::print_hwbench(&hwbench);
if let Some(ref mut telemetry) = telemetry {
let telemetry_handle = telemetry.handle();
task_manager.spawn_handle().spawn(
"telemetry_hwbench",
None,
sc_sysinfo::initialize_hwbench_telemetry(telemetry_handle, hwbench),
);
}
}
let announce_block = {
let network = network.clone();
Arc::new(move |hash, data| network.announce_block(hash, data))
@@ -434,6 +453,7 @@ pub async fn start_parachain_node(
polkadot_config: Configuration,
collator_options: CollatorOptions,
id: ParaId,
hwbench: Option<sc_sysinfo::HwBench>,
) -> sc_service::error::Result<(
TaskManager,
Arc<TFullClient<Block, RuntimeApi, NativeElseWasmExecutor<TemplateRuntimeExecutor>>>,
@@ -443,7 +463,7 @@ pub async fn start_parachain_node(
polkadot_config,
collator_options,
id,
|_| Ok(Default::default()),
|_| Ok(RpcModule::new(())),
parachain_build_import_queue,
|client,
prometheus_registry,
@@ -508,6 +528,7 @@ pub async fn start_parachain_node(
},
))
},
hwbench,
)
.await
}
parachain-template/polkadot-launch/config.json
+1 -1
View File
@@ -17,7 +17,7 @@
},
"parachains": [
{
"bin": "../target/release/parachain-collator",
"bin": "../target/release/polkadot-parachain",
"id": "200",
"balance": "1000000000000000000000",
"nodes": [
parachain-template/runtime/Cargo.toml
+4 -4
View File
@@ -17,10 +17,10 @@ substrate-wasm-builder = { git = "https://github.com/paritytech/substrate", bran
[dependencies]
codec = { package = "parity-scale-codec", version = "3.0.0", default-features = false, features = ["derive"] }
hex-literal = { version = "0.3.4", optional = true }
log = { version = "0.4.16", default-features = false }
log = { version = "0.4.17", default-features = false }
scale-info = { version = "2.1.1", default-features = false, features = ["derive"] }
serde = { version = "1.0.132", optional = true, features = ["derive"] }
smallvec = "1.6.1"
serde = { version = "1.0.143", optional = true, features = ["derive"] }
smallvec = "1.9.0"
# Local
pallet-template = { path = "../pallets/template", default-features = false }
@@ -73,7 +73,7 @@ cumulus-primitives-core = { path = "../../primitives/core", default-features = f
cumulus-primitives-timestamp = { path = "../../primitives/timestamp", default-features = false }
cumulus-primitives-utility = { path = "../../primitives/utility", default-features = false }
pallet-collator-selection = { path = "../../pallets/collator-selection", default-features = false }
parachain-info = { path = "../../polkadot-parachains/pallets/parachain-info", default-features = false }
parachain-info = { path = "../../parachains/pallets/parachain-info", default-features = false }
[features]
default = [
parachain-template/runtime/src/lib.rs
+5 -2
View File
@@ -9,6 +9,7 @@ include!(concat!(env!("OUT_DIR"), "/wasm_binary.rs"));
mod weights;
pub mod xcm_config;
use cumulus_pallet_parachain_system::RelayNumberStrictlyIncreases;
use smallvec::smallvec;
use sp_api::impl_runtime_apis;
use sp_core::{crypto::KeyTypeId, OpaqueMetadata};
@@ -353,6 +354,7 @@ parameter_types! {
}
impl pallet_transaction_payment::Config for Runtime {
type Event = Event;
type OnChargeTransaction = pallet_transaction_payment::CurrencyAdapter<Balances, ()>;
type WeightToFee = WeightToFee;
type LengthToFee = ConstantMultiplier<Balance, TransactionByteFee>;
@@ -369,11 +371,12 @@ impl cumulus_pallet_parachain_system::Config for Runtime {
type Event = Event;
type OnSystemEvent = ();
type SelfParaId = parachain_info::Pallet<Runtime>;
type OutboundXcmpMessageSource = XcmpQueue;
type DmpMessageHandler = DmpQueue;
type ReservedDmpWeight = ReservedDmpWeight;
type OutboundXcmpMessageSource = XcmpQueue;
type XcmpMessageHandler = XcmpQueue;
type ReservedXcmpWeight = ReservedXcmpWeight;
type CheckAssociatedRelayNumber = RelayNumberStrictlyIncreases;
}
impl parachain_info::Config for Runtime {}
@@ -474,7 +477,7 @@ construct_runtime!(
// Monetary stuff.
Balances: pallet_balances::{Pallet, Call, Storage, Config<T>, Event<T>} = 10,
TransactionPayment: pallet_transaction_payment::{Pallet, Storage} = 11,
TransactionPayment: pallet_transaction_payment::{Pallet, Storage, Event<T>} = 11,
// Collator support. The order of these 4 are important and shall not change.
Authorship: pallet_authorship::{Pallet, Call, Storage} = 20,
parachain-template/runtime/src/xcm_config.rs
+76 -8
View File
@@ -2,8 +2,9 @@ use super::{
AccountId, Balances, Call, Event, Origin, ParachainInfo, ParachainSystem, PolkadotXcm, Runtime,
WeightToFee, XcmpQueue,
};
use core::marker::PhantomData;
use frame_support::{
match_types, parameter_types,
log, match_types, parameter_types,
traits::{ConstU32, Everything, Nothing},
weights::Weight,
};
@@ -18,7 +19,7 @@ use xcm_builder::{
SignedAccountId32AsNative, SignedToAccountId32, SovereignSignedViaLocation, TakeWeightCredit,
UsingComponents,
};
use xcm_executor::XcmExecutor;
use xcm_executor::{traits::ShouldExecute, XcmExecutor};
parameter_types! {
pub const RelayLocation: MultiLocation = MultiLocation::parent();
@@ -88,12 +89,79 @@ match_types! {
};
}
pub type Barrier = (
TakeWeightCredit,
AllowTopLevelPaidExecutionFrom<Everything>,
AllowUnpaidExecutionFrom<ParentOrParentsExecutivePlurality>,
// ^^^ Parent and its exec plurality get free execution
);
//TODO: move DenyThenTry to polkadot's xcm module.
/// Deny executing the xcm message if it matches any of the Deny filter regardless of anything else.
/// If it passes the Deny, and matches one of the Allow cases then it is let through.
pub struct DenyThenTry<Deny, Allow>(PhantomData<Deny>, PhantomData<Allow>)
where
Deny: ShouldExecute,
Allow: ShouldExecute;
impl<Deny, Allow> ShouldExecute for DenyThenTry<Deny, Allow>
where
Deny: ShouldExecute,
Allow: ShouldExecute,
{
fn should_execute<Call>(
origin: &MultiLocation,
message: &mut Xcm<Call>,
max_weight: Weight,
weight_credit: &mut Weight,
) -> Result<(), ()> {
Deny::should_execute(origin, message, max_weight, weight_credit)?;
Allow::should_execute(origin, message, max_weight, weight_credit)
}
}
// See issue #5233
pub struct DenyReserveTransferToRelayChain;
impl ShouldExecute for DenyReserveTransferToRelayChain {
fn should_execute<Call>(
origin: &MultiLocation,
message: &mut Xcm<Call>,
_max_weight: Weight,
_weight_credit: &mut Weight,
) -> Result<(), ()> {
if message.0.iter().any(|inst| {
matches!(
inst,
InitiateReserveWithdraw {
reserve: MultiLocation { parents: 1, interior: Here },
..
} | DepositReserveAsset { dest: MultiLocation { parents: 1, interior: Here }, .. } |
TransferReserveAsset {
dest: MultiLocation { parents: 1, interior: Here },
..
}
)
}) {
return Err(()) // Deny
}
// An unexpected reserve transfer has arrived from the Relay Chain. Generally, `IsReserve`
// should not allow this, but we just log it here.
if matches!(origin, MultiLocation { parents: 1, interior: Here }) &&
message.0.iter().any(|inst| matches!(inst, ReserveAssetDeposited { .. }))
{
log::warn!(
target: "xcm::barriers",
"Unexpected ReserveAssetDeposited from the Relay Chain",
);
}
// Permit everything else
Ok(())
}
}
pub type Barrier = DenyThenTry<
DenyReserveTransferToRelayChain,
(
TakeWeightCredit,
AllowTopLevelPaidExecutionFrom<Everything>,
AllowUnpaidExecutionFrom<ParentOrParentsExecutivePlurality>,
// ^^^ Parent and its exec plurality get free execution
),
>;
pub struct XcmConfig;
impl xcm_executor::Config for XcmConfig {
parachains/README.md
+23
View File
@@ -0,0 +1,23 @@
# Parachains
This directory is the home of Parity-developed parachain runtimes. This directory is _runtime
focused_, and does not include builds of parachain _nodes_.
The general internal structure is:
- `chain-specs`: Chain specs for the runtimes contained in its sibling dir `runtimes`.
- `common`: Common configurations, `impl`s, etc. used by several parachain runtimes.
- `integration-tests`: Integration tests to test parachain interactions via XCM.
- `pallets`: FRAME pallets that are specific to parachains.
- `runtimes`: The entry point for parachain runtimes.
## Common Good Parachains
The `runtimes` directory includes many, but is not limited to,
[common good parachains](https://wiki.polkadot.network/docs/learn-common-goods). Likewise, not all
common good parachains are in this repo.
## Releases
The project maintainers generally try to release a set of parachain runtimes for each Polkadot
Relay Chain runtime release.
parachains/chain-specs/collectives-polkadot.json
+84
View File
File diff suppressed because one or more lines are too long
parachains/chain-specs/collectives-westend.json
+104
View File
File diff suppressed because one or more lines are too long
polkadot-parachains/res/canvas-rococo.json → parachains/chain-specs/contracts-rococo.json
+6 -6
View File
@@ -1,12 +1,12 @@
{
"name": "Canvas on Rococo",
"id": "canvas-rococo",
"name": "Contracts on Rococo",
"id": "contracts-rococo",
"chainType": "Live",
"bootNodes": [
"/ip4/34.90.191.237/tcp/30333/p2p/12D3KooWKg3Rpxcr9oJ8n6khoxpGKWztCZydtUZk2cojHqnfLrpj",
"/ip4/35.204.68.28/tcp/30333/p2p/12D3KooWPEXYrz8tHU3nDtPoPw4V7ou5dzMEWSTuUj7vaWiYVAVh",
"/ip4/34.90.139.15/tcp/30333/p2p/12D3KooWEVU8AFNary4nP4qEnEcwJaRuy59Wefekzdu9pKbnVEhk",
"/ip4/35.204.99.97/tcp/30333/p2p/12D3KooWP6pV3ZmcXzGDjv8ZMgA6nZxfAKDxSz4VNiLx6vVCQgJX"
"/dns/contracts-collator-0.parity-testnet.parity.io/tcp/30333/p2p/12D3KooWKg3Rpxcr9oJ8n6khoxpGKWztCZydtUZk2cojHqnfLrpj",
"/dns/contracts-collator-1.parity-testnet.parity.io/tcp/30333/p2p/12D3KooWPEXYrz8tHU3nDtPoPw4V7ou5dzMEWSTuUj7vaWiYVAVh",
"/dns/contracts-collator-2.parity-testnet.parity.io/tcp/30333/p2p/12D3KooWEVU8AFNary4nP4qEnEcwJaRuy59Wefekzdu9pKbnVEhk",
"/dns/contracts-collator-3.parity-testnet.parity.io/tcp/30333/p2p/12D3KooWP6pV3ZmcXzGDjv8ZMgA6nZxfAKDxSz4VNiLx6vVCQgJX"
],
"telemetryEndpoints": null,
"protocolId": null,
parachains/chain-specs/rockmine.json
+80
View File
File diff suppressed because one or more lines are too long
polkadot-parachains/res/shell-statemint-head-data → parachains/chain-specs/shell-statemint-head-data
View File
polkadot-parachains/res/shell-statemint.json → parachains/chain-specs/shell-statemint.json
View File
polkadot-parachains/res/statemine.json → parachains/chain-specs/statemine.json
+9 -1
View File
@@ -5,7 +5,15 @@
"bootNodes": [
"/ip4/34.77.217.152/tcp/30334/p2p/12D3KooWF63ZxKtZMYs5247WQA8fcTiGJb2osXykc31cmjwNLwem",
"/ip4/34.77.119.77/tcp/30334/p2p/12D3KooWGowDwrXAh9cxkbPHPHuwMouFHrMcJhCVXcFS2B8vc5Ry",
"/ip4/35.188.228.71/tcp/30334/p2p/12D3KooWNqL6XroD35tDAs3DKG7do8dbKvdnACrR7Z3dcKesNcG4"
"/ip4/35.188.228.71/tcp/30334/p2p/12D3KooWNqL6XroD35tDAs3DKG7do8dbKvdnACrR7Z3dcKesNcG4",
"/dns/statemine-connect-ew1-0.parity-kusama-parachains.parity.io/tcp/30334/p2p/12D3KooWMzvdGcUXxacLdMQzRVrsP1mJrZHcrz8LtGbhLzve84Qx",
"/dns/statemine-connect-ew1-0.parity-kusama-parachains.parity.io/tcp/443/wss/p2p/12D3KooWMzvdGcUXxacLdMQzRVrsP1mJrZHcrz8LtGbhLzve84Qx",
"/dns/statemine-connect-ew1-1.parity-kusama-parachains.parity.io/tcp/30334/p2p/12D3KooWQmGf5z3DU1kKcZoLzMNgdbP31ybjuwxS1VGLKMUjq5ez",
"/dns/statemine-connect-ew1-1.parity-kusama-parachains.parity.io/tcp/443/wss/p2p/12D3KooWQmGf5z3DU1kKcZoLzMNgdbP31ybjuwxS1VGLKMUjq5ez",
"/dns/statemine-connect-ue4-0.parity-kusama-parachains.parity.io/tcp/30334/p2p/12D3KooWLm6iHcmA3YD4xn2zfbm4KLF5KSUqJJAnmt2UGr9o2PgB",
"/dns/statemine-connect-ue4-0.parity-kusama-parachains.parity.io/tcp/443/wss/p2p/12D3KooWLm6iHcmA3YD4xn2zfbm4KLF5KSUqJJAnmt2UGr9o2PgB",
"/dns/statemine-connect-ue4-1.parity-kusama-parachains.parity.io/tcp/30334/p2p/12D3KooWD8Bma5qPbq7N5qdED3Xy6GXHfvfk86TL8aVTQKxmWkHG",
"/dns/statemine-connect-ue4-1.parity-kusama-parachains.parity.io/tcp/443/wss/p2p/12D3KooWD8Bma5qPbq7N5qdED3Xy6GXHfvfk86TL8aVTQKxmWkHG"
],
"telemetryEndpoints": null,
"protocolId": null,
polkadot-parachains/res/statemine_genesis.json → parachains/chain-specs/statemine_genesis.json
View File
polkadot-parachains/res/statemine_genesis_values.json → parachains/chain-specs/statemine_genesis_values.json
View File
polkadot-parachains/res/statemint.json → parachains/chain-specs/statemint.json
+9 -1
View File
@@ -6,7 +6,15 @@
"/ip4/34.65.251.121/tcp/30334/p2p/12D3KooWG3GrM6XKMM4gp3cvemdwUvu96ziYoJmqmetLZBXE8bSa",
"/ip4/34.65.35.228/tcp/30334/p2p/12D3KooWMRyTLrCEPcAQD6c4EnudL3vVzg9zji3whvsMYPUYevpq",
"/ip4/34.83.247.146/tcp/30334/p2p/12D3KooWE4jFh5FpJDkWVZhnWtFnbSqRhdjvC7Dp9b8b3FTuubQC",
"/ip4/104.199.117.230/tcp/30334/p2p/12D3KooWG9R8pVXKumVo2rdkeVD4j5PVhRTqmYgLHY3a4yPYgLqM"
"/ip4/104.199.117.230/tcp/30334/p2p/12D3KooWG9R8pVXKumVo2rdkeVD4j5PVhRTqmYgLHY3a4yPYgLqM",
"/dns/statemint-connect-ew6-0.parity-polkadot-parachains.parity.io/tcp/30334/p2p/12D3KooWLHqbcQtoBygf7GJgVjVa3TaeLuf7VbicNdooaCmQM2JZ",
"/dns/statemint-connect-ew6-0.parity-polkadot-parachains.parity.io/tcp/443/wss/p2p/12D3KooWLHqbcQtoBygf7GJgVjVa3TaeLuf7VbicNdooaCmQM2JZ",
"/dns/statemint-connect-ew6-1.parity-polkadot-parachains.parity.io/tcp/30334/p2p/12D3KooWNDrKSayoZXGGE2dRSFW2g1iGPq3fTZE2U39ma9yZGKd3",
"/dns/statemint-connect-ew6-1.parity-polkadot-parachains.parity.io/tcp/443/wss/p2p/12D3KooWNDrKSayoZXGGE2dRSFW2g1iGPq3fTZE2U39ma9yZGKd3",
"/dns/statemint-connect-uw1-0.parity-polkadot-parachains.parity.io/tcp/30334/p2p/12D3KooWApa2JW4rbLtgzuK7fjLMupLS9HZheX9cdkQKyu6AnGrP",
"/dns/statemint-connect-uw1-0.parity-polkadot-parachains.parity.io/tcp/443/wss/p2p/12D3KooWApa2JW4rbLtgzuK7fjLMupLS9HZheX9cdkQKyu6AnGrP",
"/dns/statemint-connect-uw1-1.parity-polkadot-parachains.parity.io/tcp/30334/p2p/12D3KooWRsVeHqRs2iKmjLiguxp8myL4G2mDAWhtX2jHwyWujseV",
"/dns/statemint-connect-uw1-1.parity-polkadot-parachains.parity.io/tcp/443/wss/p2p/12D3KooWRsVeHqRs2iKmjLiguxp8myL4G2mDAWhtX2jHwyWujseV"
],
"telemetryEndpoints": null,
"protocolId": null,
polkadot-parachains/res/statemint_genesis.json → parachains/chain-specs/statemint_genesis.json
View File
polkadot-parachains/res/statemint_genesis_values.json → parachains/chain-specs/statemint_genesis_values.json
View File
polkadot-parachains/res/statemint_genesis_values.scale → parachains/chain-specs/statemint_genesis_values.scale
View File
polkadot-parachains/res/tick.json → parachains/chain-specs/tick.json
View File

Some files were not shown because too many files have changed in this diff Show More