update polkadot, substrate, cumulus readme (#1182)

* update readmes

* temporary ReadMe for the Polkadot SDK

* delete welcome readme

* update links on substrate readme

* update links on polkadot readme

* update links on cumulus readme

* update overseer feature comment

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update cumulus/README.md

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update cumulus/README.md

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update polkadot/README.md

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update polkadot/README.md

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* Update polkadot/README.md

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* update gitlab links

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

* terminal friendly convention

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
This commit is contained in:
Joyce Siqueira
2023-08-29 13:37:16 +02:00
committed by GitHub
parent b728724318
commit c80f76f187
3 changed files with 559 additions and 21 deletions
+264 -7
View File
@@ -1,13 +1,270 @@
Dear contributors and users,
# Cumulus ☁️
We would like to inform you that we have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into a single new repository: [the Polkadot SDK](https://github.com/paritytech/polkadot-sdk). Go ahead and make sure to support us by giving a star ⭐️ to the new repo.
[![Doc](https://img.shields.io/badge/cumulus%20docs-master-brightgreen)](https://paritytech.github.io/cumulus/)
By consolidating our codebase, we aim to enhance collaboration and provide a more efficient platform for future development.
This repository contains both the Cumulus SDK and also specific chains implemented on top of this
SDK.
If you currently have an open pull request in any of the merged repositories, we kindly request that you resubmit your PR in the new repository. This will ensure that your contributions are considered within the updated context and enable us to review and merge them more effectively.
If you only want to run a **Polkadot Parachain Node**, check out our [container
section](./docs/container.md).
We appreciate your understanding and ongoing support throughout this transition. Should you have any questions or require further assistance, please don't hesitate to [reach out to us](https://forum.polkadot.network/t/psa-parity-is-currently-working-on-merging-the-polkadot-stack-repositories-into-one-single-repository/2883).
## Cumulus SDK
Best Regards,
A set of tools for writing [Substrate](https://substrate.io/)-based
[Polkadot](https://wiki.polkadot.network/en/)
[parachains](https://wiki.polkadot.network/docs/en/learn-parachains). Refer to the included
[overview](docs/overview.md) for architectural details, and the [Connect to a relay chain how-to
guide](https://docs.substrate.io/reference/how-to-guides/parachains/connect-to-a-relay-chain/) for a
guided walk-through of using these tools.
Parity Technologies
It's easy to write blockchains using Substrate, and the overhead of writing parachains'
distribution, p2p, database, and synchronization layers should be just as low. This project aims to
make it easy to write parachains for Polkadot by leveraging the power of Substrate.
Cumulus clouds are shaped sort of like dots; together they form a system that is intricate,
beautiful and functional.
### Consensus
[`parachain-consensus`](https://github.com/paritytech/polkadot-sdk/blob/master/cumulus/client/consensus/common/src/parachain_consensus.rs)
is a [consensus engine](https://docs.substrate.io/v3/advanced/consensus) for Substrate that follows
a Polkadot [relay chain](https://wiki.polkadot.network/docs/en/learn-architecture#relay-chain). This
will run a Polkadot node internally, and dictate to the client and synchronization algorithms which
chain to follow,
[finalize](https://wiki.polkadot.network/docs/en/learn-consensus#probabilistic-vs-provable-finality),
and treat as best.
### Collator
A Polkadot [collator](https://wiki.polkadot.network/docs/en/learn-collator) for the parachain is
implemented by the `polkadot-parachain` binary (previously called `polkadot-collator`).
You may run `polkadot-parachain` locally after building it or using one of the container option
described [here](./docs/container.md).
### Relay Chain Interaction
To operate a parachain node, a connection to the corresponding relay
chain is necessary. This can be achieved in one of three ways:
1. Run a full relay chain node within the parachain node (default)
2. Connect to an external relay chain node via WebSocket RPC
3. Run a light client for the relay chain
#### In-process Relay Chain Node
If an external relay chain node is not specified (default behavior), then a full relay chain node is
spawned within the same process.
This node has all of the typical components of a regular Polkadot node and will have to fully sync
with the relay chain to work.
##### Example command
```bash
polkadot-parachain \
--chain parachain-chainspec.json \
--tmp \
-- \
--chain relaychain-chainspec.json
```
#### External Relay Chain Node
An external relay chain node is connected via WebsSocket RPC by using the
`--relay-chain-rpc-urls` command line argument. This option accepts one or more
space-separated WebSocket URLs to a full relay chain node. By default, only the
first URL will be used, with the rest as a backup in case the connection to the
first node is lost.
Parachain nodes using this feature won't have to fully sync with the relay chain
to work, so in general they will use fewer system resources.
**Note:** At this time, any parachain nodes using this feature will still spawn a
significantly cut-down relay chain node in-process. Even though they lack the
majority of normal Polkadot subsystems, they will still need to connect directly
to the relay chain network.
##### Example command
```bash
polkadot-parachain \
--chain parachain-chainspec.json \
--tmp \
--relay-chain-rpc-urls \
"ws://relaychain-rpc-endpoint:9944" \
"ws://relaychain-rpc-endpoint-backup:9944" \
-- \
--chain relaychain-chainspec.json
```
#### Relay Chain Light Client
An internal relay chain light client provides a fast and lightweight approach
for connecting to the relay chain network. It provides relay chain notifications
and facilitates runtime calls.
To specify which chain the light client should connect to, users need to supply
a relay chain chain-spec as part of the relay chain arguments.
**Note:** At this time, any parachain nodes using this feature will still spawn
a significantly cut-down relay chain node in-process. Even though they lack the
majority of normal Polkadot subsystems, they will still need to connect directly
to the relay chain network.
##### Example command
```bash
polkadot-parachain \
--chain parachain-chainspec.json \
--tmp \
--relay-chain-light-client \
-- \
--chain relaychain-chainspec.json
```
## Installation and Setup
Before building Cumulus SDK based nodes / runtimes prepare your environment by
following Substrate [installation instructions](https://docs.substrate.io/main-docs/install/).
To launch a local network, you can use [zombienet](https://github.com/paritytech/zombienet)
for quick setup and experimentation or follow the [manual setup](#manual-setup).
### Zombienet
We use Zombienet to spin up networks for integration tests and local networks.
Follow [these installation steps](https://github.com/paritytech/zombienet#requirements-by-provider)
to set it up on your machine. A simple network specification with two relay chain
nodes and one collator is located at [zombienet/examples/small_network.toml](zombienet/examples/small_network.toml).
#### Which provider should I use?
Zombienet offers multiple providers to run networks. Choose the one that best fits your needs:
- **Podman:** Choose this if you want to spin up a network quick and easy.
- **Native:** Choose this if you want to develop and deploy your changes. Requires compilation
of the binaries.
- **Kubernetes:** Choose this for advanced use-cases or running on cloud-infrastructure.
#### How to run
To run the example network, use the following commands:
```bash
# Podman provider
zombienet --provider podman spawn ./zombienet/examples/small_network.toml
# Native provider, assumes polkadot and polkadot-parachains binary in $PATH
zombienet --provider native spawn ./zombienet/examples/small_network.toml
```
### Manual Setup
#### Launch the Relay Chain
```bash
# Clone
git clone https://github.com/paritytech/polkadot-sdk
# Compile Polkadot
cargo build --release --bin polkadot
# Generate a raw chain spec
./target/release/polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > rococo-local-cfde.json
# Alice
./target/release/polkadot --chain rococo-local-cfde.json --alice --tmp
# Bob (In a separate terminal)
./target/release/polkadot --chain rococo-local-cfde.json --bob --tmp --port 30334
```
#### Launch the Parachain
```bash
# Clone
git clone https://github.com/paritytech/polkadot-sdk
# Compile
cargo build --release --bin polkadot-parachain
# Export genesis state
./target/release/polkadot-parachain export-genesis-state > genesis-state
# Export genesis wasm
./target/release/polkadot-parachain export-genesis-wasm > genesis-wasm
# Collator1
./target/release/polkadot-parachain --collator --alice --force-authoring --tmp --port 40335 --rpc-port 9946 -- --chain ../polkadot/rococo-local-cfde.json --port 30335
# Collator2
./target/release/polkadot-parachain --collator --bob --force-authoring --tmp --port 40336 --rpc-port 9947 -- --chain ../polkadot/rococo-local-cfde.json --port 30336
# Parachain Full Node 1
./target/release/polkadot-parachain --tmp --port 40337 --rpc-port 9948 -- --chain ../polkadot/rococo-local-cfde.json --port 30337
```
#### Register the parachain
![image](https://user-images.githubusercontent.com/2915325/99548884-1be13580-2987-11eb-9a8b-20be658d34f9.png)
## Asset Hub 🪙
This repository also contains the Asset Hub runtimes. Asset Hub is a system parachain
providing an asset store for the Polkadot ecosystem.
### Build & Launch a Node
To run an Asset Hub node, you will need to compile the `polkadot-parachain` binary:
```bash
cargo build --release --locked --bin polkadot-parachain
```
Once the executable is built, launch the parachain node via:
```bash
CHAIN=asset-hub-westend # or asset-hub-kusama
./target/release/polkadot-parachain --chain $CHAIN
```
Refer to the [setup instructions](#manual-setup) to run a local network for development.
## Contracts 📝
See [the `contracts-rococo` readme](parachains/runtimes/contracts/contracts-rococo/README.md) for details.
## Bridge-hub 📝
See [the `bridge-hubs` readme](parachains/runtimes/bridge-hubs/README.md) for details.
## Rococo 👑
[Rococo](https://polkadot.js.org/apps/?rpc=wss://rococo-rpc.polkadot.io) is becoming a
[Community Parachain Testbed](https://polkadot.network/blog/rococo-revamp-becoming-a-community-parachain-testbed/)
for parachain teams in the Polkadot ecosystem. It supports multiple parachains with the
differentiation of long-term connections and recurring short-term connections, to see
which parachains are currently connected and how long they will be connected for
[see here](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-rpc.polkadot.io#/parachains).
Rococo is an elaborate style of design and the name describes the painstaking effort that
has gone into this project.
### Build & Launch Rococo Collators
Collators are similar to validators in the relay chain. These nodes build the blocks that
will eventually be included by the relay chain for a parachain.
To run a Rococo collator you will need to compile the following binary:
```bash
cargo build --release --locked --bin polkadot-parachain
```
Once the executable is built, launch collators for each parachain (repeat once each for chain
`tick`, `trick`, `track`):
```bash
./target/release/polkadot-parachain --chain $CHAIN --validator
```
You can also build [using a container](./docs/container.md).
### Parachains
* [Asset Hub](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-statemint-rpc.polkadot.io#/explorer)
* [Contracts on Rococo](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-contracts-rpc.polkadot.io#/explorer)
* [RILT](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo.kilt.io#/explorer)
The network uses horizontal message passing (HRMP) to enable communication between
parachains and the relay chain and, in turn, between parachains. This means that every
message is sent to the relay chain, and from the relay chain to its destination parachain.
+257 -7
View File
@@ -1,13 +1,263 @@
Dear contributors and users,
# Polkadot
We would like to inform you that we have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into a single new repository: [the Polkadot SDK](https://github.com/paritytech/polkadot-sdk). Go ahead and make sure to support us by giving a star ⭐️ to the new repo.
Implementation of a <https://polkadot.network> node in Rust based on the Substrate framework.
By consolidating our codebase, we aim to enhance collaboration and provide a more efficient platform for future development.
> **NOTE:** In 2018, we split our implementation of "Polkadot" from its development framework >
"Substrate". See the [Substrate][substrate-repo] repo for git history prior to 2018.
If you currently have an open pull request in any of the merged repositories, we kindly request that you resubmit your PR in the new repository. This will ensure that your contributions are considered within the updated context and enable us to review and merge them more effectively.
[substrate-repo]: https://github.com/paritytech/substrate
We appreciate your understanding and ongoing support throughout this transition. Should you have any questions or require further assistance, please don't hesitate to [reach out to us](https://forum.polkadot.network/t/psa-parity-is-currently-working-on-merging-the-polkadot-stack-repositories-into-one-single-repository/2883).
This repo contains runtimes for the Polkadot, Kusama, and Westend networks. The README provides
information about installing the `polkadot` binary and developing on the codebase. For more specific
guides, like how to be a validator, see the [Polkadot
Wiki](https://wiki.polkadot.network/docs/getting-started).
Best Regards,
## Installation
Parity Technologies
If you just wish to run a Polkadot node without compiling it yourself, you may either run the latest
binary from our [releases](https://github.com/paritytech/polkadot-sdk/releases) page, or install
Polkadot from one of our package repositories.
Installation from the Debian repository will create a `systemd` service that can be used to run a
Polkadot node. This is disabled by default, and can be started by running `systemctl start polkadot`
on demand (use `systemctl enable polkadot` to make it auto-start after reboot). By default, it will
run as the `polkadot` user. Command-line flags passed to the binary can be customized by editing
`/etc/default/polkadot`. This file will not be overwritten on updating polkadot. You may also just
run the node directly from the command-line.
### Debian-based (Debian, Ubuntu)
Currently supports Debian 10 (Buster) and Ubuntu 20.04 (Focal), and derivatives. Run the following
commands as the `root` user.
```bash
# Import the security@parity.io GPG key
gpg --recv-keys --keyserver hkps://keys.mailvelope.com 9D4B2B6EB8F97156D19669A9FF0812D491B96798
gpg --export 9D4B2B6EB8F97156D19669A9FF0812D491B96798 > /usr/share/keyrings/parity.gpg
# Add the Parity repository and update the package index
echo 'deb [signed-by=/usr/share/keyrings/parity.gpg] https://releases.parity.io/deb release main' > /etc/apt/sources.list.d/parity.list
apt update
# Install the `parity-keyring` package - This will ensure the GPG key
# used by APT remains up-to-date
apt install parity-keyring
# Install polkadot
apt install polkadot
```
## Building
### Install via Cargo
Make sure you have the support software installed from the **Build from Source** section below this
section.
If you want to install Polkadot in your PATH, you can do so with:
```bash
cargo install --git https://github.com/paritytech/polkadot-sdk --tag <version> polkadot --locked
```
### Build from Source
If you'd like to build from source, first install Rust. You may need to add Cargo's bin directory to
your PATH environment variable. Restarting your computer will do this for you automatically.
```bash
curl https://sh.rustup.rs -sSf | sh
```
If you already have Rust installed, make sure you're using the latest version by running:
```bash
rustup update
```
Once done, finish installing the support software:
```bash
sudo apt install build-essential git clang libclang-dev pkg-config libssl-dev protobuf-compiler
```
Build the client by cloning this repository and running the following commands from the root
directory of the repo:
```bash
git checkout <latest tagged release>
./scripts/init.sh
cargo build --release
```
**Note:** compilation is a memory intensive process. We recommend having 4 GiB of physical RAM or
swap available (keep in mind that if a build hits swap it tends to be very slow).
**Note:** if you want to move the built `polkadot` binary somewhere (e.g. into $PATH) you will also
need to move `polkadot-execute-worker` and `polkadot-prepare-worker`. You can let cargo do all this
for you by running:
```sh
cargo install --path . --locked
```
#### Build from Source with Docker
You can also build from source using [Parity CI docker image](https://github.com/paritytech/scripts/tree/master/dockerfiles/ci-linux):
```bash
git checkout <latest tagged release>
docker run --rm -it -w /shellhere/polkadot \
-v $(pwd):/shellhere/polkadot \
paritytech/ci-linux:production cargo build --release
sudo chown -R $(id -u):$(id -g) target/
```
If you want to reproduce other steps of CI process you can use the following
[guide](https://github.com/paritytech/scripts#gitlab-ci-for-building-docker-images).
## Networks
This repo supports runtimes for Polkadot, Kusama, and Westend.
### Connect to Polkadot Mainnet
Connect to the global Polkadot Mainnet network by running:
```bash
./target/release/polkadot --chain=polkadot
```
You can see your node on [telemetry] (set a custom name with `--name "my custom name"`).
[telemetry]: https://telemetry.polkadot.io/#list/Polkadot
### Connect to the "Kusama" Canary Network
Connect to the global Kusama canary network by running:
```bash
./target/release/polkadot --chain=kusama
```
You can see your node on [telemetry] (set a custom name with `--name "my custom name"`).
[telemetry]: https://telemetry.polkadot.io/#list/Kusama
### Connect to the Westend Testnet
Connect to the global Westend testnet by running:
```bash
./target/release/polkadot --chain=westend
```
You can see your node on [telemetry] (set a custom name with `--name "my custom name"`).
[telemetry]: https://telemetry.polkadot.io/#list/Westend
### Obtaining DOTs
If you want to do anything on Polkadot, Kusama, or Westend, then you'll need to get an account and
some DOT, KSM, or WND tokens, respectively. See the [claims
instructions](https://claims.polkadot.network/) for Polkadot if you have DOTs to claim. For
Westend's WND tokens, see the faucet
[instructions](https://wiki.polkadot.network/docs/learn-DOT#getting-westies) on the Wiki.
## Hacking on Polkadot
If you'd actually like to hack on Polkadot, you can grab the source code and build it. Ensure you
have Rust and the support software installed. This script will install or update Rust and install
the required dependencies (this may take up to 30 minutes on Mac machines):
```bash
curl https://getsubstrate.io -sSf | bash -s -- --fast
```
Then, grab the Polkadot source code:
```bash
git clone https://github.com/paritytech/polkadot-sdk.git
cd polkadot
```
Then build the code. You will need to build in release mode (`--release`) to start a network. Only
use debug mode for development (faster compile times for development and testing).
```bash
./scripts/init.sh # Install WebAssembly. Update Rust
cargo build # Builds all native code
```
You can run the tests if you like:
```bash
cargo test --workspace --release
```
You can start a development chain with:
```bash
cargo run --bin polkadot -- --dev
```
Detailed logs may be shown by running the node with the following environment variables set:
```bash
RUST_LOG=debug RUST_BACKTRACE=1 cargo run --bin polkadot----dev
```
### Development
You can run a simple single-node development "network" on your machine by running:
```bash
cargo run --bin polkadot --release -- --dev
```
You can muck around by heading to <https://polkadot.js.org/apps> and choose "Local Node" from the
Settings menu.
### Local Two-node Testnet
If you want to see the multi-node consensus algorithm in action locally, then you can create a local
testnet. You'll need two terminals open. In one, run:
```bash
polkadot --chain=polkadot-local --alice -d /tmp/alice
```
And in the other, run:
```bash
polkadot --chain=polkadot-local --bob -d /tmp/bob --port 30334 --bootnodes '/ip4/127.0.0.1/tcp/30333/p2p/ALICE_BOOTNODE_ID_HERE'
```
Ensure you replace `ALICE_BOOTNODE_ID_HERE` with the node ID from the output of the first terminal.
### Monitoring
[Setup Prometheus and Grafana](https://wiki.polkadot.network/docs/maintain-guides-how-to-monitor-your-node).
Once you set this up you can take a look at the [Polkadot Grafana dashboards](grafana/README.md)
that we currently maintain.
### Using Docker
[Using Docker](doc/docker.md)
### Shell Completion
[Shell Completion](doc/shell-completion.md)
## Contributing
### Contributing Guidelines
[Contribution Guidelines](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md)
### Contributor Code of Conduct
[Code of Conduct](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CODE_OF_CONDUCT.md)
## License
Polkadot is [GPL 3.0 licensed](LICENSE).
+38 -7
View File
@@ -1,13 +1,44 @@
Dear contributors and users,
# Substrate &middot; [![GitHub license](https://img.shields.io/badge/license-GPL3%2FApache2-blue)](#LICENSE) [![GitLab Status](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/badges/master/pipeline.svg)](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines) [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](docs/CONTRIBUTING.md) [![Stack Exchange](https://img.shields.io/badge/Substrate-Community%20&%20Support-24CC85?logo=stackexchange)](https://substrate.stackexchange.com/)
<p align="center">
<img src="/docs/media/sub.gif">
</p>
We would like to inform you that we have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into a single new repository: [the Polkadot SDK](https://github.com/paritytech/polkadot-sdk). Go ahead and make sure to support us by giving a star ⭐️ to the new repo.
Substrate is a next-generation framework for blockchain innovation 🚀.
By consolidating our codebase, we aim to enhance collaboration and provide a more efficient platform for future development.
## Getting Started
If you currently have an open pull request in any of the merged repositories, we kindly request that you resubmit your PR in the new repository. This will ensure that your contributions are considered within the updated context and enable us to review and merge them more effectively.
Head to [docs.substrate.io](https://docs.substrate.io) and follow the
[installation](https://docs.substrate.io/install/) instructions. Then try out one of the
[tutorials](https://docs.substrate.io/tutorials/). Refer to the [Docker instructions](./docker/README.md) to quickly run Substrate, Substrate Node Template, Subkey, or to build a chain spec.
We appreciate your understanding and ongoing support throughout this transition. Should you have any questions or require further assistance, please don't hesitate to [reach out to us](https://forum.polkadot.network/t/psa-parity-is-currently-working-on-merging-the-polkadot-stack-repositories-into-one-single-repository/2883).
## Community & Support
Best Regards,
Join the highly active and supportive community on the
[Substrate Stack Exchange](https://substrate.stackexchange.com/) to ask questions about use and problems you run into using this software.
Please do report bugs and [issues here](https://github.com/paritytech/polkadot-sdk/issues) for anything you suspect requires action in the source.
Parity Technologies
## Contributions & Code of Conduct
Please follow the contributions guidelines as outlined in
[`docs/CONTRIBUTING.md`](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md).
In all communications and contributions, this project follows the [Contributor Covenant Code of Conduct](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CODE_OF_CONDUCT.md).
## Security
The security policy and procedures can be found in
[`docs/SECURITY.md`](https://github.com/paritytech/polkadot-sdk/blob/master/docs/SECURITY.md).
## License
- Substrate Primitives (`sp-*`), Frame (`frame-*`) and the pallets (`pallets-*`), binaries (`/bin`)
and all other utilities are licensed under [Apache 2.0](LICENSE-APACHE2). - Substrate Client
(`/client/*` / `sc-*`) is licensed under [GPL v3.0 with a classpath linking
exception](LICENSE-GPL3).
The reason for the split-licensing is to ensure that for the vast majority of teams using Substrate
to create feature-chains, then all changes can be made entirely in Apache2-licensed code, allowing
teams full freedom over what and how they release and giving licensing clarity to commercial teams.
In the interests of the community, we require any deeper improvements made to Substrate's core logic
(e.g. Substrate's internal consensus, crypto or database code) to be contributed back so everyone
can benefit.