Chris Sosnin
6ef1117685
* Update substrate & polkadot * min changes to make async backing compile * (async backing) parachain-system: track limitations for unincluded blocks (#2438) * unincluded segment draft * read para head from storage proof * read_para_head -> read_included_para_head * Provide pub interface * add errors * fix unincluded segment update * BlockTracker -> Ancestor * add a dmp limit * Read para head depending on the storage switch * doc comments * storage items docs * add a sanity check on block initialize * Check watermark * append to the segment on block finalize * Move segment update into set_validation_data * Resolve para head todo * option watermark * fix comment * Drop dmq check * fix weight * doc-comments on inherent invariant * Remove TODO * add todo * primitives tests * pallet tests * doc comments * refactor unincluded segment length into a ConsensusHook (#2501) * refactor unincluded segment length into a ConsensusHook * add docs * refactor bandwidth_out calculation Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * test for limits from impl * fmt * make tests compile * update comment * uncomment test * fix collator test by adding parent to state proof * patch HRMP watermark rules for unincluded segment * get consensus-common tests to pass, using unincluded segment * fix unincluded segment tests * get all tests passing * fmt * rustdoc CI * aura-ext: limit the number of authored blocks per slot (#2551) * aura_ext consensus hook * reverse dependency * include weight into hook * fix tests * remove stray println Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> * fix test warning * fix doc link --------- Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com> Co-authored-by: Chris Sosnin <chris125_@live.com> * parachain-system: ignore go ahead signal once upgrade is processed (#2594) * handle goahead signal for unincluded segment * doc comment * add test * parachain-system: drop processed messages from inherent data (#2590) * implement `drop_processed_messages` * drop messages based on relay parent number * adjust tests * drop changes to mqc * fix comment * drop test * drop more dead code * clippy * aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658) * aura-ext: check slot in consensus hook * convert relay chain slot * Make relay chain slot duration generic * use fixed velocity hook for pallets with aura * purge timestamp inherent * fix warning * adjust runtime tests * fix slots in tests * Make `xcm-emulator` test pass for new consensus hook (#2722) * add pallets on_initialize * tests pass * add AuraExt on_init * ".git/.scripts/commands/fmt/fmt.sh" --------- Co-authored-by: command-bot <> --------- Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com> * update polkadot git refs * CollationGenerationConfig closure is now optional (#2772) * CollationGenerationConfig closure is now optional * fix test * propagate network-protocol-staging feature (#2899) * Feature Flagging Consensus Hook Type Parameter (#2911) * First pass * fmt * Added as default feature in tomls * Changed to direct dependency feature * Dealing with clippy error * Update pallets/parachain-system/src/lib.rs Co-authored-by: asynchronous rob <rphmeier@gmail.com> --------- Co-authored-by: asynchronous rob <rphmeier@gmail.com> * fmt * bump deps and remove warning * parachain-system: update RelevantMessagingState according to the unincluded segment (#2948) * mostly address 2471 with a bug introduced * adjust relevant messaging state after computing total * fmt * max -> min * fix test implementation of xcmp source * add test * fix test message sending logic * fix + test * add more to unincluded segment test * fmt --------- Co-authored-by: Chris Sosnin <chris125_@live.com> * Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864) * add a comment * refactor client/service utilities * deprecate start_collator * update parachain-template * update test-service in the same way * update polkadot-parachain crate * fmt * wire up new SubmitCollation message * some runtime utilities for implementing unincluded segment runtime APIs * allow parachains to configure their level of sybil-resistance when starting the network * make aura-ext compile * update to specify sybil resistance levels * fmt * specify relay chain slot duration in milliseconds * update Aura to explicitly produce Send futures also, make relay_chain_slot_duration a Duration * add authoring duration to basic collator and document params * integrate new basic collator into parachain-template * remove assert_send used for testing * basic-aura: only author when parent included * update polkadot-parachain-bin * fmt * some fixes * fixes * add a RelayNumberMonotonicallyIncreases * add a utility function for initializing subsystems * some logging for timestamp adjustment * fmt * some fixes for lookahead collator * add a log * update `find_potential_parents` to account for sessions * bound the loop * restore & deprecate old start_collator and start_full_node functions. * remove unnecessary await calls * fix warning * clippy * more clippy * remove unneeded logic * ci * update comment Co-authored-by: Marcin S. <marcin@bytedude.com> * (async backing) restore `CheckInherents` for backwards-compatibility (#2977) * bring back timestamp * Restore CheckInherents * revert to empty CheckInherents * make CheckInherents optional * attempt * properly end system blocks * add some more comments * ignore failing system parachain tests * update refs after main feature branch merge * comment out the offending tests because CI runs ignored tests * fix warnings * fmt * revert to polkadot master * cargo update -p polkadot-primitives -p sp-io --------- Co-authored-by: asynchronous rob <rphmeier@gmail.com> Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com> Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com> Co-authored-by: Marcin S. <marcin@bytedude.com> Co-authored-by: eskimor <eskimor@users.noreply.github.com> Co-authored-by: Andronik <write@reusable.software>
Cumulus ☁️
This repository contains both the Cumulus SDK and also specific chains implemented on top of this SDK.
If you only want to run a Polkadot Parachain Node, check out our container section.
Cumulus SDK
A set of tools for writing Substrate-based Polkadot parachains. Refer to the included overview for architectural details, and the Connect to a relay chain how-to guide for a guided walk-through of using these tools.
It's easy to write blockchains using Substrate, and the overhead of writing parachains' distribution, p2p, database, and synchronization layers should be just as low. This project aims to make it easy to write parachains for Polkadot by leveraging the power of Substrate.
Cumulus clouds are shaped sort of like dots; together they form a system that is intricate, beautiful and functional.
Consensus
parachain-consensus is a
consensus engine for Substrate
that follows a Polkadot
relay chain. This will run
a Polkadot node internally, and dictate to the client and synchronization algorithms which chain
to follow,
finalize,
and treat as best.
Collator
A Polkadot collator for the parachain is
implemented by the polkadot-parachain binary (previously called polkadot-collator).
You may run polkadot-parachain locally after building it or using one of the container option described here.
Relay Chain Interaction
To operate a parachain node, a connection to the corresponding relay chain is necessary. This can be achieved in one of two ways:
- Run a full relay chain node within the parachain node (default)
- Connect to an external relay chain node via WebSocket RPC
In-process Relay Chain Node
If an external relay chain node is not specified (default behavior), then a full relay chain node is spawned within the same process.
This node has all of the typical components of a regular Polkadot node and will have to fully sync with the relay chain to work.
Example command
polkadot-parachain \
--chain parachain-chainspec.json \
--tmp \
-- \
--chain relaychain-chainspec.json
External Relay Chain Node
An external relay chain node is connected via WebsSocket RPC by using the --relay-chain-rpc-urls
command line argument. This option accepts one or more space-separated WebSocket URLs to a full relay
chain node. By default, only the first URL will be used, with the rest as a backup in case the
connection to the first node is lost.
Parachain nodes using this feature won't have to fully sync with the relay chain to work, so in general they will use fewer system resources.
Note: At this time, any parachain nodes using this feature will still spawn a significantly cut-down relay chain node in-process. Even though they lack the majority of normal Polkadot subsystems, they will still need to connect directly to the relay chain network.
Example command
polkadot-parachain \
--chain parachain-chainspec.json \
--tmp \
--relay-chain-rpc-urls \
"ws://relaychain-rpc-endpoint:9944" \
"ws://relaychain-rpc-endpoint-backup:9944" \
-- \
--chain relaychain-chainspec.json
Installation and Setup
Before building Cumulus SDK based nodes / runtimes prepare your environment by following Substrate installation instructions.
To launch a local network, you can use zombienet for quick setup and experimentation or follow the manual setup.
Zombienet
We use Zombienet to spin up networks for integration tests and local networks. Follow these installation steps to set it up on your machine. A simple network specification with two relay chain nodes and one collator is located at zombienet/examples/small_network.toml.
Which provider should I use?
Zombienet offers multiple providers to run networks. Choose the one that best fits your needs:
- Podman: Choose this if you want to spin up a network quick and easy.
- Native: Choose this if you want to develop and deploy your changes. Requires compilation of the binaries.
- Kubernetes: Choose this for advanced use-cases or running on cloud-infrastructure.
How to run
To run the example network, use the following commands:
# Podman provider
zombienet --provider podman spawn ./zombienet/examples/small_network.toml
# Native provider, assumes polkadot and polkadot-parachains binary in $PATH
zombienet --provider native spawn ./zombienet/examples/small_network.toml
Manual Setup
Launch the Relay Chain
# Clone
git clone https://github.com/paritytech/polkadot
cd polkadot
# Compile Polkadot with the real overseer feature
cargo build --release --bin polkadot
# Generate a raw chain spec
./target/release/polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > rococo-local-cfde.json
# Alice
./target/release/polkadot --chain rococo-local-cfde.json --alice --tmp
# Bob (In a separate terminal)
./target/release/polkadot --chain rococo-local-cfde.json --bob --tmp --port 30334
Launch the Parachain
# Clone
git clone https://github.com/paritytech/cumulus
cd cumulus
# Compile
cargo build --release --bin polkadot-parachain
# Export genesis state
./target/release/polkadot-parachain export-genesis-state > genesis-state
# Export genesis wasm
./target/release/polkadot-parachain export-genesis-wasm > genesis-wasm
# Collator1
./target/release/polkadot-parachain --collator --alice --force-authoring --tmp --port 40335 --rpc-port 9946 -- --chain ../polkadot/rococo-local-cfde.json --port 30335
# Collator2
./target/release/polkadot-parachain --collator --bob --force-authoring --tmp --port 40336 --rpc-port 9947 -- --chain ../polkadot/rococo-local-cfde.json --port 30336
# Parachain Full Node 1
./target/release/polkadot-parachain --tmp --port 40337 --rpc-port 9948 -- --chain ../polkadot/rococo-local-cfde.json --port 30337
Register the parachain
Asset Hub 🪙
This repository also contains the Asset Hub runtimes. Asset Hub is a system parachain providing an asset store for the Polkadot ecosystem.
Build & Launch a Node
To run an Asset Hub node, you will need to compile the polkadot-parachain binary:
cargo build --release --locked --bin polkadot-parachain
Once the executable is built, launch the parachain node via:
CHAIN=asset-hub-westend # or asset-hub-kusama
./target/release/polkadot-parachain --chain $CHAIN
Refer to the setup instructions to run a local network for development.
Contracts 📝
See the contracts-rococo readme for details.
Bridge-hub 📝
See the bridge-hubs readme for details.
Rococo 👑
Rococo is becoming a Community Parachain Testbed for parachain teams in the Polkadot ecosystem. It supports multiple parachains with the differentiation of long-term connections and recurring short-term connections, to see which parachains are currently connected and how long they will be connected for see here.
Rococo is an elaborate style of design and the name describes the painstaking effort that has gone into this project.
Build & Launch Rococo Collators
Collators are similar to validators in the relay chain. These nodes build the blocks that will eventually be included by the relay chain for a parachain.
To run a Rococo collator you will need to compile the following binary:
cargo build --release --locked --bin polkadot-parachain
Once the executable is built, launch collators for each parachain (repeat once each for chain
tick, trick, track):
./target/release/polkadot-parachain --chain $CHAIN --validator
You can also build using a container.
Parachains
The network uses horizontal message passing (HRMP) to enable communication between parachains and the relay chain and, in turn, between parachains. This means that every message is sent to the relay chain, and from the relay chain to its destination parachain.