pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-05-30 14:01:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Squashed 'bridges/' changes from 23dda62482..407bf44a8a

Browse Source 
407bf44a8a add missing license header (#1204)
9babb19810  Custom relay strategy (#1198)
c287872a11 fix clippy things (#1200)
3a40e62789 Expose some const value and type (#1186)
32b61476d1 increase sleep before connectingMillau (#1195)
aabe7041fa revert messages transactions mortality (#1194)
3651f4f909 Message transactions mortality (#1191)
364d6e155d Bump dependencies (#1180)
f0389acc08 cargo +nightly fmt --all (#1192)
b270b6a016 Unify error enums in substrate and ethereum clients with `thiserror` (#1094)
58c4946f74 Limit max call size of Rialto/Millau runtimes (#1187)
fd56a8cd56 Add UI to the deployment (#1047)
16f01dc736 Westend -> Millau alerts are pending before notifications are sent (#1184)
5628c11ece replace collective flip with babe randomness in Rialto (#1188)
1094a63b00 ignore another (pretty bad) RUSTSEC (#1185)
379fe323ea fix/ignore cargo deny issues (#1183)
92af5e6e64 additional log in finality relay + rephrase "failed" (#1182)
b996a3b681 Rialto parachain in test deployments (#1178)
28d9332b44 Resubmit transactions strategy for Polkadot/Kusama (#1175)
d0172c6847 Playing with CI (#1179)
fb6f42456d fix checks order when registering parachain (#1177)
ee828c005a Register-parachain subcommand of substrate-relay (#1170)
8cd2b1a112 Token swap pallet benchmarks (#1174)
bb811accb1 fix collision with westend bridge (#1172)
8d2fba70ed add token swaps to test deployments (#1169)
b6d1bdfe2c publish rialto parachain collator image (#1171)
834ae4a10a Fix OutboundLaneData types (#1159)
5ee0ea1626 copypasted -> copied (#1168)
c3bb835f18 fix spelling (#1167)
f90d041dc9 Upgrade `jsonrpsee` to v0.3 (#1051)
598c9b6d0d add some basic tests for swap tokens (#1164)
05e88c61f5 publish images when tag of specific format(e.g. v2021-09-27 + v2021-09-27-1) is published (#1166)
7f3f94a6e0 Fix CI again (#1165)
ff37de332f Move calculation relayer reward into `MessageDeliveryAndDispatchPayment` (#1153)
36fbba839b fix clippy warning (#1163)
16da44d018 explicit wasm build (#1158)
c9c8226449 Match substrate's fmt (#1148)
2fdd7f3e5e Fix/ignore clippy warnings (#1157)
43dfcc2686 Adding LookupAddress (#1156)
951eaa5582 Add rialto-parachain runtime and node (#1142)
803d266d61 Rename MessageId -> BridgeMessageId (#1152)
5f234484fc Box large arguments of GRANDPA pallet (#1154)
cf9abc1011 Fix spelling (#1150)
ab83ba2e58 Relay subcommand that performs token RLT <> MLAU token swap (#1141)
832536caf0 Polkadot <> Kusama relayers (#1122)
6d0daa8975 Add `OnMessageAccepted` callback (#1134)
5d03a20b3e Integrate token swap pallet into Millau runtime (#1099)
ea4cfa833e Adding MultiAddress type and ValidationCodeHash (#1139)
c20325a784 Add tests for `Raw` and `BridgeSendMessage` enum `Call` variants (#1125)
6d802416e2 increase pause before pining Rialto nodes (#1137)
b54fa56b62 calculate fee using full message payload (#1132)
ca5d8178f5 Add parachain pallets to rialto runtime (#1053)
9eaae4142e fix transaction resubmitter limits for Millau -> Rialto transactions (#1135)
9d4e17783c add --mandatory-headers-only cli option to complex relay (#1129)
1c5e0ec1cb Add local CI info to README (#1131)
a8e0929e14 chore: spellchecker fixes (#1130)
3b8e2118e3 set fee for importing mandatory headers to zero (#1127)
49bba9aa52 another bunch of words for spellchecker (#1128)
8a72eafef6 Increase pause before messages generation start (#1126)
1f0ba9a191 Move some associated types from relay_substrate_client::Chain to bp_runtime::Chain (#1087)
74bc1a5b54 Transactions resubmitter (#1083)
21ba001f26 log max balance drop when sending message (#1117)
638a7ddffa Code Cleaning (#1124)
be6555c51b Fix buildah logout (#1120)
87539c4a98 Format code work (#1116)
526fe7fdd7 fix spelling (#1119)
bd4ce7f241 Fix spelling (#1118)
3c1147858e added missing constants to Kusama/Polkadot primitives (#1114)
52093b22ab Fix delivery transaction estimation used by rational relayer (#1109)
77a2f2fbed Remove fund account checks from upgrade. (#1111)
824334802b Rename param and update comment (#1108)
d7784bfe06 Fix spellcheck (#1110)
0b18f5906a Refactor substrate messages source and substrate messages target (#1105)
b27240bbff fix compilation (#1107)
9697da4fe8 Emit mortal transactions from relay (#1073)
b29396c077 Change vault vars type to env vars (#1084)
35e0bbdc0c Make clippy mandatory. (#1103)
a517e8541f Remove unused deps (#1102)
873dae608a Remove unnessary deps (#1101)
13450b74ee Stored conversion rate updater (#1005)
74389829f3 [BREAKING] Migrate messages pallet to frame v2 (#1088)
424da938dd README fix (#1100)
865744c909 upgrade currency exchange pallet to frame v2 (#1097)
b5038148b3 Add missing docs (#1095)
0791e911c1 Common crate for substrate-relay (#1082)
3834c9d880 Update high-level-overview.md (#1093)
c93553face Increase the time window for messaging alerts. (#1092)
8b9cc3cecd migrate pallet-shift-session-manager to frame v2 (#1090)
dc91813c22 migrate eth PoA pallet to frame v2 (#1091)
f16bb098cc Migrate dispatch pallet  to frame v2 (#1089)
19f4325348 Bridge/This Chain Ids should be exposed as constants on pallet level. (#1085)
6381122df7 Change ChainSpec::from_genesis for Rialto and Millau chains to reflect the chain names. (#1079)
0f1d33e973 Make CI happy again (#1086)
238e65d96f fix typo (#1080)
fc008457b6 Token-swap-over-bridge pallet (#944)
3fb97fa5ef Fix full spellcheck (#1076)
eae4ed7170 fixed wrong trace (#1075)
219a0fad04 merge two weight-related loops in messages pallet (#1071)
fc85632fdb increase_message_fee depends on stored mesage size (#1066)
530f37a23b companion for https://github.com/paritytech/polkadot/pull/3507 (#1067)
53b8cba683 sc_basic_authorship=trace for millau nodes (#1074)
9874e05e98 Improve traces of message generator scripts (#1069)
7b5ee84fbb extract message_details impl into runtime common (#1070)
5a4aed5a8b refund weight for mot pruning messages (#1062)
90e3d1e111 Fix Westend -> Millau sync (#1064)
427d30ddfc When restarting client, also "restart" tokio runtime (#1065)
d47c05eeef Change get pipeline sensitive variables from Vault instead of GitLab settings (#1063)
d775a85415 use tokio reactor to execute jsonrpsee futures (#1061)
15c8cd61cb Use BABE to author blocks on Rialto (previously: Aura) (#1050)
5186293500 Allow reading suri && password override from file (#1059)
b506298262 Update jsonrpsee reference (#1049)
1734d00517 enable weight fee adjustent in Rialto/Millau (#1044)
607265afae Pay dispatch fee at target chain cli option (#1043)
ce79ef91be bump dependencies before start referencing polkadot repo (#1048)
924fa24f6d Cli option for greedy relayer + run no-losses relayer by default (#1042)
e21eba7b59 Yrong README Fixup + M1 Fixes (#1045)
20d08204a2 Confirm delivery detects when more than expected messages are confirmed (#1039)
994b846b52 pre and post dispatch weights of OnDeliveryConfirmed callback (#1040)
1dd5297e84 give real value to Rialto and Millau tokens (#1038)
035bee8715 Use real conversion rate in greedy relayer strategy (#1035)
9cfaecd0f7 fixed metrics prefix (#1037)
1d8d224937 Use kebab-case for bridge arguments (#1036)
f30a4c79a6 Shared reference to conversion rate metric value (#1034)
c34d7a5cbb estimate transaction fee (#1015)
93404b18bb change alert period from 2m to 10m for Westend -> Millau (GRANDPA or public node itself is lagging sometimes) (#1032)

git-subtree-dir: bridges
git-subtree-split: 407bf44a8a5f4e60aceef2dc755cd9ff09929ac3
This commit is contained in:
antonio-dropulic
2021-12-01 09:24:53 +01:00
parent feefc34567
commit 392447f5c8
1020 changed files with 30080 additions and 179754 deletions
Download Patch File Download Diff File Expand all files Collapse all files
polkadot/bridges/deployments/README.md
-254
View File
@@ -1,254 +0,0 @@
# Bridge Deployments
## Requirements
Make sure to install `docker` and `docker-compose` to be able to run and test bridge deployments. If
for whatever reason you can't or don't want to use Docker, you can find some scripts for running the
bridge [here](https://github.com/svyatonik/parity-bridges-common.test).
## Networks
One of the building blocks we use for our deployments are _networks_. A network is a collection of
homogenous blockchain nodes. We have Docker Compose files for each network that we want to bridge.
Each of the compose files found in the `./networks` folder is able to independently spin up a
network like so:
```bash
docker-compose -f ./networks/rialto.yml up
```
After running this command we would have a network of several nodes producing blocks.
## Bridges
A _bridge_ is a way for several _networks_ to connect to one another. Bridge deployments have their
own Docker Compose files which can be found in the `./bridges` folder. These Compose files typically
contain bridge relayers, which are services external to blockchain nodes, and other components such
as testing infrastructure, or user interfaces.
Unlike the network Compose files, these *cannot* be deployed on their own. They must be combined
with different networks.
In general, we can deploy the bridge using `docker-compose up` in the following way:
```bash
docker-compose -f <bridge>.yml \
-f <network_1>.yml \
-f <network_2>.yml \
-f <monitoring>.yml up
```
If you want to see how the Compose commands are actually run, check out the source code of the
[`./run.sh`](./run.sh).
One thing worth noting is that we have a _monitoring_ Compose file. This adds support for Prometheus
and Grafana. We cover these in more details in the [Monitoring](#monitoring) section. At the moment
the monitoring Compose file is _not_ optional, and must be included for bridge deployments.
### Running and Updating Deployments
We currently support two bridge deployments
1. Ethereum PoA to Rialto Substrate
2. Rialto Substrate to Millau Substrate
These bridges can be deployed using our [`./run.sh`](./run.sh) script.
The first argument it takes is the name of the bridge you want to run. Right now we only support two
bridges: `poa-rialto` and `rialto-millau`.
```bash
./run.sh poa-rialto
```
If you add a second `update` argument to the script it will pull the latest images from Docker Hub
and restart the deployment.
```bash
./run.sh rialto-millau update
```
You can also bring down a deployment using the script with the `stop` argument.
```bash
./run.sh poa-rialto stop
```
### Adding Deployments
We need two main things when adding a new deployment. First, the new network which we want to
bridge. A compose file for the network should be added in the `/networks/` folder. Secondly we'll
need a new bridge Compose file in `./bridges/`. This should configure the bridge relayer nodes
correctly for the two networks, and add any additional components needed for the deployment. If you
want you can also add support in the `./run` script for the new deployment. While recommended it's
not strictly required.
## General Notes
Rialto authorities are named: `Alice`, `Bob`, `Charlie`, `Dave`, `Eve`.
Rialto-PoA authorities are named: `Arthur`, `Bertha`, `Carlos`.
Millau authorities are named: `Alice`, `Bob`, `Charlie`, `Dave`, `Eve`.
Both authorities and following accounts have enough funds (for test purposes) on corresponding Substrate chains:
- on Rialto: `Ferdie`, `George`, `Harry`.
- on Millau: `Ferdie`, `George`, `Harry`.
Names of accounts on Substrate (Rialto and Millau) chains may be prefixed with `//` and used as
seeds for the `sr25519` keys. This seed may also be used in the signer argument in Substrate
and PoA relays. Example:
```bash
./substrate-relay relay-headers RialtoToMillau \
--source-host rialto-node-alice \
--source-port 9944 \
--target-host millau-node-alice \
--target-port 9944 \
--source-signer //Harry \
--prometheus-host=0.0.0.0
```
Some accounts are used by bridge components. Using these accounts to sign other transactions
is not recommended, because this may lead to nonces conflict.
Following accounts are used when `poa-rialto` bridge is running:
- Rialto's `Alice` signs relay transactions with new Rialto-PoA headers;
- Rialto's `Bob` signs relay transactions with Rialto-PoA -> Rialto currency exchange proofs.
- Rialto-PoA's `Arthur`: signs relay transactions with new Rialto headers;
- Rialto-PoA's `Bertha`: signs currency exchange transactions.
Following accounts are used when `rialto-millau` bridge is running:
- Millau's `Charlie` signs complex headers+messages relay transactions on Millau chain;
- Rialto's `Charlie` signs complex headers+messages relay transactions on Rialto chain;
- Millau's `Dave` signs Millau transactions which contain messages for Rialto;
- Rialto's `Dave` signs Rialto transactions which contain messages for Millau;
- Millau's `Eve` signs relay transactions with message delivery confirmations (lane 00000001) from Rialto to Millau;
- Rialto's `Eve` signs relay transactions with messages (lane 00000001) from Millau to Rialto;
- Millau's `Ferdie` signs relay transactions with messages (lane 00000001) from Rialto to Millau;
- Rialto's `Ferdie` signs relay transactions with message delivery confirmations (lane 00000001) from Millau to Rialto.
Following accounts are used when `westend-millau` bridge is running:
- Millau's `George` signs relay transactions with new Westend headers.
### Docker Usage
When the network is running you can query logs from individual nodes using:
```bash
docker logs rialto_poa-node-bertha_1 -f
```
To kill all left over containers and start the network from scratch next time:
```bash
docker ps -a --format "{{.ID}}" | xargs docker rm # This removes all containers!
```
### Docker Compose Usage
If you're not familiar with how to use `docker-compose` here are some useful commands you'll need
when interacting with the bridge deployments:
```bash
docker-compose pull # Get the latest images from the Docker Hub
docker-compose build # This is going to build images
docker-compose up # Start all the nodes
docker-compose up -d # Start the nodes in detached mode.
docker-compose down # Stop the network.
```
Note that for the you'll need to add the appropriate `-f` arguments that were mentioned in the
[Bridges](#bridges) section. You can read more about using multiple Compose files
[here](https://docs.docker.com/compose/extends/#multiple-compose-files). One thing worth noting is
that the _order_ the compose files are specified in matters. A different order will result in a
different configuration.
You can sanity check the final config like so:
```bash
docker-compose -f docker-compose.yml -f docker-compose.override.yml config > docker-compose.merged.yml
```
## Docker and Git Deployment
It is also possible to avoid using images from the Docker Hub and instead build
containers from Git. There are two ways to build the images this way.
### Git Repo
If you have cloned the bridges repo you can build local Docker images by running the following
command at the top level of the repo:
```bash
docker build . -t local/<project_you're_building> --build-arg=PROJECT=<project>
```
This will build a local image of a particular component with a tag of
`local/<project_you're_building>`. This tag can be used in Docker Compose files.
You can configure the build using using Docker
[build arguments](https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg).
Here are the arguments currently supported:
- `BRIDGE_REPO`: Git repository of the bridge node and relay code
- `BRIDGE_HASH`: Commit hash within that repo (can also be a branch or tag)
- `ETHEREUM_REPO`: Git repository of the OpenEthereum client
- `ETHEREUM_HASH`: Commit hash within that repo (can also be a branch or tag)
- `PROJECT`: Project to build withing bridges repo. Can be one of:
- `rialto-bridge-node`
- `millau-bridge-node`
- `ethereum-poa-relay`
- `substrate-relay`
### GitHub Actions
We have a nightly job which runs and publishes Docker images for the different nodes and relayers to
the [ParityTech Docker Hub](https://hub.docker.com/u/paritytech) organization. These images are used
for our ephemeral (temporary) test networks. Additionally, any time a tag in the form of `v*` is
pushed to GitHub the publishing job is run. This will build all the components (nodes, relayers) and
publish them.
With images built using either method, all you have to do to use them in a deployment is change the
`image` field in the existing Docker Compose files to point to the tag of the image you want to use.
### Monitoring
[Prometheus](https://prometheus.io/) is used by the bridge relay to monitor information such as system
resource use, and block data (e.g the best blocks it knows about). In order to visualize this data
a [Grafana](https://grafana.com/) dashboard can be used.
As part of the Rialto `docker-compose` setup we spin up a Prometheus server and Grafana dashboard. The
Prometheus server connects to the Prometheus data endpoint exposed by the bridge relay. The Grafana
dashboard uses the Prometheus server as its data source.
The default port for the bridge relay's Prometheus data is `9616`. The host and port can be
configured though the `--prometheus-host` and `--prometheus-port` flags. The Prometheus server's
dashboard can be accessed at `http://localhost:9090`. The Grafana dashboard can be accessed at
`http://localhost:3000`. Note that the default log-in credentials for Grafana are `admin:admin`.
### Environment Variables
Here is an example `.env` file which is used for production deployments and network updates. For
security reasons it is not kept as part of version control. When deploying a network this
file should be correctly populated and kept in the appropriate [`bridges`](`./bridges`) deployment
folder.
The `UI_SUBSTRATE_PROVIDER` variable lets you define the url of the Substrate node that the user
interface will connect to. `UI_ETHEREUM_PROVIDER` is used only as a guidance for users to connect
Metamask to the right Ethereum network. `UI_EXPECTED_ETHEREUM_NETWORK_ID` is used by
the user interface as a fail safe to prevent users from connecting their Metamask extension to an
unexpected network.
```bash
GRAFANA_ADMIN_PASS=admin_pass
GRAFANA_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/
GRAFANA_SERVER_DOMAIN=server.domain.io
MATRIX_ACCESS_TOKEN="access-token"
WITH_PROXY=1 # Optional
UI_SUBSTRATE_PROVIDER=ws://localhost:9944
UI_ETHEREUM_PROVIDER=http://localhost:8545
UI_EXPECTED_ETHEREUM_NETWORK_ID=105
```
### UI
Use [wss://rialto.bridges.test-installations.parity.io/](https://polkadot.js.org/apps/)
as a custom endpoint for [https://polkadot.js.org/apps/](https://polkadot.js.org/apps/).
### Polkadot.js UI
To teach the UI decode our custom types used in the pallet, go to: `Settings -> Developer`
and import the [`./types.json`](./types.json)
## Scripts
The are some bash scripts in `scripts` folder that allow testing `Relay`
without running the entire network within docker. Use if needed for development.