Commit Graph

6 Commits

Author SHA1 Message Date
Max Inden d742e88e79 client/authority-discovery: Allow to be run by sentry node (#5568)
* client/authority-discovery: Allow to be run by sentry node

When run as a sentry node, the authority discovery module does not
publish any addresses to the dht, but still discovers validators and
sentry nodes of validators.

* client/authority-discovery/src/lib: Wrap lines at 100 characters

* client/authority-discovery: Remove TODO and unused import

* client/authority-discovery: Pass role to new unit tests

* client/authority-discovery: Apply suggestions

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* bin/node/cli/src/service: Use expressions instead of statements

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-04-16 18:11:26 +01:00
Max Inden 61d64e2ca1 .maintain/sentry-node: Add monitoring to docker-compose stack (#5321)
* Substrate Dashboard example

* Improve README

* Update README_dashboard.md

* Add screenshots

* Minor fix

* Minor fix, image link

* .maintain/sentry-node: Add monitoring to docker-compose stack

With this patch a user can run the following fully configured and
monitored setup with a single command:

`docker-compose -f .maintain/sentry-node/docker-compose.yml up`

- 2 validators in two different network namespaces, connected via one
sentry node.

- Polkadot-js/apps to connect to one of the nodes above.

- Prometheus scraping the 3 Substrate nodes.

- Grafana displaying data from Prometheus with community dashboards

* .maintain/monitoring/grafana: Change default datasource name

* .maintain/monitoring/grafana: Add metric namespace option

* .maintain/monitoring/grafana: Remove `host` metric from most metrics

* .maintain/monitoring/grafana: Remove underscore from metric_namespace

* .maintain/monitoring: Use `instance` label instead of `hostname`

To identify a scrape target, one should use `instance` and not
`hostname` as multiple targets might run on the same node.

See https://prometheus.io/docs/concepts/jobs_instances/ for details.

* .maintain/monitoring: Introduce instance variable

* .maintain/monitoring/grafana: Rename substrate_block_height_number

* .maitain/monitoring/grafana: Use instance instead of host in legend

* .maintain/monitoring: Remove node exporter dependency

* .maintain/sentry-node/prometheus: Simplify configuration

* .maintain/monitoring/grafana: Update README and remove images

* .maintain/sentry-node: Improve docs

* .maintain/monitoring/grafana: Use metric_namespace template variable

* Use --sentry from v0.7.29 instead of a reserved-node

* .maintain/sentry-node: Revert sentry-a using validator-b as bootnode

Co-authored-by: DerFredy - @derfredy:matrix.org <derfredy@gmail.com>
Co-authored-by: david <davidd@custom.home>
2020-04-14 16:08:09 +02:00
Max Inden c9c7d655f1 .maintain/sentry-node/docker-compose: Expose Prometheus endpoin… (#5196)
This patch adds the Prometheus endpoint port of validator A to the
exposed ports to access it from the host network namespace.
2020-03-11 18:32:47 +01:00
Max Inden 5cf682cece client/authority-discovery: Limit number of connections to authorities (#4487)
* client/authority-discovery: Limit number of connections to authorities

Instead of connecting to all sentry nodes of all authorities, with this
patch the authority discovery module does the following:

- Choose one sentry node per authority at random.

- Choose MAX_NUM_AUTHORITY_CONN out of the above at random.

The module uses randomness to prevent hot spots, e.g. all nodes trying
to connect to a single node. If the authority discovery module would
choose the nodes to connect to at random on each new address that it
learns of, the node would go through a lot of connection churn.  Instead
it creates a random seed at start up and uses this seed for its RNG on
each update cycle.

* client/authority-discovery: Extract address cache into own module

* client/authority-discovery/src/addr_cache: Add basic unit tests

* client/authority-discovery: Replace unwrap with expect on [u8] cmp

* .maintain/sentry-node/docker-compose.yml: Prefix endpoint flags

* client/authority-discovery/src/addr_cache: Use sort_unstable and cmp

* client/authority-discovery: Use BTreeMap in addr_cache for sorted iter

To reduce connection churn it is preferrable to have `get_subset` of the
`addr_cache` to return the same result on repeated calls. `get_subset`
iterates a map. To make the process of iteration deterministic, use a
`BTreeMap` instead of a `HashMap`.
2020-01-03 20:40:19 +00:00
Max Inden 2de6f82b8a client: Introduce --sentry-nodes flag (#4285)
* client/authority-discovery: Add smoke tests for intervall_at

* client/authority-discovery: Fix interval_at to fire on start

* .maintain/sentry-node: Update docker compose file

* client: Introduce --sentry-nodes flag

Enable operators to specify the public addresses of sentry nodes infront
of a validator node so that the validator node can announce the sentry
node addresses instead of its own public addresses on the DHT via the
authority discovery module.

* client/authority-discovery: Break lines at 100 characters

Limit line length to 100 instead of 120 characters.
2019-12-03 18:03:08 +01:00
Benjamin Kampmann 60e5011c72 Reorganising the repository - external renames and moves (#4074)
* Adding first rough ouline of the repository structure

* Remove old CI stuff

* add title

* formatting fixes

* move node-exits job's script to scripts dir

* Move docs into subdir

* move to bin

* move maintainence scripts, configs and helpers into its own dir

* add .local to ignore

* move core->client

* start up 'test' area

* move test client

* move test runtime

* make test move compile

* Add dependencies rule enforcement.

* Fix indexing.

* Update docs to reflect latest changes

* Moving /srml->/paint

* update docs

* move client/sr-* -> primitives/

* clean old readme

* remove old broken code in rhd

* update lock

* Step 1.

* starting to untangle client

* Fix after merge.

* start splitting out client interfaces

* move children and blockchain interfaces

* Move trie and state-machine to primitives.

* Fix WASM builds.

* fixing broken imports

* more interface moves

* move backend and light to interfaces

* move CallExecutor

* move cli off client

* moving around more interfaces

* re-add consensus crates into the mix

* fix subkey path

* relieve client from executor

* starting to pull out client from grandpa

* move is_decendent_of out of client

* grandpa still depends on client directly

* lemme tests pass

* rename srml->paint

* Make it compile.

* rename interfaces->client-api

* Move keyring to primitives.

* fixup libp2p dep

* fix broken use

* allow dependency enforcement to fail

* move fork-tree

* Moving wasm-builder

* make env

* move build-script-utils

* fixup broken crate depdencies and names

* fix imports for authority discovery

* fix typo

* update cargo.lock

* fixing imports

* Fix paths and add missing crates

* re-add missing crates
2019-11-14 21:51:17 +01:00