Commit Graph
83 Commits
Author SHA1 Message Date
Andrei Sanduandordian 039d183be9 adder-collator: add velocity measurement and make elastic scaling test more robust (#4016)
Improves `adder-collator` to also compute the parachain velocity. The
velocity is defined as number of parachain blocks progressing per relay
chain block.

In this test we're asserting that the elastic parachain always
progresses by 3 blocks per RCB, while the non-elastic parachain
progresses normally - 1 block per RCB.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: ordian <write@reusable.software>
2024-04-08 12:29:35 +00:00
Andrei Sandu 0832f0f36d Rococo/Westend: publish claim_queue Runtime API (#4005)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-04-05 15:50:43 +00:00
Andrei SanduandJavier Viola 0dc0d407c1 Elastic scaling: add e2e test (#3929)
On top of https://github.com/paritytech/polkadot-sdk/pull/3879

I've also moved the previous test where we ensure multiple cores per
para doesn't break non elastic parachains.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
2024-04-05 09:06:43 +00:00
Andrei Sandu 0f4e849e0a Add ClaimQueue wrapper (#3950)
Remove `fetch_next_scheduled_on_core` in favor of new wrapper and
methods for accessing it.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-04-03 15:01:34 +00:00
Andrei Sandu e8e201f0ff statement-distribution: fix filtering of statements for elastic parachains (#3879)
fixes https://github.com/paritytech/polkadot-sdk/issues/3775

Additionally moves the claim queue fetch utilities into
`subsystem-util`.

TODO:
- [x] fix tests
- [x] add elastic scaling tests

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-04-03 08:34:50 +00:00
Andrei Sandu bf1ca86f87 pallet-scheduler: fix test (#3923)
fix https://github.com/paritytech/polkadot-sdk/issues/3921

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-04-01 21:36:18 +00:00
Andrei Sandu 07720dd120 Improve HostConfiguration consistency check (#3897)
fixes https://github.com/paritytech/polkadot-sdk/issues/3886

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-04-01 09:23:44 +00:00
Andrei Sandu 30ef8651ed collation-genereation: fix tests (#3883)
Somehow https://github.com/paritytech/polkadot-sdk/pull/3795 was merged
but tests are failing now on master. I suspect that CI is not even
running these tests anymore which is a big issue.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-03-29 07:49:34 +00:00
Andrei Sandu 417c54c61c collation-generation + collator-protocol: collate on multiple assigned cores (#3795)
This works only for collators that implement the `collator_fn` allowing
`collation-generation` subsystem to pull collations triggered on new
heads.

Also enables
`request_v2::CollationFetchingResponse::CollationWithParentHeadData` for
test adder/undying collators.

TODO:
- [x] fix tests
- [x] new tests
- [x] PR doc

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-03-27 14:44:10 +00:00
Andrei Sanduandordian 66051adb61 testnet genesis: enable approval voting v2 assignments and coalescing (#3827)
This is a long due chore ...

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: ordian <write@reusable.software>
2024-03-27 04:16:03 +00:00
Andrei Sandu 4987d79824 Add elastic scaling support in ParaInherent BenchBuilder (#3690)
Extracted Benchbuilder enhancements used in
https://github.com/paritytech/polkadot-sdk/pull/3644 . Might still
require some work to fully support all scenarios when disputing elastic
scaling parachains, but it should be useful in writing elastic scaling
runtime tests.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-03-15 11:42:58 +00:00
Andrei Sandu c16fcc47bc ParaInherent create: update apply_weight_limit wrt elastic scaling (#3573)
Changes the way we perform the random selection of backed candidates
when there isn't enough room for all of them. Instead of picking
individual backed candidates `apply_weight` now operates on chains of
candidates. This is fully backwards compatible and relies on the node
side (provisioner/prospective parachains) doing the heavy lifting and
providing the candidates in the order they form a chain.

The same approach can be implemented for bitfields random selection once
https://github.com/paritytech/polkadot-sdk/pull/3479 is merged.

The approach taken in this PR aims for reduced additional complexity at
the cost of being less fair wrt how many backed candidates from each
chain are picked. It favors elastic scaling parachains vs parachains not
using elastic scaling, but from my perspective it should be fine as this
should happen under exceptional circumstances like dispute storms.

Note: to make things more fair we can consider specializing `random_sel`
such that it will try to pick candidates one by one in the order
provided by the provisioner such that non elastic scaling parachains
have the same chance of getting a candidate backed.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-03-07 09:11:43 +00:00
Andrei Sandu 7f5d308d29 Enable elastic scaling node feature in local testnets genesis (#3509)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-02-29 13:32:58 +00:00
Andrei Sanduandalindima 2431001ec0 Runtime: allow backing multiple candidates of same parachain on different cores (#3231)
Fixes https://github.com/paritytech/polkadot-sdk/issues/3144

Builds on top of https://github.com/paritytech/polkadot-sdk/pull/3229

### Summary
Some preparations for Runtime to support elastic scaling, guarded by
config node features bit `FeatureIndex::ElasticScalingMVP`. This PR
introduces a per-candidate `CoreIndex` but does it in a hacky way to
avoid changing `CandidateCommitments`, `CandidateReceipts` primitives
and networking protocols.

#### Including `CoreIndex` in `BackedCandidate`
If the `ElasticScalingMVP` feature bit is enabled then
`BackedCandidate::validator_indices` is extended by 8 bits.
The value stored in these bits represents the assumed core index for the
candidate.

It is temporary solution which works by creating a mapping from
`BackedCandidate` to `CoreIndex` by assuming the `CoreIndex` can be
discovered by checking in which validator group the validator that
signed the statement is.

TODO:
- [x] fix tests
- [x] add new tests
- [x] Bump runtime API for Kusama, so we have that node features thing!
-> https://github.com/polkadot-fellows/runtimes/pull/194

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: alindima <alin@parity.io>
Co-authored-by: alindima <alin@parity.io>
2024-02-23 16:35:48 +00:00
Andrei Sanduandalindima 60e537b95f Elastic scaling: use an assumed CoreIndex in candidate-backing (#3229)
First step in implementing
https://github.com/paritytech/polkadot-sdk/issues/3144

### Summary of changes
- switch statement `Table` candidate mapping from `ParaId` to
`CoreIndex`
- introduce experimental `InjectCoreIndex`  node feature.
- determine and assume a `CoreIndex` for a candidate based on statement
validator index. If the signature is valid it means validator controls
the validator that index and we can easily map it to a validator
group/core.
- introduce a temporary provisioner fix until we fully enable elastic
scaling in the subystem. The fix ensures we don't fetch the same
backable candidate when calling `get_backable_candidate` for each core.

TODO:
- [x] fix backing tests
- [x] fix statement table tests
- [x] add new test

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: alindima <alin@parity.io>
Co-authored-by: alindima <alin@parity.io>
2024-02-22 08:22:31 +00:00
Andrei Sandu 4220503d28 Chore: link code TODO for removing redundant message information (#3126)
fixes #675

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-01-30 21:55:02 +00:00
Andrei Sandu 7f28ac6204 Remove duplicate subsystem-bench from Cargo.toml (#3073)
🤦🏼

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-01-26 10:55:38 +00:00
Andrei Sandu 47e46d178b Add subsystem benchmarks for availability-distribution and biftield-distribution (availability write) (#2970)
Introduce a new test objective : `DataAvailabilityWrite`.

The new benchmark measures the network and cpu usage of
`availability-distribution`, `biftield-distribution` and
`availability-store` subsystems from the perspective of a validator node
during the process when candidates are made available.

Additionally I refactored the networking emulation to support bandwidth
acounting and limits of incoming and outgoing requests.

Screenshot of succesful run


<img width="1293" alt="Screenshot 2024-01-17 at 19 17 44"
src="https://github.com/paritytech/polkadot-sdk/assets/54316454/fde11280-e25b-4dc3-9dc9-d4b9752f9b7a">

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-01-25 17:02:24 +00:00
Andrei Sandu b4dfad838c approval-distribution: aggresion must target unfinalized chain rather than unapproved chain (#2988)
Found the issue while investigating the recent finality stall on Westend
after upgrading to 1.6.0. Approval distribution aggression is supposed
to trade off bandwidth and re-send assignemnts/approvals until enough
approvals are be received by at least 2/3 validators. This is supposed
to be a catch all mechanism when network connectivity goes south or many
validators reboot at the same time.

This fix ensures that we always resend approvals starting with the first
unfinalized block even in the case when it appears approved from the
node's perspective.

TODO:
- [x] Versi test

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-01-23 12:59:09 +00:00
Andrei Sandu b469960461 approval-distribution: downgrade errors to debug (#2989)
This is not actually an error of the node, but an issue with the
incoming assignment.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2024-01-18 17:28:55 +00:00
Andrei Sandu 8a6e9ef189 Introduce subsystem benchmarking tool (#2528)
This tool makes it easy to run parachain consensus stress/performance
testing on your development machine or in CI.

## Motivation
The parachain consensus node implementation spans across many modules
which we call subsystems. Each subsystem is responsible for a small part
of logic of the parachain consensus pipeline, but in general the most
load and performance issues are localized in just a few core subsystems
like `availability-recovery`, `approval-voting` or
`dispute-coordinator`. In the absence of such a tool, we would run large
test nets to load/stress test these parts of the system. Setting up and
making sense of the amount of data produced by such a large test is very
expensive, hard to orchestrate and is a huge development time sink.

## PR contents
- CLI tool 
- Data Availability Read test
- reusable mockups and components needed so far
- Documentation on how to get started

### Data Availability Read test

An overseer is built with using a real `availability-recovery` susbsytem
instance while dependent subsystems like `av-store`, `network-bridge`
and `runtime-api` are mocked. The network bridge will emulate all the
network peers and their answering to requests.

The test is going to be run for a number of blocks. For each block it
will generate send a “RecoverAvailableData” request for an arbitrary
number of candidates. We wait for the subsystem to respond to all
requests before moving to the next block.
At the same time we collect the usual subsystem metrics and task CPU
metrics and show some nice progress reports while running.

### Here is how the CLI looks like:

```
[2023-11-28T13:06:27Z INFO  subsystem_bench::core::display] n_validators = 1000, n_cores = 20, pov_size = 5120 - 5120, error = 3, latency = Some(PeerLatency { min_latency: 1ms, max_latency: 100ms })
[2023-11-28T13:06:27Z INFO  subsystem-bench::availability] Generating template candidate index=0 pov_size=5242880
[2023-11-28T13:06:27Z INFO  subsystem-bench::availability] Created test environment.
[2023-11-28T13:06:27Z INFO  subsystem-bench::availability] Pre-generating 60 candidates.
[2023-11-28T13:06:30Z INFO  subsystem-bench::core] Initializing network emulation for 1000 peers.
[2023-11-28T13:06:30Z INFO  subsystem-bench::availability] Current block 1/3
[2023-11-28T13:06:30Z INFO  substrate_prometheus_endpoint] 〽️ Prometheus exporter started at 127.0.0.1:9999
[2023-11-28T13:06:30Z INFO  subsystem_bench::availability] 20 recoveries pending
[2023-11-28T13:06:37Z INFO  subsystem_bench::availability] Block time 6262ms
[2023-11-28T13:06:37Z INFO  subsystem-bench::availability] Sleeping till end of block (0ms)
[2023-11-28T13:06:37Z INFO  subsystem-bench::availability] Current block 2/3
[2023-11-28T13:06:37Z INFO  subsystem_bench::availability] 20 recoveries pending
[2023-11-28T13:06:43Z INFO  subsystem_bench::availability] Block time 6369ms
[2023-11-28T13:06:43Z INFO  subsystem-bench::availability] Sleeping till end of block (0ms)
[2023-11-28T13:06:43Z INFO  subsystem-bench::availability] Current block 3/3
[2023-11-28T13:06:43Z INFO  subsystem_bench::availability] 20 recoveries pending
[2023-11-28T13:06:49Z INFO  subsystem_bench::availability] Block time 6194ms
[2023-11-28T13:06:49Z INFO  subsystem-bench::availability] Sleeping till end of block (0ms)
[2023-11-28T13:06:49Z INFO  subsystem_bench::availability] All blocks processed in 18829ms
[2023-11-28T13:06:49Z INFO  subsystem_bench::availability] Throughput: 102400 KiB/block
[2023-11-28T13:06:49Z INFO  subsystem_bench::availability] Block time: 6276 ms
[2023-11-28T13:06:49Z INFO  subsystem_bench::availability] 
    
    Total received from network: 415 MiB
    Total sent to network: 724 KiB
    Total subsystem CPU usage 24.00s
    CPU usage per block 8.00s
    Total test environment CPU usage 0.15s
    CPU usage per block 0.05s
```

### Prometheus/Grafana stack in action
<img width="1246" alt="Screenshot 2023-11-28 at 15 11 10"
src="https://github.com/paritytech/polkadot-sdk/assets/54316454/eaa47422-4a5e-4a3a-aaef-14ca644c1574">
<img width="1246" alt="Screenshot 2023-11-28 at 15 12 01"
src="https://github.com/paritytech/polkadot-sdk/assets/54316454/237329d6-1710-4c27-8f67-5fb11d7f66ea">
<img width="1246" alt="Screenshot 2023-11-28 at 15 12 38"
src="https://github.com/paritytech/polkadot-sdk/assets/54316454/a07119e8-c9f1-4810-a1b3-f1b7b01cf357">

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-12-14 12:57:17 +02:00
Andrei SanduandAlexandru Gheorghe 0570b6fa9e approval-voting improvement: include all tranche0 assignments in one certificate (#1178)
**_PR migrated from https://github.com/paritytech/polkadot/pull/6782_** 

This PR will upgrade the network protocol to version 3 -> VStaging which
will later be renamed to V3. This version introduces a new kind of
assignment certificate that will be used for tranche0 assignments.
Instead of issuing/importing one tranche0 assignment per candidate,
there will be just one certificate per relay chain block per validator.
However, we will not be sending out the new assignment certificates,
yet. So everything should work exactly as before. Once the majority of
the validators have been upgraded to the new protocol version we will
enable the new certificates (starting at a specific relay chain block)
with a new client update.

There are still a few things that need to be done:

- [x] Use bitfield instead of Vec<CandidateIndex>:
https://github.com/paritytech/polkadot/pull/6802
  - [x] Fix existing approval-distribution and approval-voting tests
  - [x] Fix bitfield-distribution and statement-distribution tests
  - [x] Fix network bridge tests
  - [x] Implement todos in the code
  - [x] Add tests to cover new code
  - [x] Update metrics
  - [x] Remove the approval distribution aggression levels: TBD PR
  - [x] Parachains DB migration 
  - [x] Test network protocol upgrade on Versi
  - [x] Versi Load test
  - [x] Add Zombienet test
  - [x] Documentation updates
- [x] Fix for sending DistributeAssignment for each candidate claimed by
a v2 assignment (warning: Importing locally an already known assignment)
 - [x]  Fix AcceptedDuplicate
 - [x] Fix DB migration so that we can still keep old data.
 - [x] Final Versi burn in

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
2023-11-06 15:21:32 +02:00
Andrei Sandu e165eacd58 Logs: add extra debug log for negative rep changes (#1205)
* refactor and more debug

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 🤦

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-09-01 14:55:23 +03:00
Andrei Sandu 73915d3459 add missing feature (#1310)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-08-30 20:06:42 +02:00
Andrei SanduandJavier Viola b46f07ff71 Fix polkadot zombienet tests (#1276)
* fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* ZOMBIENET_INTEGRATION_TEST_SECONDARY_IMAGE

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* deleted by mistake

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove LOCAL_DIR override

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix secondary image

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix get BUILD_RELEASE_VERSION in pipeline

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
2023-08-30 17:58:37 +03:00
Andrei Sandu 57169b22b9 ParachainsInherent: apply weight limit only in ProvideInherent (#7530)
* move filtering and panic in enter

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 🤦🤦🤦 Test only `polkadot-runtime-parachains`

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-07-25 09:00:44 +00:00
Andrei Sandu a200d4c9d7 uncompressed pov metrics (#7524)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-07-20 07:40:47 +00:00
Andrei Sandu e8d567a1f5 backing: Remove redundant erasure encoding (#7469)
* Remove redundant erasure encoding

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-07-07 08:20:30 +00:00
Andrei Sandu a0814490d2 availability-recovery: move cpu burners in blocking tasks (#7417)
* Move expensive computations to blocking thread

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add internal error and fix dependent subystems

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix test fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* minor refactor and TODOs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Impl Feedback for Review

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* More docs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add some example timings in comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-07-04 09:50:49 +00:00
Andrei Sandu 02d3fd025d availability recovery: measure re-encoding time (#7409)
* Measure re-encoding time

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-06-21 17:02:57 +03:00
Andrei Sandu edfbbc8c51 adjust buckets (#7368)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-06-14 15:02:53 +00:00
Andrei Sandu 2ca3750f0f Prefer fetching small PoVs from backing group (#7173)
* impl QueryChunkSize

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* QueryChunkSize message

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* enable fetching from backing group for small pov

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Refactor `bypass_availability_store`

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-05-05 09:56:54 +00:00
Andrei Sandu 634b2f6ad2 Fix rolling session window (#7126)
* Use stored sessions 🤦

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove redundant

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-04-24 18:14:36 +00:00
Andrei Sandu ecb6523b5c approval distribution: trigger assignment/votes resend based on approval checking lag (#7038)
* Send lag update message

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Process ApprovalCheckingLagUpdate

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Comput min age based on lag

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix test build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Make the spawnhandle optional

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove unused

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-04-13 10:10:53 +00:00
Andrei Sandu fc2cba1805 Subsystem channel tweaks (#6905)
* subsystem tweaks for higher scale

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Adjust queue sizes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Orchestra 0.0.5

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-03-24 17:19:37 +00:00
Andrei Sandu bd209c2d70 inherent disputes: remove per block initializer and disputes timeout event (#6937)
* Limit disputes weight and remove initializer code

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* const

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove timeout test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback #1

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove the new weight limiting for disputes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove dispute_conclusion_by_time_out_period

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Enable migrations

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update guide

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* More guide fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Also migrate pending configs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-03-24 12:48:56 +01:00
Andrei Sandu b59d0b832d Add pov_size and code_size histograms (#6633)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-01-26 13:55:45 +02:00
Andrei Sandu f552df7bbf Add task type label to metrics (#13240)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-01-26 06:22:51 +00:00
Andrei Sandu 7a961f303a Remove unused code (#6525)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2023-01-09 11:34:30 +00:00
Andrei Sandu 7f020eb527 approval-distribution: batched approval/assignment sending (#6401)
* Imple batched send

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add batch tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* pub MAX_NOTIFICATION_SIZE

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spell check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck ...

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* o.O

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 2 constants

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Ensure batch size is at least 1 element

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* feedback impl

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-12-13 14:06:05 +02:00
Andrei Sandu eb00a21ef7 dispute-coordinator: fix earliest session checks for pruning and import (#6358)
* RollingSession: add fn contains

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* handle_import_statements fix ancient dispute check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* work with earliest session instead of latest

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-11-28 17:43:13 +01:00
Andrei Sandu ad41e56e6e Dedup subsystem name (#6305)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-11-17 12:59:07 +00:00
Andrei Sandu c261353380 Make rolling session more resilient in case of long finality stalls (#6106)
* Impl dynamic window size. Keep sessions for unfinalized chain

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Stretch also in contructor plus  tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix approval-voting tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* grunting: dispute coordinator tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add session window column

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* integrate approval vote and fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix rolling session tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Small refactor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* WIP, tests failing

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix approval voting tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix dispute-coordinator tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove uneeded param

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix loose ends

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* allow failure and tests for it

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* comment fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* style fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* new col doesn't need to be ordered

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt and spellcheck

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* db persist tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add v2 config and cols

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* DB upgrade WIP

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add todo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update to parity-db to "0.4.2"

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* migration complete

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* One session window size

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build errors

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* comment fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* make error more explicit

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* refactor conflict merge

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* rename col_data

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add doc comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* migration: move all cols to v2

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-11-08 18:16:31 +01:00
Andrei Sandu 351d37e324 Check if approval voting db is empty on startup (#6219)
* Add approval voting db sanity check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add newline back

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* no need for overlay to read

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-10-31 20:57:50 +01:00
Andrei Sandu dc555b9510 Expose node subcommands in Malus CLI (#6135)
* Expose the full Cli through malus

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix lonely test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-10-11 12:07:28 +00:00
Andrei Sandu 7114a8cfca Keep sessions in window for the full unfinalized chain (#6054)
* Impl dynamic window size. Keep sessions for unfinalized chain

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Stretch also in contructor plus  tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix approval-voting tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* grunting: dispute coordinator tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-10-04 13:36:42 +03:00
Andrei SanduandAndronik d7eaaeee65 Remove orchestra and metered channel (#6086)
* Use orchestra and metered channel from repo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update node/subsystem-util/Cargo.toml

Co-authored-by: Andronik <write@reusable.software>

* Update node/subsystem-types/Cargo.toml

Co-authored-by: Andronik <write@reusable.software>

* Update node/metrics/Cargo.toml

Co-authored-by: Andronik <write@reusable.software>

* Update node/overseer/Cargo.toml

Co-authored-by: Andronik <write@reusable.software>

* Update node/overseer/Cargo.toml

Co-authored-by: Andronik <write@reusable.software>

* update cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Andronik <write@reusable.software>
2022-10-03 12:26:23 +02:00
Andrei Sandu 372e6a95f3 add para_id to fetch_pov logging (#6084)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-09-30 11:56:45 +03:00
Andrei Sandu 6e424467a2 Proper naming wrt expectations (#12311)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-09-20 16:17:34 +02:00
Andrei SanduandAndronik 4db5bbc49a Zombienet: test disputes with malus garbage candidates (#5857)
* 1/3 validators suggest garbage candidates,
but paras should make progress

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* scale down test a bit

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Use docker paths and more paras

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* dispute finality lag < 2

* scale down

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* scale down more

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* attempt fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* one malus node

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update zombienet_tests/functional/0003-parachains-garbage-candidate.toml

Co-authored-by: Andronik <write@reusable.software>

* Update zombienet_tests/functional/0003-parachains-garbage-candidate.toml

Co-authored-by: Andronik <write@reusable.software>

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Andronik <write@reusable.software>
2022-08-11 10:31:01 +00:00
Andrei SanduandAndronik 5221fd667e Parachains db column "migration" (#5797)
* Column migration for parityDB

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove columns

* warn

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump paritydb

* use clear_column

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* logs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* paritydb 0.3.16

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update node/service/Cargo.toml

Co-authored-by: Andronik <write@reusable.software>

* ParityDB versioning

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review + proper version constants

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Co-authored-by: Andronik <write@reusable.software>
2022-07-27 09:34:27 +00:00
d54e20708d Zombienet: paritydb test (#5310)
* Add test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* gitlab integration

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* tune test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Try waiting for nodes to be up before checks

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Use js check for paritydb folder

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* artifacts true

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* change error message

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* debug

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* try again

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* use log line contains for test

* bump zombienet verision for test

* Is approval checking lag 0 on rocksdb ?

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Is approval checking lag 1 with paritydb ?

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update zombienet test for parityDb

* Update zombienet_tests/misc/0001-check_paritydb.sh

Co-authored-by: Chevdor <chevdor@users.noreply.github.com>

Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
2022-07-18 10:55:38 +00:00
Andrei Sandu e7cded019b Update metric name and doc (#5716)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-06-28 09:10:12 +00:00
Andrei Sandu f05b9b31d6 Update metric before bailing out (#5706)
* Update authority metrics before bailing out

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove redundant call

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* doc

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-06-22 09:10:24 +00:00
4060017059 Availability: provisioner bitfield metrics (#5484)
* Add histogram for inherent data bitfields

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* -500ms bitfield sign job delay, +500ms bitfield gossip

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Revert "-500ms bitfield sign job delay, +500ms bitfield gossip"

This reverts commit 3d3f2c0fae651653ffdca50c1ad38c5757166edc.

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update node/core/provisioner/src/metrics.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

Co-authored-by: Andronik <write@reusable.software>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
2022-05-19 18:53:43 +02:00
Andrei Sandu 0ce850f959 runtime-api: remove internal queue to make ToFs relevant again (#5545)
* Make back pressure great again

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Test commit - parallel  vs caching

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Increase concurrency

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixups

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update todo issue

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* another doc change

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* More comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-05-18 12:56:03 +00:00
Andrei Sandu 7f3001a491 Add counter for bounded channel sends that block. (#5490)
* Add counter for bounded channel sends that block.

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix typos

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix bounded sent metric

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* refactor a bit and test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Return disconnect errors early.

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* future proof error handling

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-05-17 08:01:48 +00:00
Andrei Sandu e29d8e91da Improve dispute-coordinator message burstiness handling (#5471)
* Increase message channel size to 2048

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Use unbounded channel for reading data

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-05-09 10:00:05 +00:00
cddd5749d3 Malus: improvements in dispute ancestor and suggest garbage candidate implementation (#5011)
* Implement fake validation results

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* refactor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spell check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typos

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* move stuff around

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* chores

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Impl valid - still wip

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Pull Ladi's implementation:
https://github.com/paritytech/polkadot/pull/4711

Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
Co-authored-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Logs and comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* WIP: suggest garbage candidate + implement validation result caching

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Do commitment hash checks in candidate validation

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Minor refactor in approval, backing, dispute-coord

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Working version of suggest garbage candidate

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Dedup

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cleanup #1

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove debug leftovers

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Accidentally commited some local test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* some more fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Refactor and fix it

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* tests review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* refactor disputer

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix zombienet disputes test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix ui tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2022-04-13 13:45:39 +00:00
Andrei Sandu 784ca5bf0f bump metric timeout (#5231)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-04-01 16:28:43 +00:00
Andrei Sandu 3e357ec0a8 Zombienet: add cpu/mem resource requests (#5080)
* Ensure request/limits for CPU and memory for nodes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* docker.io/paritytech/zombienet:v1.2.19

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Deny failure

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add retry back

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-03-14 10:47:37 +00:00
0fa91e7043 Zombienet: add new test collator and integration test (#4797)
* WIP: Wasm compilation perf

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Undying collator WIP

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update test with undying

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Correctly compute post hash

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update helper

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* squash bugs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add --pov-size cli parameter

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* refactor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix strings

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add pov-size param to export genesis state

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix graveyard size

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* docs + fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix PVF bug and switch to u8 graves

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Build/publish undying collator as colander img

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add undying-collator to colander image

* Fix transaction overflow

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* warn fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add parachain id for export genesis cli

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix the build

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* scale test up

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* default parachain id

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add PVF complexity param

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Hash on each iteration

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update pvf metric histogram buckets

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Ladi attempt to fix tests

* Fix test

* Fix typos

* Fix pvf typo

* CI integration

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock missing

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix clap merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add zombienet image back

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Use collator image from env

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump zombienet version

* update test to check pvf prep/exec

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* delete file

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* use default bootnode in upgrade test

* FIx tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add some stress -  pvf exec times up to 2s

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix name

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Dial down pvf execution time < 2s

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 1100

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* try again

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add connectivity check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixes and refactor folder struct

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* change toml name to match

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump zombienet

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* impl Feedback for Review 😎

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spell check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Zombienet: add disputes test (#4859)

* Zombienet disputes test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* CI: add zombienet-parachain-disputes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Bump zombienet

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* reduce duration

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update zombienet_tests/functional/0002-parachains-disputes.feature

fix test

* Update zombienet_tests/functional/0002-parachains-disputes.feature

fix fix

* more logs and set collator image

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix cargo lock damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* merge fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump zombienet image

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* debugging CI run - scale down test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix cargo merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* default command fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Revert "debugging CI run - scale down test"

This reverts commit eec2ba7e65ede3f929e2f0a8fe44930df2119450.

* more review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com>
2022-02-28 10:27:01 +00:00
sandreim a0ebff4909 Add more specific errors (#4905)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-02-11 18:32:55 +01:00
sandreim 7cae61726e enable runtime debug and parachain tracing targets (#4901)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-02-11 14:44:06 +01:00
sandreim 50828ad145 Update pvf metric histogram buckets (#4818)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-01-31 17:28:58 +00:00
sandreim 806cb541b7 Refactor and fix usage of get_session_index() and get_session_info_by_index() (#4735)
* Rename/refactor around get_session_index

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* choose proper head for fetching session

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* revert rename

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* renaming and more comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Run Fetch task in correct session

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Log warning when ancestors unavailable

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-01-26 15:17:46 +00:00
sandreim 2661930b77 relay chain selection and dispute-coordinator fixes and improvements (#4752)
* Dont error in finality_target_with_longest_chain

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add error flag

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add error flag in dispute-coordinator

Make sure to send errors to subsystems requesting data depending on missing session info

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Scrape ancestors

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix naming

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 💬 fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* consume

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Bump scraped blocks LRU capacity

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 🧯 🔥

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove prints

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Increase scraped blocks cache size

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* another fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix target_ancestor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Scrape up to max finalized block

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* undo comment change

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Limit ancestry lookup to last finalized block or
max finality lag

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* debug damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-01-26 14:06:27 +00:00
sandreim 6e268a48f6 Add node authority status metric (#4699)
* Check authority status on active leaves update

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo changes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add metric for authority status

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Revert "Fix tests"

This reverts commit 5bd56bb367ec0b01ecca04498b3974ab67bf3189.

* Revert "cargo changes"

This reverts commit ffea18fe3ca12b27e6471ad9f44592799ec90956.

* Revert "Check authority status on active leaves update"

This reverts commit 55a30ac81bf32a72e0b79ca2e7bb612344a5c43d.

* Test fixups

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* undo damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* dont update status on runtime errors

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix inconsistency

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Dont derive primitive Default

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add dummy_session_info helper

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* unset parachain validator status if no longer authority

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* damn

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 🤦

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2022-01-13 17:41:10 +00:00
sandreimandparity-processbot 599da09a40 Companion for #4639 (Remove sleep from tests ) (#890)
* fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update lockfile for polkadot

Co-authored-by: parity-processbot <>
2022-01-10 11:33:10 +00:00
sandreimparity-processbotSergei Shulepov
0da05cac26 Remove sleep from tests (#4639)
* Remove sleep and use polkadot test service

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* updates

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix other tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Run metrics tests separately

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* copy some substrate utilities

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update runtime metric test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove sleep from cli tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Polkadot companion for Substrate#10463 (#4519)

* Grandpa and Beefy protocol names include chain id

Signed-off-by: acatangiu <adrian@parity.io>

* chain_spec: include fork id

* use simplified protocol name

* fix after merge

* avoid using hash default, even for protocol names

* update lockfile for substrate

Co-authored-by: parity-processbot <>

* configuration: Update upgrade validation delay doc (#4662)

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* use testnet profile

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Don't run with runtime-benchmark feature

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* conditional compile up one level

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Co-authored-by: Sergei Shulepov <sergei@parity.io>
2022-01-10 11:00:46 +00:00
sandreimandBastian Köcher b93c6e68ab Parachain runtime metrics followup (#4602)
* initial changes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove file

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove pallet

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix copyright year

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove metric registration op

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Register runtime metrics in client

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix build without `runtime-metrics`

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* reduce visibility

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove metric prefix logic, use hardcoded

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Split CounterVec api so it doesn't need mutability

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Const beautify

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix docs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Merge web ui feedback.

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2022-01-05 15:33:02 +02:00
sandreimandJavier Viola 90d986121d Add Parachain Grafana dashboards (#4578)
* Read me.

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add dashboards and readme

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update root readme

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add dashboard links in readme

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* updates

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Updates

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bump zombienet version

Co-authored-by: Javier Viola <javier@parity.io>
2021-12-28 20:18:13 +00:00
sandreim 979fa1f5ab Bump sleep from 10 to 30s (#4613)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-12-27 18:44:42 +01:00
sandreimandparity-processbot 34f605004d Companion for #10440 (Add logger configuration hook) (#856)
* update `DefaultConfigurationValues` impl

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update lockfile for substrate

Co-authored-by: parity-processbot <>
2021-12-16 11:57:17 +00:00
sandreimandparity-processbot e75ad88ae8 Add support for wasm runtime metrics try #2 (#4483)
* Add runtime metrics provider

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Runner changes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Some sample metrics in paras_inherent

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update cargo toml

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* bug

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fmt after merge

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Refactor metric prefix override

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove bug comment

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add runtime metric primitives

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Impl trace event parsing

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update metrics

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix target check

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Runtime metrics primitives

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Runtime metrics crate

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Node side runtime metric changes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* use runtime CounterVec instead of macro

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt nice

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove dead code

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* base58 decoding

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* base58 encoding

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix warn

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Finish label support

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* please compile

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add feature gate

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Comment cargo toml

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix cargo toml description

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update doc.

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* switch to `runtime-metrics` feature

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo toml

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* better ux

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* from_utf8_unchecked is safe

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add Counter and refactor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* add integration test

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* dev deps

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* gitlab script update

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix merge damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Run tests twice

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* small fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo lock

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* spellcheck happy ?

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Guard tracing init

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* missing copyright

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update lockfile for substrate

Co-authored-by: parity-processbot <>
2021-12-16 11:56:25 +00:00
sandreim 9b73a8a6fc Add logger configuration hook (#10440)
* Initial poc

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Make config available to logger hook

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add metric prefix option in sc_cli::RunCmd

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Remove print

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix docs

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-12-16 11:18:24 +00:00
sandreim e4e22f405d Fix Provisioner dispute metrics naming (#4374)
* update dispute metric names

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-11-26 11:31:22 +01:00
sandreim 0912d6a95e Fix spellcheck (#4363)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-11-24 16:15:04 +01:00
sandreim 66aed0d0a4 fix provisioner metric docs (#4359)
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-11-24 15:12:08 +01:00
sandreim e08b0fb506 Add Provisioner dispute metrics (#4352)
* Metrics for InherentDataProvider

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Integrate metrics

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more changes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* avoid naming confusion

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Move to Provisioner.

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add metric documentation

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-11-24 10:35:46 +00:00
sandreimandAndronik Ordian 5f5190625d Companion to substrate #10196 - Add group name in task metrics (#732)
* Use blank task group name

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update Cargo.lock and switch from 0.9.12 to master

* format with LATEST nightly

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-11-11 19:41:36 +00:00
sandreimandAndronik Ordian b0f89bbfbc Per subsystem CPU usage tracking (#4239)
* SubsystemContext: add subsystem name str

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Overseer builder proc macro changes

* initilize SubsystemContext name field.
* Add subsystem name in TaskKind::launch_task()

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Update ToOverseer enum

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Assign subsystem names to orphan tasks

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* cargo fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* SubsystemContext: add subsystem name str

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Overseer builder proc macro changes

* initilize SubsystemContext name field.
* Add subsystem name in TaskKind::launch_task()

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Update ToOverseer enum

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Assign subsystem names to orphan tasks

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* cargo fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Rebase changes for new spawn() group param

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add subsystem constat in JobTrait

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add subsystem string

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix spawn() calls

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix more tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Address PR review feedback #1

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Address PR review round 2

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixes
- remove JobTrait::Subsystem
- fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* update Cargo.lock

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-11-11 18:53:37 +00:00
sandreim fdb3c64243 Add group name in task metrics (#10196)
* SpawnNamed: add new trait methods

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Implement new methods

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* cargo fmt

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* SpawnNamed: add new trait methods

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* Implement new methods

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* cargo fmt

Signed-off-by: Andrei Sandu <sandu.andrei@gmail.com>

* New approach - spaw() group param

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update traits: SpawnNamed and SpawnNamed

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update TaskManager tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Update test TaskExecutor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* grunt work: fix spawn() calls

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove old code

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* cargo fmt - the right version

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Implement review feedback

- use Option group name in SpawnNamed methods
- switch to kebab case
- implement default group name
- add group name to some tasks

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
2021-11-11 17:15:09 +00:00