Commit Graph

134 Commits

Author SHA1 Message Date
dependabot[bot] ee7b4f06d7 Bump hex-literal from 0.3.1 to 0.3.3 (#9974)
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.3.1 to 0.3.3.
- [Release notes](https://github.com/RustCrypto/utils/releases)
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.3.1...hex-literal-v0.3.3)

---
updated-dependencies:
- dependency-name: hex-literal
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-09 14:23:57 +00:00
dependabot[bot] 36d25d4994 Bump assert_cmd from 2.0.1 to 2.0.2 (#9972)
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.1...v2.0.2)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-08 11:18:05 +00:00
dependabot[bot] 5edb6fc224 Bump jsonrpsee-ws-client from 0.3.0 to 0.3.1 (#9963)
Bumps [jsonrpsee-ws-client](https://github.com/paritytech/jsonrpsee) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/paritytech/jsonrpsee/releases)
- [Changelog](https://github.com/paritytech/jsonrpsee/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/jsonrpsee/compare/v0.3.0...v0.3.1)

---
updated-dependencies:
- dependency-name: jsonrpsee-ws-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-08 09:01:04 +00:00
Bastian Köcher 451413f47e Transaction pool: Adds benchmark and improves performance (#9958)
* Yep

* Make it compile

* Make the benchmark work

* Some stuff

* Optimize transaction pool `BestIterator`

* Some docs

* Fix more warnings

* Fix compilation

* FMT
2021-10-08 10:02:25 +02:00
Squirrel 07761c22f7 remove dep on unmaintained difference crate (#9946) 2021-10-06 18:56:37 +02:00
Squirrel e64693933f Less sleeps (#9848)
* Less sleeps

* No need to use tokio-test crate

* Less sleep

* Avoid leaving zombie substrates around

(when panicing in tests)

* Remove unused imports

* Incorporating feedback

* rename method

* Use rpc_api

* Update bin/node/cli/tests/temp_base_path_works.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update bin/node/cli/tests/temp_base_path_works.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-09-29 10:55:31 +00:00
dependabot[bot] 559faf1c61 Bump async-std from 1.9.0 to 1.10.0 (#9759)
Bumps [async-std](https://github.com/async-rs/async-std) from 1.9.0 to 1.10.0.
- [Release notes](https://github.com/async-rs/async-std/releases)
- [Changelog](https://github.com/async-rs/async-std/blob/master/CHANGELOG.md)
- [Commits](https://github.com/async-rs/async-std/compare/v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: async-std
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-29 10:53:15 +02:00
Bastian Köcher c09d52ead7 Use tokio runtime handle instead of TaskExecutor abstraction (#9737)
* Use tokio runtime handle instead of TaskExecutor abstraction

Before this pr we had the `TaskExecutor` abstraction which theoretically
allowed that any futures executor could have been used. However, this
was never tested and is currently not really required. Anyone running a
node currently only used tokio and nothing else (because this was hard
coded in CLI). So, this pr removes the `TaskExecutor` abstraction and
relies directly on the tokio runtime handle.

Besides this changes, this pr also makes sure that the http and ws rpc
server use the same tokio runtime. This fixes a panic that occurred when
you drop the rpc servers inside an async function (tokio doesn't like
that a tokio runtime is dropped in the async context of another tokio
runtime).

As we don't use any custom runtime in the http rpc server anymore, this
pr also removes the `rpc-http-threads` cli argument. If external parties
complain that there aren't enough threads for the rpc server, we could
bring support for increasing the thread count of the tokio runtime.

* FMT

* Fix try runtime

* Fix integration tests and some other optimizations

* Remove warnings
2021-09-12 12:29:11 +00:00
Falco Hirschenberger 16144e7404 Store the database in a role specific subdirectory (#9645)
* Store the database in a role specific subdirectory

This is a cleaned up version of #8658 fixing #6880

polkadot companion: paritytech/polkadot#2923

* Disable prometheus in tests

* Also change p2p port

* Fix migration logic

* Use different identification file for rocks and parity db

Add tests for paritydb migration
2021-09-07 13:31:25 +00:00
Bastian Köcher f92e5c5e65 Upgrade tokio to 1.10 (#9575)
* Upgrade tokio to 1.10

* Fix test runner

* Try fix it

* Update Cargo.lock

* Review feedback

* ahhhh

* FML

* FMT

* Fix tests
2021-08-24 14:31:19 +00:00
Ashley bad4544507 Make choosing an executor (native/wasm) an explicit part of service construction (#9525)
* Split native executor stuff from wasm executor stuff

* Remove `native_runtime_version` in places

* Fix warning

* Fix test warning

* Remove redundant NativeRuntimeInfo trait

* Add a warning for use_native

* Run cargo fmt

* Revert "Add a warning for use_native"

This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.

* Make choosing an executor (native/wasm) an explicit part of service construction

* Add Cargo.lock

* Rename Executor to ExecutorDispatch

* Update bin/node/executor/src/lib.rs

Co-authored-by: Squirrel <gilescope@gmail.com>

* Fix tests

* Fix minor node-executor error

* Fix node cli command thing

Co-authored-by: Squirrel <gilescope@gmail.com>
2021-08-18 14:26:41 +02:00
David 2de7e51c2a Remove substrate-in-the-browser (#9541)
* Comment out browser stuff

* Remove browser stuff

* Remove more wasm transport code

* Remove ExtTransport and rework how telemetry initialises.

* Change (most) wasm-timer using code to use std::time

* Rename CI-job

* Aura does not compile for wasm

* Remove testing in the browser on CI

* Update README

* Leave `StreamSink` be

* fmt
2021-08-17 18:06:23 +00:00
Trevor Arjeski d9f02296ab RUSTSEC-2021-0076 bump libsecp256k1 (#9391)
* RUSTSEC-2021-0076 bump libsecp256k1

libsecp256k1 allows overflowing signatures
https://rustsec.org/advisories/RUSTSEC-2021-0076

Changes were made to conform to libsecp256k1 version differences.

Closes #9356

* parse_standard_slice() -> parse_overflowing_slice()

* Added v2 host function for ecdsa_verify

* Add feature tag over helpers

* Added ecdsa_verify v2 to test runner

* PR feedback

- Spaces -> tabs
- renamed two helper functions

* Fixed imports after rebasing

* Bump rest of libsecp256k1 (and libp2p)

libp2p also uses libsecp256k1 so it is required to be bumped too, along
with all the version difference changes.

* Add version2 for ecdsa pubkey recovery

* libp2p rebase master fixes

* Fix test panic when non Behaviour event is returned

* Update bin/node/browser-testing/Cargo.toml

* Update primitives/core/src/ecdsa.rs

* Update primitives/core/src/ecdsa.rs

* Update Cargo.lock

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-08-16 14:17:10 +00:00
Squirrel 7e9b8d278e depend-o-pocalipse (#9450)
Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.  
Updated code related to deprecated warning on tracing-subscriber `scope()` 
( See https://github.com/tokio-rs/tracing/issues/1429 )
2021-08-13 15:18:37 +01:00
Bastian Köcher c44aba89e6 Upgrade jsonrpc to 0.18.0 (#9547)
* Upgrade jsonrpc to 0.18.0

I think this says all :P

* 🤦

* Fmt etc

* Fix tests

* Fix tests again...

* Better impl

* Revert "Tell dependabot to ignore jsonrpc-* updates (#9518)"

This reverts commit 6e0cd5587d.
2021-08-13 06:46:07 +00:00
Arkadiy Paronyan 52b0a4845f Storage chain fixes + guide (#9504)
* Transaction storage guide and fixes

* Apply suggestions from code review

Co-authored-by: Bruno Škvorc <bruno@skvorc.me>

* Update frame/transaction-storage/README.md

Co-authored-by: Bruno Škvorc <bruno@skvorc.me>

* Extended example

Co-authored-by: Bruno Škvorc <bruno@skvorc.me>
2021-08-05 12:07:23 +00:00
Bastian Köcher 6e4d30e8ad Decouples light-sync state from chain spec (#9491)
* Decouples light-sync state from chain spec

This decouples the light-sync state from chain spec. First, the
light-sync state currently only works with BABE+Grandpa, so not
all *Substrate* based chains can use this feature. The next problem was
also that this pulled the `sc-consensus-babe` and `sc-finality-grandpa`
crate into `sc-chain-spec`.

If a chain now wants to support the light-sync state, it needs to add
the `LightSyncStateExtension` to the chain spec as an extension. This is
documented in the crate level docs of `sc-sync-state-rpc`. If this
extension is not available, `SyncStateRpc` fails at initialization.

* Fix compilation for browser

* Fmt
2021-08-04 13:38:14 +00:00
Arkadiy Paronyan f56232602f Warp sync part I (#9227)
* Started warp sync

* BABE & GRANDPA recovery

* Warp sync protocol

* Sync warp proofs first

* Added basic documentation

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Style changes

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* fmt

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fixed chage trie pruning wrt missing blocks

* Restore parent finalization

* fmt

* fmt

* Revert pwasm-utils bump

* Change error type & check API version

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Build fix

* Fixed target block check

* Formatting

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-08-02 08:27:18 +00:00
dependabot[bot] 31011e7a03 Bump serde from 1.0.124 to 1.0.126 (#9099)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.124 to 1.0.126.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.124...v1.0.126)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-13 08:09:05 +00:00
Benjamin Kampmann c78c2892a4 Attempting to fix publishing (#9140)
* mark template and utils as non-publish

* switch to development version for testing

* activate unleash check

* maybe if I disable all rules...

* Fix isolated compilation of `max-encoded-len-derive` with `syn`

error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
  --> src/lib.rs:88:29
   |
88 |             .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
   |                            --------- ^^ ----------------------------------- _
   |                            |
   |                            syn::Path

error: aborting due to previous error

For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`

* WIP: bump changes crates since v3 tag to next breaking

cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0

FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`

* Update lockfile

* WIP: Bump sp-transaction-pool as well

* WIP: Bump sp-offchain as well

* WIP: Bump frame-system-rpc-runtime-api as well

* WIP: Bump sp-authority-discovery as well

* Manually deactivate dev-deps before `cargo unleash check`

Otherwise we run into `Cycle detected` error.

* Bump sp-consensus-slots

* Add missing Cargo.lock change

* Bump sp-consensus-vrf as well

* Bump sp-keyring as well

* Bump sp-consensus-pow as well

* Try to speed up the `unleash-check` job

Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help

* fixup: Ensure the temp target dir exists for unleash check

* Bump pallet-transaction-payment-rpc-runtime-api as well

Needed for Polkadot

* Bump pallet-transaction-payment-rpc as well

Needed for Polkadot

* Try updating crates after patching in the Polkadot CI job

* Use another approach to update patched Substrate crates

* Try to update all sp-core versions in Polkadot CI job

* Simplify sp-core version checking

* Apply another shellcheck lint

* Just do the simplest thing I guess

* Welp don't do --offline then

* Clean up `unleash-check` job triggers

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Fix a note in unleash-check cache step

* Add a note about temporary optimization in cargo-unleash

* Pin a newer version of cargo-unleash

Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
2021-07-11 12:17:53 +00:00
Seun Lanlege 2ae9d36758 sc-transcation-pool refactor (#9228)
* Use TransactionPool trait

* sc-transaction-pool-primitives

* sc-transaction-pool-api

* TP

* bye sc_transaction_graph

* fix line widths

* fix import errors

* fix import errors

* fix import errors 🤦🏾‍♂️

* fix import errors 🤦🏾‍♂️🤦🏾‍♂️🤦🏾‍♂️

* remove sp-keyring
2021-07-08 13:33:34 +00:00
Jon Häggblad acef22bbd2 Migrate pallet-grandpa to attribute macros (#8724)
* frame/grandpa: migrate Config

* frame/grandpa: migrate decl_module

* frame/grandpa: migrate decl_event

* frame/grandpa: migrate decl_error

* frame/grandpa: migrate decl_storage

* frame/grandpa: make report_equivocation_unsigned pub(super)

* frame/grandpa: remove unused imports

* frame/grandpa: replace deprecated Module with Pallet

* frame/grandpa: add RawEvent for compatibility

* frame/grandpa: create migration to new storage prefix

* frame/grandpa: bump version to 4.0.0

* frame/grandpa: address review comments

* Try using version 3.1 instead

* frame/grandpa: tweak log text to say cancelled
2021-05-26 16:26:15 +00:00
Bastian Köcher 2675741a09 Rework inherent data client side (#8526)
* Lol

* Yeah

* Moare

* adaasda

* Convert AURA to new pallet macro

* AURA: Switch to `CurrentSlot` instead of `LastTimestamp`

This switches AURA to use `CurrentSlot` instead of `LastTimestamp`.

* Add missing file

* Update frame/aura/src/migrations.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Remove the runtime side provide inherent code

* Use correct weight

* Add TODO

* Remove the Inherent from AURA

* 🤦

* Remove unused stuff

* Update primitives authorship

* Fix babe inherent data provider

* Fix consensus-uncles

* Fix BABE

* Do some further changes to authorship primitives... :D

* More work

* Make it compile the happy path

* Make it async!

* Take hash

* More stuff

* Hacks

* Revert "Hacks"

This reverts commit cfffad88668cfdebf632a59c4fbfada001ef8251.

* Fix

* Make `execute_block` return the final block header

* Move Aura digest stuff

* Make it possible to disable equivocation checking

* Fix fix fix

* Some refactorings

* Comment

* Fixes fixes fixes

* More cleanups

* Some love

* Better love

* Make slot duration being exposed as `Duration` to the outside

* Some slot info love

* Add `build_aura_worker` utility function

* Copy copy copy

* Some stuff

* Start fixing pow

* Fix pow

* Remove some bounds

* More work

* Make grandpa work

* Make slots use `async_trait`

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make grandpa work

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make it async

* Fix fix

* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* Make grandpa tests compile

* More Babe tests work

* Fix network test

* Start fixing service test

* Finish service-test

* Fix sc-consensus-aura

* Fix fix fix

* More fixes

* Make everything compile *yeah*

* Make manual-seal compile

* More fixes

* Start fixing Aura

* Fix Aura tests

* Fix Babe tests

* Make everything compile

* Move code around and switch to async_trait

* Fix Babe

* Docs docs docs

* Move to FRAME

* Fix fix fix

* Make everything compile

* Last cleanups

* Fix integration test

* Change slot usage of the timestamp

* We really need to switch to `impl-trait-for-tuples`

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Some extra logging

* Remove dbg!

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-05-03 14:39:25 +00:00
Pierre Krieger d64f79924a Update to libp2p 0.37 (#8625)
* Update to libp2p 0.37

* Line widths

* Fix tests
2021-04-18 08:04:45 +00:00
Bastian Köcher 2998b42311 Get rid of test-helpers feature in sc-consensus-babe (#8486)
This is not really required and having a special `test-helpers` feature
is a bad idea anyway.
2021-03-30 08:10:54 +02:00
Roman Borschel e1570b9ab4 Update to libp2p-0.36 (#8420)
* Update to libp2p-0.36

* Some more Cargo.lock updates.
2021-03-24 18:29:35 +00:00
Cecile Tonglet 8031b6eacb Rework telemetry to replace the use of tracing with an object we pass around (#8143)
polkadot companion: paritytech/polkadot#2535
2021-03-11 11:05:45 +01:00
Alexander Theißen 4bf0387237 contracts: Release as v3.0.0 and add reserved field to ContractInfoOf (#8175)
* contracts: Update README

* contracts: Add CHANGELOG.md

* contracts: Bump version to v3.0.0 and allow publish

* Typos

Co-authored-by: Andrew Jones <ascjones@gmail.com>

* Improve wording in the changelog

* contracts: Add reserved field to ContractInfoOf for future proofing

* also bump frame-benchmarking

* update lockfile

Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Benjamin Kampmann <ben@parity.io>
2021-02-25 17:04:48 +01:00
Kian Paimani 16a27c28a9 Migration testing runtime API/Bot (#8038)
* A clean new attempt

* Checkpoint to move remote.

* A lot of dependency wiring to make it feature gated.

* bad macro, bad macro.

* Undo the DB mess.

* Update frame/support/src/traits.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* unbreak the build

* Update frame/try-runtime/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/try-runtime/cli/Cargo.toml

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update frame/try-runtime/Cargo.toml

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Address most review grumbles.

* Fix build

* Add some comments

* Remove allowing one pallet at a time.

* More grumbles.

* relocate remote-ext

* Fix build

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2021-02-19 14:52:09 +00:00
Cecile Tonglet 27274c42cf Fix telemetry span not entering properly attempt 3 (#8043)
* Fix tracing tests (#8022)

* Fix tracing tests

The tests were not working properly.

1. Some test was setting a global subscriber, this could lead to racy
conditions with other tests.

2. A logging test called `process::exit` which is completly wrong.

* Update client/tracing/src/lib.rs

Co-authored-by: David <dvdplm@gmail.com>

* Review comments

Co-authored-by: David <dvdplm@gmail.com>

* Fix tracing spans are not being forwarded to spawned task (#8009)

* Fix tracing spans are not being forwarded to spawned task

There is a bug that tracing spans are not forwarded to spawned task. The
problem was that only the telemetry span was forwarded. The solution to
this is to use the tracing provided `in_current_span` to capture the
current active span and pass the telemetry span explictely. We will now
always enter the span when the future is polled. This is essentially the
same strategy as tracing is doing with its `Instrumented`, but now
extended for our use case with having multiple spans active.

* More tests

* Proper test for telemetry and prefix span

* WIP

* Fix test (need to create & enter the span at the same time)

* WIP

* Remove telemtry_span from sc_service config

* CLEANUP

* Update comment

* Incorrect indent

* More meaningful name

* Dedent

* Naming XD

* Attempt to make a more complete test

* lint

* Missing licenses

* Remove user data

* CLEANUP

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* CLEANUP

* Apply suggestion

* Update bin/node/cli/tests/telemetry.rs

Co-authored-by: David <dvdplm@gmail.com>

* Wrapping lines

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
2021-02-17 07:44:25 +00:00
Ashley 019f04e99f Update sc-finality-grandp-warp-sync to 0.9.0 and remove 'publish = false' (#8109) 2021-02-12 18:08:55 +01:00
Benjamin Kampmann 6a964c5df0 Releasing 3.0 (#8098)
* bumping version for next release
* add changelog
* add guide
2021-02-10 19:23:18 +01:00
Benjamin Kampmann 8e36d87ca8 Update dependencies ahead of next release (#8015)
Updates dependencies:
parity-db 0.2.2
paste
prometheus 0.11
cfg-if 1.0
strum 0.20
env_logger 0.8
pin-project
prost
nix
platforms
quickcheck 1.0
2021-02-04 19:17:42 +01:00
Guillaume Thiolliere f48296e9ad Update parity-scale-codec to 2.0 (#7994)
* update cargo.toml

* use 2.0 in mmmr
2021-01-29 12:22:45 +00:00
Ashley 87cc216774 Grandpa warp sync request-response protocol (#7711)
* Made a start

* So the proof between authority set is phragmen one, this is crazy big,
or is there some signing of the result : that is the storage key, damn?

* ok getting from header digest seems doable.

* for testing

* get set id from storage directly (should use runtime to handler change).

* move test to init

* correct auth key

* fix iteration

* Correct proof content

* actually update block number.

* actually check last justif against its header

* justification relation to new authorities through header hash check is
needed here. This assumes the hash from header is calculated.

* Few changes

* Connected up cheme's branch

* Clean up

* Move things around a bit so that adding the grandpa warp sync request response protocol happens in the node code

* Nits

* Changes to comments

* Cheme changes

* Remove todos and test compile.

* Rename _authority_ related proof function to _warp_sync_ .

* Update client/grandpa-warp-sync/src/lib.rs

quick fix

* Put the warp sync request response protocol behind a feature flag because we dont' need it on a light client.

* Update client/grandpa-warp-sync/src/lib.rs

Quick fix

* Update Cargo.lock

* Adding test, comment on limitation related to 'delay', this could
be implemented but with a cost.

* Set between a delay override last fragment.

* Check for pending authority set change at start.

* adjust index

* custom cache is not a good idea.

* Use a simple cache instead.

* restore broken indentation

* Address crate rename

* Merge conflict badly resolved, sorry

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2021-01-21 17:14:07 +00:00
Cecile Tonglet 970cc25cef Telemetry per node (#7463) 2021-01-20 12:28:56 +01:00
Bernhard Schuster f7fccb3122 make helper error types generics (#7878)
* make helper error types generics

* avoid From<io::Error> dep in runner helper logic

* slip of the pen, bump futures to 0.3.9

* more generics

* generic var spaces

Co-authored-by: Andronik Ordian <write@reusable.software>

* network-gossip: add metric for number of local messages (#7871)

* network-gossip: add metric for number of local messages

* grandpa: fix GossipEngine missing metrics registry parameter

* network-gossip: increase known messages cache size

* network-gossip: fix tests

* grandpa: remove unnecessary clone

Co-authored-by: Max Inden <mail@max-inden.de>

* network-gossip: count registered and expired messages separately

* network-gossip: add comment on known messages cache size

* network-gossip: extend comment with cache size in memory

Co-authored-by: Max Inden <mail@max-inden.de>

* Clean-up pass in network/src/protocol.rs (#7889)

* Remove statistics system

* Remove ContextData struct

* Remove next_request_id

* Some TryFrom nit-picking

* Use constants for peer sets

* contracts: Don't read the previous value when overwriting a storage item (#7879)

* Add `len` function that can return the length of a storage item efficiently

* Make use of the new len function in contracts

* Fix benchmarks

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Remove unused imports

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* Fix clear prefix check to avoid erasing child trie roots. (#7848)

* Fix clear prefix check to avoid erasing child trie roots.

* Renaming and extend existing test with check.

* last nitpicks.

* use follow paths to std standarad components

* line width

Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
2021-01-13 19:44:46 +01:00
Guillaume Thiolliere 014e6f03c6 bumpd minor version (#7873) 2021-01-12 09:25:47 +01:00
Pierre Krieger a4032b1dfb Update to futures 0.3.9 (#7854) 2021-01-08 15:47:33 +00:00
Andronik Ordian 2b48e09b24 upgrade a few dependencies (#7831)
* upgrade a few dependencies

* make it compile at the expense of duplicate deps

* fix web-wasm and a warning

* introduce activate-wasm-bindgen-features crate

* Revert "introduce activate-wasm-bindgen-features crate"

This reverts commit 5a6e41e683f8a4844c0a735dcd08caabb2313f11.

* add getrandom feature to sc-consensus-aura
2021-01-06 16:12:17 +00:00
dependabot[bot] 26830a20df Bump tracing from 0.1.21 to 0.1.22 (#7589)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.21 to 0.1.22.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.21...tracing-0.1.22)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-11-24 09:44:01 +00:00
dependabot[bot] ee78dd3d0e Bump wasm-bindgen-futures from 0.4.17 to 0.4.18 (#7572)
Bumps [wasm-bindgen-futures](https://github.com/rustwasm/wasm-bindgen) from 0.4.17 to 0.4.18.
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-11-22 21:04:39 +01:00
Jon Häggblad 7cf78c166e slots: incrementally backoff claiming slots if finality lags behind (#7186)
* babe: backoff authoring blocks when finality lags

* babe: move backoff authoring params to default constructor

* babe: deduplicate the test a bit

* babe: set backoff constants in service

* babe: use better names for backoff authoring block parameters

* babe: remove last unwrap

* babe: slight style tweak

* babe: fix comment

* slots: move backoff block authorship logic to SimpleSlotWorker

* aura: append SlotInfo in on_slot

* slots: use the correct types for parameters

* slots: fix review comments

* aura: add missing backoff authoring blocks parameters

* slots: add comments for default values

* slots: add additional checks in test

* slots: update implementation for new master

* slots: revert the change to SlotInfo

* Fix review comments

* slots: rework unit tests for backing off claiming slots

* slots: add test for asymptotic behaviour for slot claims

* slots: address review comments

* slots: add test for max_interval

* slots: add assertion for intervals between between claimed slots

* slots: remove rustfmt directive

* slots: another attempt at explaining authoring_rate

* slots: up unfinalized_slack to 50 by default

* slots: add tests for time to reach max_interval

* slots: fix typo in comments

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* slots: additional tweaks to comments and info calls

* slots: rename to BackoffAuthoringOnFinalizedHeadLagging

* slots: make the backing off strategy generic

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* slots: implement backoff trait for () for simplicity

* slots: move logging inside backing off function to make it more specific

* aura: add missing function parameter

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-11-11 13:29:19 +00:00
André Silva f3ea1624f6 frame: remove finality-tracker (#7228)
* frame: remove finality-tracker

* node: remove unused parameter types

* node: bump spec_version
2020-10-15 20:58:51 +00:00
Rakan Alhneiti 3aa4bfacfc Async keystore + Authority-Discovery async/await (#7000)
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 20:56:35 +00:00
Benjamin Kampmann ecdc94420e Releasing 2.0 – two dot 😮 (#7182)
Tagging the release of substrate 2.0


[ci: skip-checks]
2020-09-22 19:47:38 +02:00
Benjamin Kampmann a9c73113a8 Tracing for wasm with bridging to native (#6916)
* implement events handling, implement parent_id for spans & events

* add events to sp_io::storage

* update test

* add tests

* adjust limit

* let tracing crate handle parent_ids

* re-enable current-id tracking

* add test for threads with CurrentSpan

* fix log level

* remove redundant check for non wasm traces

* remove duplicate definition in test

* Adding conditional events API

* prefer explicit parent_id over current,

enhance test

* limit changes to client::tracing event implementation

* remove From impl due to fallback required on parent_id

* make tracing codecable

* replace with global tracing

* new tracing interface

* impl TracingSubscriber in client

* implement access to global TracingSubscriber from primitives

* span for wasm

* increment towards Wasm Tracing Subscriber implementation

* increment, remove sp-tracing from runtime-interface

* increment, it compiles

* attained original functionality with new mechanism

* implement remaining TracingSubscriber functions

* remove spans from decl_module

* add handling for encoded values

* Revert "replace with global tracing"

This reverts commit 8824a60deea54d9b437407a21c8ceaf6a1902ee5.

* Wasm Side Tracing

* tracing on wasm

* enable tracing wasm on node-runtime

* export all the macros in std

* tracing subscriber on wasm-side only

* pass spans and events over and record them

* reactivate previous code and  cleanup

* further cleaning up

* extend the span macros, activate through executive

* tracking the actual extrinsic, too

* style

* fixing tests

* spaces -> tabs

* attempting to reactivate params

* activate our tests in CI

* some passing

* tests passing

* with core lazy

* global tracer for wasm side with pass over

* fixing metadata referencing

* remove const_fn feature requirement

* reenable dispatch traces

* reset client tracing

* further cleaning up

* fixing runtime-test

* move tracing-build setup into runtime-test

* Merge DebugWriter from tracing and frame-support, move to sp-std

* remove dangling fixme

* Docs for tracing primitives

* cleaning up a bit more

* Wasm interface docs

* optimise docs.rs setup

* adding tracing flags to uncomment

* remove brace

* fixing imports

* fixing broken syntax

* add required modules

* nicer formatting

* better target management

* adding low level storage tracing events into frame

* add custom Debug impl for WasmMetadata

* cloning profiler

* adding info about cloning profiler

* using in-scope for within calls

* proper time tracing, cleaning up println

* allow to disable tracing on runtime_interface-macro

* disable tracing for wasm-tracing-interface

* simplify wasm-tracing-api

* update client to new interface

* fixing docs and tests for sp-tracing

* update integration tests

* re-activating enter_span

* dropping FIXME, it's documented

* fix formatting

* fix formatting

* fix imports

* more debug info

* inform wasm about it being disabled by returning 1

* only one tracer, but enabled multi-all support

* make trait pub again for tests

* Apply suggestions from code review

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* fixing wasm doc tests for proper usage

* remove unnecessary import

* fixing formatting

* minor style fixes

* downgrading wabt

* update error message for UI

* Fix interface test

* next attempt to fix macros

* geee

* revert tracing on hashed for future PR

* remove local macros, use originals

* we are able to convert to static items

* implement more WasmValue types

* adding support to convert str, debug and encoded values

* more minor fixes

* revert unsafe 'static making

* fix indentation

* remove commented lines

* bump all them tracing versions

* cleaning up docs and info

* document new flag

* the new layered system handles span cloning better

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: David <dvdplm@gmail.com>
2020-09-18 06:10:59 +00:00
Jon Häggblad 2da6adfdf1 grandpa-rpc don't share subscription manager, only executor (#7039)
* service builder: fix todo about jsonrpc Option workaround

* grandpa-rpc: only share executor instead of sub manager

* grandpa-rpc: fix compilation

* grandpa-rpc: rename to subscription_executor

* node/cli: remove another unused jsonrpc dependency

* grandpa: apply style fixes from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-09-14 11:02:37 +00:00
Qinxuan Chen 8ae1695526 Remove duplicated dependencies (#6930)
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
2020-08-21 16:25:58 +02:00
Benjamin Kampmann 2416aa7999 prepping for releasing rc6 (#6922)
* Bump version

* update test-utils crates to be ready for publishing

* adding changelog

* Adding automaticly generated READMEs

* fixing versions

* another version mishap
2020-08-20 17:04:42 +02:00