Commit Graph

45 Commits

Author SHA1 Message Date
Seun Lanlege 6328f8740c Allow BabeConsensusDataProvider fork existing chain (#7078)
* parent affbc38afd
author Seun Lanlege <seunlanlege@gmail.com> 1599568164 +0100
committer Seun Lanlege <seunlanlege@gmail.com> 1604321289 +0100
gpgsig -----BEGIN PGP SIGNATURE-----

 iQGzBAABCgAdFiEECvQ02MnjnssnSbjr3HzzEhjN254FAl+gAAkACgkQ3HzzEhjN
 254soAv+KO5JA0HXSe0R0XS5TnwA3IxYsW+UvdF5dXFeC3jFdGTMvor818uoBePD
 dxzYEsUK6gjsNcM9+hpFhoy5JnUrUPInd2BZ7pmZiDuXmYJrHi0s7K5qL0EYDoe0
 m1egPNNyRR125ozJ24M+09c3OQsi3bvTx1TJaV9Aov8hK4So8UmlJTHWpkLw97ku
 HuTre2IPSFbV4GwJE40V+KNuDVHxaKL7zrInYScqbr6/hOTqBCvFn4ib3CjpF5HG
 zDAA5S2PrcbL9NQOothVcVB/TZr3IkhglCFqEjVyCX80IL0JkNZkw8jAh0B8uqXx
 Ug/c1/Mssa8F1jLZMmW45Cway60txqVbcWntPJAymGJbrRErOO/++oUrV0u1C65u
 LW7gXAaIJWQTX9KnX0SEyejNod7ubZktBz7n5WfkJAPIzdw5wtJalhLa673YTgQ9
 zyTPKiWjJj2myCq1AYrJvlK8hSsIBqbBFcUf1zX4SzZWKS+5mtp51o4gfVzcCRPd
 z/6/iPbB
 =g5tx
 -----END PGP SIGNATURE-----

BabeConsensusDataProvider works with existing chains

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-11-04 14:16:08 +01:00
Rakan Alhneiti 3aa4bfacfc Async keystore + Authority-Discovery async/await (#7000)
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 20:56:35 +00:00
Benjamin Kampmann ecdc94420e Releasing 2.0 – two dot 😮 (#7182)
Tagging the release of substrate 2.0


[ci: skip-checks]
2020-09-22 19:47:38 +02:00
Steve Degosserie 22632efc5f Update READMEs, fix links (#7176)
* Re-generate READMEs to fix doc links; set readme field in package manifests

* Re-generate READMEs to fix doc links; set readme field in package manifests

* Re-generate READMEs to fix doc links; set readme field in package manifests

* Re-generate READMEs to fix doc links; set readme field in package manifests

* Revert stuff that shouldn't have been committed

* Revert stuff that shouldn't have been committed

* Fix parent relative link generation

* Manually remove this incorrect link for now.
2020-09-22 13:38:23 +00:00
Maciej Hirsz ae52587f41 Bump jsonrpc-* dependencies to v15 (#7157)
* Bump jsonrpc-* dependencies to v15

* Remove unused imports
2020-09-21 08:46:10 +00:00
Benjamin Kampmann 2416aa7999 prepping for releasing rc6 (#6922)
* Bump version

* update test-utils crates to be ready for publishing

* adding changelog

* Adding automaticly generated READMEs

* fixing versions

* another version mishap
2020-08-20 17:04:42 +02:00
Benjamin Kampmann 629a801b6a Fixup cargo unleash (#6689)
* Bumping to rc5

* Revert "Bumping to rc5"

This reverts commit 5a63a8230e5a12db1b4ce4ac62e4967c52291da7.

* Bumping for rc5, correctly this time
2020-07-21 10:26:47 +00:00
Benjamin Kampmann f6ebd5f87d Releasing rc4 – Rhinoceros (#6515)
Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-06-25 23:18:43 +02:00
Rakan Alhneiti d25f460b63 Babe VRF Signing in keystore (#6225)
* Introduce trait

* Implement VRFSigner in keystore

* Use vrf_sign from keystore

* Convert output to VRFInOut

* Simplify conversion

* vrf_sign secondary slot using keystore

* Fix RPC call to claim_slot

* Use Public instead of Pair

* Check primary threshold in signer

* Fix interface to return error

* Move vrf_sign to BareCryptoStore

* Fix authorship_works test

* Fix BABE logic leaks

* Acquire a read lock once

* Also fix RPC acquiring the read lock once

* Implement a generic way to construct VRF Transcript

* Use make_transcript_data to call sr25519_vrf_sign

* Make sure VRFTranscriptData is serializable

* Cleanup

* Move VRF to it's own module

* Implement & test VRF signing in testing module

* Remove leftover

* Fix feature requirements

* Revert removing vec macro

* Drop keystore pointer to prevent deadlock

* Nitpicks

* Add test to make sure make_transcript works

* Fix mismatch in VRF transcript

* Add a test to verify transcripts match in babe

* Return VRFOutput and VRFProof from keystore
2020-06-18 14:37:49 -04:00
Xiliang Chen 7d30ae7ba8 add system_dryRun (#6300)
* add system_dryRun

* fix build error

* delete unneeded code

* return ApplyExtrinsicResult directly

* line width

* mark dry run unsafe

* line width

* fix test

* add test

* update comment
2020-06-16 22:51:03 +02:00
Benjamin Kampmann 881072e590 Intent to release rc3 (#6290) 2020-06-08 23:29:52 +02:00
Hernando Castano 9bd23e7f01 Use Subscription Manager from jsonrpc-pubsub: The Sequel (#6254)
* Bump jsonrpc pubsub, core, http, and ws

Right now these are the packages which _need_ to be updated
so I can just the latest `jsonrpc-pubsub` code. Once a release
it cut upstream the rest of the dependencies should be updated
as well.

* Use jsonrpc-pubsub's SubscriptionManager

This places sc-rpc-api::Subscriptions

* Bump jsonrpc-core outside of sc-rpc-*

* Update client/rpc tests

Right now one of the `author` tests is failing, I
need to think a bit about how best to fix it.

* Remove Subscriptions manager

There's no need for this implementation since we're
using the one from `jsonrpc-pubsub` now

* Fix author RPC test

This test used to check for a numerial subscription ID,
whereas now it uses a string based ID which is the default
provided by `jsonrpc-pubsub`'s subscription manager.

* Remove unused NumericIdProvider

* Add missing bracket

Removed one too many with that last one, lol

* Bump `jsonrpc` to v14.2

There's an exception though. `jsonrpc-derive` cannot be bumped
past v14.0.5 just yet since it has a dependency on `quote` pinned
to v1.0.1. This means that at the moment it won't build on Substrate
since it's using v1.0.3.

* Track `jsonrpc-derive` master branch

* Bump `quote` version to v1.0.6

* Bump `jsonrpc-derive` to v14.2.1

This includes support for `quote` v1.0.6

* Use exact version for jsonrpc crates

Doing this to make sure any updates in jsonrpc don't
accidently trickle down to Polkadot.

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-06-08 17:47:09 +01:00
André Silva 8f5a52fe1a Revert "Use Subscription Manager from jsonrpc-pubsub (#6208)" (#6252)
This reverts commit ea1eb4e57f.
2020-06-04 15:39:57 +01:00
Hernando Castano ea1eb4e57f Use Subscription Manager from jsonrpc-pubsub (#6208)
* Bump jsonrpc pubsub, core, http, and ws

Right now these are the packages which _need_ to be updated
so I can just the latest `jsonrpc-pubsub` code. Once a release
it cut upstream the rest of the dependencies should be updated
as well.

* Use jsonrpc-pubsub's SubscriptionManager

This places sc-rpc-api::Subscriptions

* Bump jsonrpc-core outside of sc-rpc-*

* Update client/rpc tests

Right now one of the `author` tests is failing, I
need to think a bit about how best to fix it.

* Remove Subscriptions manager

There's no need for this implementation since we're
using the one from `jsonrpc-pubsub` now

* Fix author RPC test

This test used to check for a numerial subscription ID,
whereas now it uses a string based ID which is the default
provided by `jsonrpc-pubsub`'s subscription manager.

* Remove unused NumericIdProvider

* Add missing bracket

Removed one too many with that last one, lol

* Bump `jsonrpc` to v14.2

There's an exception though. `jsonrpc-derive` cannot be bumped
past v14.0.5 just yet since it has a dependency on `quote` pinned
to v1.0.1. This means that at the moment it won't build on Substrate
since it's using v1.0.3.

* Track `jsonrpc-derive` master branch

* Bump `quote` version to v1.0.6

* Bump `jsonrpc-derive` to v14.2.1

This includes support for `quote` v1.0.6
2020-06-04 09:50:22 +01:00
Benjamin Kampmann b64f55089d Releasing rc2 (#6136) 2020-05-26 14:32:23 +02:00
Benjamin Kampmann fb32ac8c51 Tagging as rc1 2020-05-25 18:30:48 +02:00
André Silva 0ddd5cc278 babe: treat epoch_authorship RPC method as unsafe (#6069)
* service: pass DenyUnsafe to rpc extensions

* node: add DenyUnsafe to rpc full node dependencies

* client: fix whitespace in rpc policy file

* babe: treat epochAuthorship rpc method as unsafe

* babe: add test for unsafe rpc method

* babe: rename babe rpc handler

* service: traitify rpc extension builder

* service: make the rpc extensions builder api non-breaking

* service: revert changes from light node rpc extensions builder

* node: remove unnecessary type in service creation

* service: cleanup with_rpc_extensions implementation

* service: add missing docs to RpcExtensionBuilder
2020-05-21 13:58:04 +02:00
Pierre Krieger 717fa95bc7 Restore the empty line after the license (#6088) 2020-05-20 13:08:27 +00:00
Nikolay Volf d5411969ac fix whitespace (#6062) 2020-05-18 13:51:28 +02:00
Benjamin Kampmann 7c565085e9 resetting to -dev (#6050) 2020-05-16 19:03:04 +02:00
Benjamin Kampmann d7463d348f Releasing Alpha.8 (#6048) 2020-05-15 21:56:56 +02:00
Rakan Alhneiti f36f57b0bf Use sign_with in consensus (#6008)
* Add derive_more to sp_core

* Convert Vec to Signature

* Use sign_with in AURA and BABE

* Signing errors

* Update slots to return consensus result

* Fix use

* Clone public key

* Match block_params

* WIP

* Use to_public_crypto_pair

* Pass public key only to block import params

* Address PR review

* Fix consensus RPC

* Fix babe tests

* adjust uses

* Fix line widths
2020-05-15 15:03:52 +00:00
Benjamin Kampmann be8c96adec Relax substrate licensing scheme (#5947) 2020-05-15 13:21:56 +02:00
André Silva c91e681db2 babe: minor rpc nits (#5974)
* babe: fix formatting

* babe: re-use same epoch data in epoch_authorship rpc method
2020-05-12 15:18:15 +02:00
Jon Häggblad 66931028f4 Don't repeatedly lookup keys in babe_epochAuthorship rpc function (#5962)
* babe: don't repeatedly lookup keys in authorship rpc function

Expose a new function `claim_slot_using_keypars` in Babe so that the `babe_epochAuthorship` can
lookup authorship for all slots in the epoch without repeatedly looking up keys in the keystore.

Time to run the `babe_epochAuthorship` RPC call goes from 7s to 25ms on a local dev chain on my
machine.

* babe: pass reference to slice instead of ref to Vec

* babe: fix bunch of clippy warnings
2020-05-11 10:48:34 +02:00
Benjamin Kampmann 3860999ea3 post release dev reset (#5911) 2020-05-05 22:08:31 +01:00
Benjamin Kampmann 45886bd197 Release of Alpha.7 (#5904) 2020-05-05 20:50:46 +02:00
Benjamin Kampmann 0ca88bbb18 activate compat feature on futures 2020-04-29 14:23:38 +02:00
Wei Tang 969720c2ad babe: secondary blocks with VRF (#5501)
* babe: secondary blocks with VRF

* Fix node runtime compile

* Fix test-utils runtime interface

* Fix babe tests

* typo: v == 2

* babe: support online configuration upgrades

* Fix rpc tests

* Fix runtime version tests

* Switch to use NextConfigDescriptor instead of changing runtime interface

* Fix tests

* epoch-changes: map function that allows converting with different epoch types

* Add migration script for the epoch config change

* Fix docs for PrimaryAndSecondaryVRFSlots

* Add docs of `SecondaryVRF` in babe crate

* babe-primitives: Secondary -> SecondaryPlain

* babe-client: Secondary -> SecondaryPlain

* Fix migration tests

* test-utils-runtime: Secondary -> SecondaryPlain

* Fix missing name change in test-utils-runtime

* Fix migration: Epoch should be EpochV0

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* Fix new epochChanges version

* Fix babe-primitives naming changes

* Fix merge issues in babe-client

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-04-24 16:03:03 +01:00
Wei Tang 770cc24c47 babe: support online configuration upgrades (#5514)
* babe: support online configuration upgrades

* Switch to use NextConfigDescriptor instead of changing runtime interface

* Fix tests

* epoch-changes: map function that allows converting with different epoch types

* Add migration script for the epoch config change

* Fix migration tests

* Fix migration: Epoch should be EpochV0

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>

* Fix new epochChanges version

* Fix unused imports

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-04-24 14:59:14 +01:00
Benjamin Kampmann 3426d662f7 Switch pre-release version to dev (#5637) 2020-04-15 13:18:08 +02:00
Benjamin Kampmann 51f9bb3c0e Prepping release of alpha.6 (#5629)
* bumping version

* cargo update

* adding changelog
2020-04-14 21:41:49 +02:00
Nikolay Volf 159d830d62 remove ws feature from json client (#5478) 2020-04-01 01:16:24 +02:00
pscott 408455f8bc Build for only one target for docs.rs (#5427)
* Add docs.rs metadata to all cargo.toml files

* Remove docs.rs metada in substrate's cargo.toml
2020-03-30 09:46:30 +02:00
Nikolay Volf eed0045a0c update futures dependency (#5426) 2020-03-27 12:35:48 +01:00
Benjamin Kampmann ba06d5ef64 Releasing 2.0.0-alpha.5 (#5340)
* bump pre-version

* Changelog alpha.4->alpha.5

* repo + homepage to sp-consensus-vrf

* Add default rocksdb feature to `frame-benchmarking-cli` (#5367)

* Add default rocksdb feature

* Update Cargo.toml

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-03-24 15:08:02 +01:00
Benjamin Kampmann 25e25a83b9 Releasing 2.0.0-alpha.4 (#5279)
* Version bump

* Split generate_changelog.sh into separate script

Can be run in the format `generate_changelog.sh $previous_version $version`.

* remove early exit from publish_draft_release.sh

* adding changelog

* ci: change last_github_release to also find pre-releases

Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-03-18 14:37:21 +01:00
Wei Tang 846a9ce8c6 epoch-changes: separate epoch header with epoch data (#4881)
* fork-tree: prune returns all pruned node data

* epoch-changes: split EpochHeader vs epoch data

* EpochChanges::viable_epoch and add missing comments

* Incoperate the new epoch_changes interface for BABE

* Fix BABE tests

* Fix fork-tree pruning issue

* Fix tests

* Fix pruning algorithm

* fork-tree: implement map function for mapping one value type to another

* Add migration script for new epoch changes scheme

* Update utils/fork-tree/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update client/consensus/slots/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Remove authorities_len.is_none check, which is duplicate of unwrap_or(false)

* Update client/consensus/epochs/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update client/consensus/epochs/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* No trailing ; for return statement

* Use VERSION_KEY for migration

* Fix issues that removed nodes are not properly added into removed list

* Add comments indicating end_slot is non-inclusive

* fork-tree: use &mut F for map type declaration

* Add tests for v0 epoch_changes migration

* Fix babe RPC tests

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-03-16 17:06:13 -04:00
Seun Lanlege 9898fc7e81 removes use of sc_client::Client from sc_network (#5147)
* removes use of sc_client::Client from sc_network

* rename BlockProvider to BlockBackend

* fix broken test
2020-03-12 14:02:04 +01:00
Benjamin Kampmann 25d1b7878a prepping for Alpha.3 (#5080)
* Bump to alpha.3

* update gitlab-ci
2020-02-27 22:06:08 +01:00
Benjamin Kampmann d6aa8e954c set missing metadata fields, prepping alpha.2 (#5067)
* setting first batch of descriptions

* fix what I just broke

* next batch

* and pallets, too

* last batch

* set cargo.lock

* keep'em dev-deps

* bump version to alpha.2
2020-02-27 00:22:44 +01:00
Benjamin Kampmann e2776f42f9 prepare version to alpha.1 (#5055)
bump version to -alpha.1
2020-02-25 21:44:23 +01:00
Benjamin Kampmann c412c6230e Cargo.toml fixups for the release (#4975)
* setting versions to development pre-release

fixing version in dependencies

* unset already released wasm-builder

* do not publish test crates

* adding licenses

* setting homepage metadata

* set repository url
2020-02-21 17:07:00 +01:00
Seun Lanlege 832b2ef642 Removes use of sc_client::Client from sc_consensus_babe (#5014)
* removes use of sc_client::Client from sc_consensus_babe

* Update client/consensus/babe/src/lib.rs

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
2020-02-21 12:13:16 +01:00
Seun Lanlege 5567d18b6a Adds babe rpc support (#4729)
* babe_epochAuthorship
remove test-helpers from sp-keyring, bump spec_version, impl_version

* bump Cargo.lock

* add BabeRPC to node-rpc

* rename to BabeApi, remove err_derive

* pass &ServiceBuilder to with_rpc_extensions callback

* sc-consensus-babe-rpc

* Update client/consensus/babe/src/lib.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Better docs, code style chanegs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* new line at the end of Cargo.toml

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
2020-02-17 18:05:30 +00:00