Commit Graph

91 Commits

Author SHA1 Message Date
Shawn Tabrizi 36a684a14b Support Multiple Instances with Benchmarks (#7669)
* Support multiple instances with benchmarks

* fix tests

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* docs

* fix output

* Update lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-12-11 09:20:10 +00:00
Shawn Tabrizi 3ac070b61e Use Handlebars Template for Benchmark CLI Output (#7390)
* add_handlebar_template_to_benchmark

- add benchmark-cli arg to take in a handlebar-template file

* update to always use template

* rewrite writer for handlebars

* polish

* pass cmd data

* update docs

* support custom filename output

* Update command.rs

* Create frame-weight-template.hbs

* use a vector to maintain benchmark order

* fix tests

* Custom string serializer, remove feature flag

* update docs

* docs on public objects

* small fix

Co-authored-by: Ezadkiel Marbella <zadkiel.m@gmail.com>
2020-10-27 12:25:57 +00:00
Rakan Alhneiti 3aa4bfacfc Async keystore + Authority-Discovery async/await (#7000)
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 20:56:35 +00:00
Shawn Tabrizi 4e5f9fb72b More Improvements to Benchmarking CLI + Generic WeightInfo (#7168)
* Generic WeightInfo, Output File Path

* Make `WeightInfo` generic

* Add support for header file

* fix line width

* add `--spaces` flag

* Configurable trait and struct strings

* make elections phragmen generic

* Add date to generated file

* Apply suggestions from code review

Co-authored-by: Gavin Wood <gavin@parity.io>

* fixes

* add settings metadata to output

Co-authored-by: nikvolf <nikvolf@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
2020-09-23 13:56:30 +00:00
Benjamin Kampmann ecdc94420e Releasing 2.0 – two dot 😮 (#7182)
Tagging the release of substrate 2.0


[ci: skip-checks]
2020-09-22 19:47:38 +02:00
Steve Degosserie 22632efc5f Update READMEs, fix links (#7176)
* Re-generate READMEs to fix doc links; set readme field in package manifests

* Re-generate READMEs to fix doc links; set readme field in package manifests

* Re-generate READMEs to fix doc links; set readme field in package manifests

* Re-generate READMEs to fix doc links; set readme field in package manifests

* Revert stuff that shouldn't have been committed

* Revert stuff that shouldn't have been committed

* Fix parent relative link generation

* Manually remove this incorrect link for now.
2020-09-22 13:38:23 +00:00
Benjamin Kampmann 2416aa7999 prepping for releasing rc6 (#6922)
* Bump version

* update test-utils crates to be ready for publishing

* adding changelog

* Adding automaticly generated READMEs

* fixing versions

* another version mishap
2020-08-20 17:04:42 +02:00
Shawn Tabrizi dd6ef0a3a5 Improve Benchmark Writer: Remove Unused Components, Remove Multiply by Zero, Files Split by Pallet (#6785)
* initial improvements

* better file management, ignore unused components

* Output warning when components unused

* update comment

* Write even when base weight is zero

* remove unwrap where possible

* Dont sort components to dedup

* undo delete

* improve clarity of unused components

* remove unused dep

* Update Process.json
2020-08-03 12:41:54 +00:00
Benjamin Kampmann 629a801b6a Fixup cargo unleash (#6689)
* Bumping to rc5

* Revert "Bumping to rc5"

This reverts commit 5a63a8230e5a12db1b4ce4ac62e4967c52291da7.

* Bumping for rc5, correctly this time
2020-07-21 10:26:47 +00:00
Shawn Tabrizi 2019f70768 Benchmarks Writer CLI (#6567)
* initial mockup

* add and wipe

* track writes

* start to add to pipeline

* return all reads/writes

* Log reads and writes from bench db

* causes panic

* Allow multiple commits

* commit before ending benchmark

* doesn't work???

* fix

* Update lib.rs

* switch to struct for `BenchmarkResults`

* add to output

* fix test

* line width

* @kianenigma review

* Add Whitelist to DB Tracking in Benchmarks Pipeline (#6405)

* hardcoded whitelist

* Add whitelist to pipeline

* Remove whitelist pipeline from CLI, add to runtime

* clean-up unused db initialized whitelist

* Add regression analysis to DB Tracking (#6475)

* Add selector

* add tests

* debug formatter for easy formula

* initial idea

* use all benchmarks

* broken

* working without trait

* Make work for multiple pallets

* Fix merge issues

* writer appends to file

* implement () for balances weight trait

* update name of trait

* Weights to WeightInfo

* auto trait writer

* Heap pages are configurable

* clean out runtime changes

* more clean up

* Fix string generation

* Update comments

* Update bin/node/runtime/src/lib.rs

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
2020-07-06 09:34:24 +00:00
Benjamin Kampmann f6ebd5f87d Releasing rc4 – Rhinoceros (#6515)
Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-06-25 23:18:43 +02:00
Arkadiy Paronyan 7b34438178 Validate encoding of extrinsics passed to runtime (#6442)
* Validate encoding of extrinsics passed to runtime

* Bump codec version explicitly
2020-06-19 19:27:16 +00:00
Benjamin Kampmann 881072e590 Intent to release rc3 (#6290) 2020-06-08 23:29:52 +02:00
pscott 05ed129673 Make all features explicit (#6267)
* make all features explicit

* Change to -feature suffix to with- prefix

* Add newline at the end of the Cargo.toml file

* Remove rhd feature

* Remove some features from Cargo.toml

* Remove test-helpers feature in tx pool

* Return db_open_error("with-"..

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Rename subdb feature to with-subdb

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove 'strict' feature and cfg_attr

* Check for with-subdb feature instead of subdb

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-06-08 15:43:00 +02:00
Benjamin Kampmann b64f55089d Releasing rc2 (#6136) 2020-05-26 14:32:23 +02:00
Benjamin Kampmann fb32ac8c51 Tagging as rc1 2020-05-25 18:30:48 +02:00
Benjamin Kampmann 7c565085e9 resetting to -dev (#6050) 2020-05-16 19:03:04 +02:00
Benjamin Kampmann d7463d348f Releasing Alpha.8 (#6048) 2020-05-15 21:56:56 +02:00
Benjamin Kampmann be8c96adec Relax substrate licensing scheme (#5947) 2020-05-15 13:21:56 +02:00
Benjamin Kampmann 3860999ea3 post release dev reset (#5911) 2020-05-05 22:08:31 +01:00
Benjamin Kampmann 45886bd197 Release of Alpha.7 (#5904) 2020-05-05 20:50:46 +02:00
Seun Lanlege 4fa5941f44 Move sc-client into sc-service (#5502)
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api

* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa

* drop sc-service from sc-rpc

* drop sc-service from sc-consensus-aura

* drop sc-client from manual-seal and babe

* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli

* drop sc-client from bin/node and bin/node-template

* drop sc-client

* fix tests

* remove check -p sc-client from gitlab.yml

* fix warnings

* fixes ui test

* fix light client tests

* adds associated Client type to AbstractService

* adds UsageProvider to Client

* fixed ui test, again

* tried and failed to get node-cli to compile for wasm

* thanks to tomaka for helping me get node-cli to compile for wasmm

* ui test pls pas 🙏🏾

* all tests passing 🪄

* no_run documentation code

* rm -f documentation code

* ClientProvider

* fix mega trait

* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client

* adds license to sc-consensus

Co-authored-by: Benjamin Kampmann <ben@parity.io>
2020-04-28 11:59:31 +00:00
Gavin Wood 91af5b6fcc New database trait (#5549)
* Introduce trait

* The trait

* Generic

* Basic impls.

* Remove unneeded bounds

* Minor changes

* Switch over to the new DB trait

* Integrated parity-db and added CLI for db selection

* Default impl.

* Fix logs.

* Started integrating subdb

* Apply suggestions from code review

Co-Authored-By: Cecile Tonglet <cecile@parity.io>

* Apply suggestions from code review

Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>

* Enable subdb

* Bump parity-db

* Fixed CLI macro

* Fixed browser build

* Fixed features

* Sort out features

* Use parity-db from crates.io

* Typo

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Cecile Tonglet <cecile@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-04-15 14:38:39 +02:00
Benjamin Kampmann 3426d662f7 Switch pre-release version to dev (#5637) 2020-04-15 13:18:08 +02:00
Benjamin Kampmann 51f9bb3c0e Prepping release of alpha.6 (#5629)
* bumping version

* cargo update

* adding changelog
2020-04-14 21:41:49 +02:00
pscott 408455f8bc Build for only one target for docs.rs (#5427)
* Add docs.rs metadata to all cargo.toml files

* Remove docs.rs metada in substrate's cargo.toml
2020-03-30 09:46:30 +02:00
Bastian Köcher 1242a7df3c Upgrade parity-scale-codec to 1.3.0 (#5443) 2020-03-28 22:16:05 +01:00
Benjamin Kampmann ba06d5ef64 Releasing 2.0.0-alpha.5 (#5340)
* bump pre-version

* Changelog alpha.4->alpha.5

* repo + homepage to sp-consensus-vrf

* Add default rocksdb feature to `frame-benchmarking-cli` (#5367)

* Add default rocksdb feature

* Update Cargo.toml

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-03-24 15:08:02 +01:00
Shawn Tabrizi 400a62680f Benchmark Im Online Pallet (#5318)
* Initial benchmarking setup

* Add keystore

* validate unsigned

* Update frame/im-online/src/benchmarking.rs

Co-Authored-By: Marcio Diaz <marcio.diaz@gmail.com>

* Fix verify_unsigned benchmark

* add variable for teting the external addresss length

* Update frame/im-online/src/benchmarking.rs

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Marcio Diaz <marcio.diaz@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-20 12:42:00 +01:00
Benjamin Kampmann 25e25a83b9 Releasing 2.0.0-alpha.4 (#5279)
* Version bump

* Split generate_changelog.sh into separate script

Can be run in the format `generate_changelog.sh $previous_version $version`.

* remove early exit from publish_draft_release.sh

* adding changelog

* ci: change last_github_release to also find pre-releases

Co-authored-by: Martin Pugh <pugh@s3kr.it>
2020-03-18 14:37:21 +01:00
Nikolay Volf 372745705d create parallel tasks extension (#5249) 2020-03-16 16:30:39 +01:00
Pierre Krieger f21680209f Minor fixes for Android compilation (#5209)
* Minor fixes for Android

* Update parity-multiaddr

* Bump in service Cargo.toml too
2020-03-10 20:54:58 +01:00
Bastian Köcher 1cfcf5cbfe Don't include :code by default in storage proofs (#5179)
* Don't include `:code` by default in storage proofs (#5060)

* Adds test to verify that the runtime currently is always contained in
the proof

* Start passing the runtime wasm code from the outside

* Fix compilation

* More build fixes

* Make the test work as expected now :)

* Last fixes

* Fixes benchmarks

* Review feedback

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Review feedback

* Fix compilation

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>

* Fix compilation and change the way `RuntimeCode` works

* Fix tests

* Switch to `Cow`

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
2020-03-10 11:13:20 +01:00
Arkadiy Paronyan 9a1b3b53f2 Revert "Don't include :code by default in storage proofs (#5060)" (#5136)
This reverts commit 6ee39261c8.
2020-03-05 10:55:05 +01:00
Benjamin Kampmann 6ee39261c8 Don't include :code by default in storage proofs (#5060)
* Adds test to verify that the runtime currently is always contained in
the proof

* Start passing the runtime wasm code from the outside

* Fix compilation

* More build fixes

* Make the test work as expected now :)

* Last fixes

* Fixes benchmarks

* Review feedback

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Review feedback

* Fix compilation

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
2020-03-04 20:26:16 +01:00
Bastian Köcher a1b53280b4 Update to SCALE 1.2.0 (#5113)
This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.
2020-03-03 14:40:02 +01:00
Benjamin Kampmann 25d1b7878a prepping for Alpha.3 (#5080)
* Bump to alpha.3

* update gitlab-ci
2020-02-27 22:06:08 +01:00
Benjamin Kampmann d6aa8e954c set missing metadata fields, prepping alpha.2 (#5067)
* setting first batch of descriptions

* fix what I just broke

* next batch

* and pallets, too

* last batch

* set cargo.lock

* keep'em dev-deps

* bump version to alpha.2
2020-02-27 00:22:44 +01:00
Benjamin Kampmann e2776f42f9 prepare version to alpha.1 (#5055)
bump version to -alpha.1
2020-02-25 21:44:23 +01:00
Benjamin Kampmann c412c6230e Cargo.toml fixups for the release (#4975)
* setting versions to development pre-release

fixing version in dependencies

* unset already released wasm-builder

* do not publish test crates

* adding licenses

* setting homepage metadata

* set repository url
2020-02-21 17:07:00 +01:00
Bastian Köcher b4ebd41c21 Consolidate frame benchmarking into a frame crate (#4977)
This prs cleans up some of the frame benchmarking stuff:
- Move CLI into `frame-benchmarking-cli`. No frame related CLI should
exists in the default Substrate CLI.
- Move all traits and types related to frame benchmarking into the
`frame-benchmarking` trait. Frame types should be isolated in Frame.
2020-02-19 10:22:36 +01:00