Commit Graph

55 Commits

Author SHA1 Message Date
Tomasz Drwięga f3d4355a20 Custom RPC for Merkle Mountain Range pallet (#8137)
* Add MMR custom RPC.

* Change RuntimeApi to avoid hardcoding leaf type.

* Properly implement the new RuntimeAPI and wire up RPC.

* Extract Offchain DB as separate execution extension.

* Enable offchain DB access for offchain calls.

* Fix offchain_election tests.

* Skip block initialisation for proof generation.

* Fix integration test setup.

* Fix offchain tests. Not sure how I missed them earlier 🤷.

* Fix long line.

* One more test missing.

* Update mock for multi-phase.

* Address review grumbbles.

* Address review grumbles.

* Fix line width of a comment
2021-03-10 15:28:56 +00:00
Guillaume Thiolliere 9c4e4d453c Fix doc build with --all-features (#8277)
* implement

* make default pre/post_upgrade

* simplify Cargo.toml

* revert removal of outdated/private links

* link in pallet-mmr
2021-03-05 16:08:44 +01:00
Shawn Tabrizi d25229bc89 Return number of keys removed when calling storage_kill on child trie (#8166)
* Initial piping of returning amount of keys killed

* One more test for `None` limit

* forgot to update

* fix return value

* use version 3

* Update to return `KillOutcome`

* Update name to KillChildStorageResult
2021-02-22 19:24:12 +00:00
Bastian Köcher 4f4a0c5b38 Make keystore return None when a key doesn't exist (#8163)
* Make keystore return `None` when a key doesn't exist

* Fixes

* More fixes

* Update comment

* Update primitives/keystore/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/keystore/src/local.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Address comments

* Update client/keystore/src/local.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-02-22 14:24:09 +00:00
Bastian Köcher e3e651f72c Happy new year (#7814)
* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL
2021-01-04 09:03:13 +00:00
Alexander Theißen ec2c70396e Define ss58 prefix inside the runtime (#7810)
* Add SS58Prefix type to the frame_system config trait

* Remove unused chain_id runtime interface
2020-12-30 22:07:37 +00:00
Tomasz Drwięga abe119abba Add keccak-512 to host functions. (#7531) 2020-12-11 16:41:18 +01:00
Alexander Theißen 9ce24fe1f4 Allow capping the amount of work performed when deleting a child trie (#7671)
* Allow Backend::for_keys_in_child_storage to be aborted by the closure

* Ext::kill_child_storage now takes an upper limit for backend deletion

* Add Storage::storage_kill_limited() runtime interface

* review: Use a new version of kill_storage instead of a new interface

* review: Simplify boolean expression

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* review: Rename for_keys_in_child_storage

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
2020-12-09 01:17:28 +00:00
Andrew Plaza f6dc35284f Change TRACING_SET to static (#7607)
* change TRACING_SET to static

* Update primitives/io/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* modify test with nested spans

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-11-28 09:40:24 +01:00
Nikolay Volf a062bc2f1d Runtime worker threads (#7089)
* std variant

* principal work

* format and naming

* format and naming continued

* working nested fork

* add comment

* naming and tabs

* line width

* fix wording

* address review

* refactor dynamic dispatch

* update wasmtime

* some care

* move ext

* more refactor

* doc effort

* simplify

* doc effort

* tests and docs

* address review

* naming

* explain some args

* add example

* unwinding for native and tests

* rename stray

* fix refs

* fix tests

* fix warnings

* stray naming

* fixes and comments

* Update primitives/io/src/tasks.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* make examples "compile"

* dyn_dispatch -> spawn_call

* fix impl

* address review

* Update primitives/io/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update primitives/io/src/async_externalities.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/example-parallel/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* fix compilation

* Update client/executor/common/src/wasm_runtime.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* address review

* Update client/executor/wasmtime/src/instance_wrapper.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Update client/executor/src/native_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/src/native_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/io/src/tasks.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/executor/wasmtime/src/instance_wrapper.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* address some issues

* address more issues

* wasm_only interface

* define sp_tasks

* avoid anyhow

* fix example

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-10-20 12:41:51 +00:00
Rakan Alhneiti 3aa4bfacfc Async keystore + Authority-Discovery async/await (#7000)
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 20:56:35 +00:00
Benjamin Kampmann a9c73113a8 Tracing for wasm with bridging to native (#6916)
* implement events handling, implement parent_id for spans & events

* add events to sp_io::storage

* update test

* add tests

* adjust limit

* let tracing crate handle parent_ids

* re-enable current-id tracking

* add test for threads with CurrentSpan

* fix log level

* remove redundant check for non wasm traces

* remove duplicate definition in test

* Adding conditional events API

* prefer explicit parent_id over current,

enhance test

* limit changes to client::tracing event implementation

* remove From impl due to fallback required on parent_id

* make tracing codecable

* replace with global tracing

* new tracing interface

* impl TracingSubscriber in client

* implement access to global TracingSubscriber from primitives

* span for wasm

* increment towards Wasm Tracing Subscriber implementation

* increment, remove sp-tracing from runtime-interface

* increment, it compiles

* attained original functionality with new mechanism

* implement remaining TracingSubscriber functions

* remove spans from decl_module

* add handling for encoded values

* Revert "replace with global tracing"

This reverts commit 8824a60deea54d9b437407a21c8ceaf6a1902ee5.

* Wasm Side Tracing

* tracing on wasm

* enable tracing wasm on node-runtime

* export all the macros in std

* tracing subscriber on wasm-side only

* pass spans and events over and record them

* reactivate previous code and  cleanup

* further cleaning up

* extend the span macros, activate through executive

* tracking the actual extrinsic, too

* style

* fixing tests

* spaces -> tabs

* attempting to reactivate params

* activate our tests in CI

* some passing

* tests passing

* with core lazy

* global tracer for wasm side with pass over

* fixing metadata referencing

* remove const_fn feature requirement

* reenable dispatch traces

* reset client tracing

* further cleaning up

* fixing runtime-test

* move tracing-build setup into runtime-test

* Merge DebugWriter from tracing and frame-support, move to sp-std

* remove dangling fixme

* Docs for tracing primitives

* cleaning up a bit more

* Wasm interface docs

* optimise docs.rs setup

* adding tracing flags to uncomment

* remove brace

* fixing imports

* fixing broken syntax

* add required modules

* nicer formatting

* better target management

* adding low level storage tracing events into frame

* add custom Debug impl for WasmMetadata

* cloning profiler

* adding info about cloning profiler

* using in-scope for within calls

* proper time tracing, cleaning up println

* allow to disable tracing on runtime_interface-macro

* disable tracing for wasm-tracing-interface

* simplify wasm-tracing-api

* update client to new interface

* fixing docs and tests for sp-tracing

* update integration tests

* re-activating enter_span

* dropping FIXME, it's documented

* fix formatting

* fix formatting

* fix imports

* more debug info

* inform wasm about it being disabled by returning 1

* only one tracer, but enabled multi-all support

* make trait pub again for tests

* Apply suggestions from code review

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

* fixing wasm doc tests for proper usage

* remove unnecessary import

* fixing formatting

* minor style fixes

* downgrading wabt

* update error message for UI

* Fix interface test

* next attempt to fix macros

* geee

* revert tracing on hashed for future PR

* remove local macros, use originals

* we are able to convert to static items

* implement more WasmValue types

* adding support to convert str, debug and encoded values

* more minor fixes

* revert unsafe 'static making

* fix indentation

* remove commented lines

* bump all them tracing versions

* cleaning up docs and info

* document new flag

* the new layered system handles span cloning better

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: David <dvdplm@gmail.com>
2020-09-18 06:10:59 +00:00
Bastian Köcher 9a69be0587 Fix storage::read (#7084)
* Fix `storage::read`

It should return the length of the storage item after the given offset.
Before it returned always the length of the full storage item.

* Fix tests
2020-09-12 12:17:26 +02:00
kaichao 91c0213413 Set reserved nodes with offchain worker. (#6996)
* add offchain worker api to set reserved nodes.

* new offchain api to get node public key.

* node public key from converter

* refactor set reserved nodes ocw api.

* new ndoe authorization pallet

* remove unnecessary clone and more.

* more

* tests for node authorization pallet

* remove dependency

* fix build

* more tests.

* refactor

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update frame/node-authorization/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update frame/node-authorization/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update frame/node-authorization/src/lib.rs

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* format code

* expose NetworkService

* remove NetworkStateInfo in offchain

* replace NodePublicKey with PeerId.

* set max length of peer id.

* clear more

* use BTreeSet for set of peers.

* decode opaque peer id.

* extract NetworkProvider for client offchain.

* use OpaquePeerId in node authorization pallet.

* fix test

* better documentation

* fix test

* doc

* more fix

* Update primitives/core/src/offchain/mod.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update client/offchain/src/api.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* derive serialize and deserialize

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
2020-09-10 15:26:09 +00:00
Bastian Köcher 9310f15ac2 Name all the tasks! (#6726)
* Remove any implementation of `Spawn` or `Executor` from our task executors

* Fix compilation

* Rename `SpawnBlockingExecutor`

* Update primitives/core/src/traits.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix tests

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2020-07-26 12:56:17 +00:00
Bastian Köcher 8ab23fafdc Make signature batching use specialized methods (#6616)
It was a mistake to use the `*_verify` methods for signature batching.
This pr move the signature batching into their own functions. This is
required, because otherwise transaction signature verification infers
with other signature verifications.

This pr also temporarily disables signature batching. The functionality
stays, but we need to make sure that all nodes have the new runtime
interface, before we can bring back signature batching.
2020-07-12 15:43:47 +02:00
Bastian Köcher 903cd4e801 Clarify some host interfaces (#6626) 2020-07-11 20:47:38 +02:00
Alexander Theißen bb2df2122e Implement nested storage transactions (#6269)
* Add transactional storage functionality to OverlayChanges

A collection already has a natural None state. No need to
wrap it with an option.

* Add storage transactions runtime interface

* Add frame support for transactions

* Fix committed typo

* Rename 'changes' variable to 'overlay'

* Fix renaming change

* Fixed strange line break

* Rename clear to clear_where

* Add comment regarding delete value on mutation

* Add comment which changes are covered by a transaction

* Do force the arg to with_transaction return a Result

* Use rust doc comments on every documentable place

* Fix wording of insert_diry doc

* Improve doc on start_transaction

* Rename value to overlayed in close_transaction

* Inline negation

* Improve wording of close_transaction comments

* Get rid of an expect by using get_or_insert_with

* Remove trailing whitespace

* Rename should to expected in tests

* Rolling back a transaction must mark the overlay as dirty

* Protect client initiated storage tx from being droped by runtime

* Review nits

* Return Err when entering or exiting runtime fails

* Documentation fixup

* Remove close type

* Move enter/exit runtime to excute_aux in the state-machine

* Rename Discard -> Rollback

* Move child changeset creation to constructor

* Move child spawning into the closure

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fixup for code suggestion

* Unify re-exports

* Rename overlay_changes to mod.rs and move into subdir

* Change proof wording

* Adapt a new test from master to storage-tx

* Suggestions from the latest round of review

* Fix warning message

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-06-23 11:17:53 +02:00
mattrutherford 74a989f353 Runtime interface to add support for tracing from wasm (#6381)
* Add span recording to tracing implementation

* Add tracing proxy

* switch to rustc_hash::FxHashMap

* Replace lazy_static and hashmap with thread_local and vec.

* fix marking valid span as invalid while removing invalid spans

* refactor, add wasm_tracing module in `support`

* update registered spans

* tidy up

* typos

* refactor

* update flag name to signal lost trace - `is_valid_trace`

* update flag name to signal lost trace - `is_valid_trace`

* update docs

* update docs

* Use tracing Field recording to store the actual `name` and `target`
from wasm traces.

* fix debug log in subscriber + small refactor

* add tests

* handle misuse in case trying to exit span not held

* Implement filter for wasm traces, simplify field recording for primitive types

* remove superfluous warning

* update docs

* Update primitives/tracing/src/proxy.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* update docs, apply suggestions

* move Proxy from thread_local to `Extension`, rename macro

* fix test

* unify native & wasm span macro calls

* implement wasm tracing control facility in primitives and frame

* add cli flag `--wasm-tracing`

* fix

* switch to `Option<u4>` (possible performance degradation), switch
to static mut bool

* performance improvement using u64 vs Option<u64>

* performance improvement moving concat to client

* update docs

* Update client/cli/src/params/import_params.rs

Co-authored-by: Cecile Tonglet <cecile@parity.io>

* performance improvement

* Revert "performance improvement"

This reverts commit 55ff8817a86302cd93bb6197eb4ca5bc7f4fb524.

* small refactor

* formatting

* bump impl_version

* Update client/cli/src/config.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* update docs

* small fixes, remove pub static

* nit

* add integration tests and refactor Subscriber

* tests

* revert formatting

* try fix test that works locally but not in CI

* try fix test that works locally but not in CI

* debug test that works locally but not in CI

* fix test that works locally but not in CI

* remove pub visibility from bool in runtime

* make TracingSpanGuard #[cfg(not(feature = "std"))], update docs, comments

* make TracingProxy drop implementation conditional on !empty state

* add docs for TraceHandler

* remove blank line

* update expect message

* update tests

* rename cli option to tracing_enable_wasm

* rename cli option to tracing_enable_wasm

* fix

* ensure wasm-tracing features are wasm only

* bump impl_version

* bump impl_version

* add `"pallet-scheduler/std"` to `[features]` `std` in node/runtime

* refactor service to remove sp_tracing dependency

* refactor: line width, trait bounds

* improve LogTraceHandler output

* fix test

* improve tracing log output

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* swap wasm indication from trace name to a separate value

* Update client/tracing/src/lib.rs

* add docs

* remove runtime features

remove wasm_tracing option from CLI

remove wasm_tracing flag from ProfilingSubscriber

Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Cecile Tonglet <cecile@parity.io>
2020-06-18 09:44:03 +02:00
Bernhard Schuster 95d73994c1 offchain storage lock (#6010)
* feat/offchain/storage: add remove interface method

* feat/offchain/storeage: add remote to StorageValueRef

* feat/offchain/storage: add storage lock

* fix/review: Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* refactor/offchain/storage/lock: introduce `Lockable` trait part 1 of 2

* chore/offchain/rename: _remove -> clean

* feat/offchain/storage/lock: add TimeAndBlock based part 2 of 2

* fix/offchain/storage/lock: block and time expiry must be && not ||

* chore/offchain/storage: minor fmt doc comments

* doc/comment: prefer markdown emphasis over CAPS

* doc/comment: rewrap multiline module level docs

* doc/comment: rephrase

* impl sleep_until and use the actual time for the test env

* feat/test: add more tests, ignore some sample impl doctests

* fix/review: Apply suggestions from code review

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* doc/comment: better description

* fix/review: Apply suggestions from code review

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* chore/storage: lifetime cleanup

* fix/cleanup: trait bounds, cargo-spellcheck + extra explanations

* fix/doc: periods +-

* fix/review: Apply suggestions from code review

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* cleanup: remove explicit lifetime bound, copy -> clone

* fix/review: make trait Lockable contain only static, try_lock should not return Err(Option<L>),

* chore/lifetimes: remove a couple of lifetime bounds which the compiler can figure out

* refactor: migrate to an instant based

* fix/feedback: fix, reduce, rename, docs update pending

* docs/reword: adjust to changed code

* fix/offchain/testing: timestamp and sleep_until shall not block

* chore/lines: lines must < 100 chars

* fix/docs: add missing pub field doc comments

* refactor/x: try_lock does not need to return an Option<_>

* refactor/simplify: a better way of waiting for a lock to resolve

* docs: consistency

* fix/line: < 100

* fix/doctest/use: avoid crate::

* fix/doctest: *

* fix/review: remove unused trait bound

* fix/review: pretty by const fn

* fix/review: reduce default timeout to 20s

* docs: grammar

* fix/review: add with_block_deadline

* doc: revamp BlockNumberProvider documentation to be less frame centric

* chore: fmt

* docs: add missing doc comment

Co-authored-by: Bernhard Schuster <bernhard@parity.io>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
2020-05-28 14:26:11 +02:00
satellitex ab9ff537cd Add Keccak hasher (#6101)
* fix keccak hasher

* Update hasher.rs

Co-authored-by: Gavin Wood <i@gavwood.com>
2020-05-21 19:32:23 +02:00
satellitex 302c543b49 Added RuntimePublic for ecdsa public key. (#6029)
* add generate ecdsa, etc to keystore

* impl ecdsa needed traits

* add ecdsa to sr_io

* add ecdsa to application-crypto

* add ecdsa to test-utils

* add ecdsa debug derive

* fix ecdsa public
2020-05-15 15:04:38 +00:00
Benjamin Kampmann be8c96adec Relax substrate licensing scheme (#5947) 2020-05-15 13:21:56 +02:00
Bastian Köcher bc9707b4c5 Use storage::append in the implementation of the storage types (#5889)
* Start improving `storage_append`

* Fix some stuff

* Fix compilation

* Update docs and add new test

* More tests

* Test kill + append
2020-05-05 15:09:07 +02:00
Nikolay Volf c09bb1f350 Use mem::take instead of mem::replace where appropriate (#5846)
* replace replace with take

* Update client/api/src/cht.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/api/src/cht.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-30 12:53:36 +00:00
Nikolay Volf a9261cf0ef use background task to batch for optimum size (#5832) 2020-04-30 12:22:52 +02:00
Nikolay Volf 986246a102 Replace Condvars with blocking channel (#5815)
* remove condvars

* return false on lost sender

* fix

* Update primitives/io/src/batch_verifier.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-28 17:54:52 +02:00
Arkadiy Paronyan bf61cece1c Fixed race in batch signature verification (#5805) 2020-04-28 00:10:21 +02:00
Nikolay Volf 33d00692d8 Add append api and use it to deposit events (#5795)
* use append api to deposit events

* use optimized append

* one more optimization

* fix bug

* fix issues

* address review
2020-04-27 18:06:59 +02:00
Bernhard Schuster 72ee7d5797 feat/ocw/bookkeeping (#5200)
Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
2020-04-24 16:46:19 +02:00
Bastian Köcher 46677555ac Allow missing functions when checking the new runtime's version (#5741) 2020-04-23 14:54:52 +02:00
Nikolay Volf 1a41b88430 Full block import benchmark (#5745) 2020-04-23 12:55:57 +02:00
cheme 4ffcf98d8d Child trie api changes BREAKING (#4857)
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
2020-04-20 15:21:22 +02:00
Nikolay Volf 372f8b2c7e Batch signature verification (#5023)
* create parallel tasks extension

* make type system happy

* basic externalities

* test for dynamic extensions

* batching test

* remove premature verify_batch

* shnschnorrkel batch

* alter test

* shnschnorrkel test

* executive batching

* some docs

* also multi/any signatgures

* error propagation

* styling

* make verification extension optional

* experimental ed25519 parallelization

* some merge fallout

* utilize task executor

* merge fallout

* utilize task executor more

* another merge fallout

* feature-gate sp-io

* arrange toml

* fix no-std

* sr25519 batching and refactoring

* add docs

* fix name

* add newline

* fix block import test

* long sr25519 test

* blocking instead of parking

* move everything in crypto

* return batch_verify to check :)

* use condvars

* use multi-threaded executor for benches

* don't call via host interface

* try no spawning

* add true

* cleanup

* straighten batching

* remove signature check from this test (?)

* remove now pointless test

* remove another now useless test

* fix warnings

* Revert "remove another now useless test"

This reverts commit bbdec24bb67ed4373072daef7c863e1a8825bd8b.

* rethink the sp-io-part

* Revert "remove now pointless test"

This reverts commit 4d553066322e65782264caa6053d4cd5538df977.

* fix wording

* add  wording

* add todo and fix

* return check and fix

* add logging in sp-io

* Update primitives/io/src/batch_verifier.rs

Co-Authored-By: cheme <emericchevalier.pro@gmail.com>

* address review and use std condvar

* account for early exit

* address reivew

* address review

* more suggestions

* add docs for batch verification

* remove unused

* more review suggestions

* move to sp-runtime

* add expects

* remove blocks

* use entry

* Update primitives/io/src/batch_verifier.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/externalities/src/extensions.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* update overlooked note

* remove stupid return

* Update primitives/io/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/io/src/lib.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* fix wording

* bump spec_version

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-04-16 21:40:04 +02:00
Rakan Alhneiti e17a23e907 Introduce sign_with method in keystore (#4925)
* Add KEY_KIND_ID to the public trait

This change is being introduced for the purpose of identifying a public
key with it's identifier and algorithm "kind".

* Use `sign_with` as implemented in BareCryptoStore

* Implement `sign_with` in sc_keystore

* Fix inconsistencies, use *_KIND_ID in sp_core testing

* Rename KeyKindId to CryptoTypeId

* Remove pair-returning functions from BareCryptoStore trait

* Define CryptoTypeId in app-crypto macros

* Add functions to get keys supported by keystore

* Fix sign_with signature to include CryptoTypePublicPair

* Add `sign_with_any` and `sign_with_all`

* Use keystore.sign_with in auth_discovery

* Rename get_supported_keys -> supported_keys

* Added headers to function docstrings

* Use chain instead of extending a temp vector

* Fixed some code formatting

* Restrict size of CryptoTypeId

This is to be able to use Encode/Decode derives and the overcome having
the size being unknown at compile-time.

* Implement sign_with in the trait itself

* Remove whitespace

* Use key_type also as a CryptoTypeId in app_crypto macros

* Rename `get_keys` to `keys` in BareCryptoStore

* Remove usage of key_pair funcs in tests

* Adjust docstring for *_CYPTO_ID constants

* Fix failures

* Simplify mapping on keys

* Remove one let

* Fixed typo

* PR feedback

* remove whitespace

* Zip keys and signatures

* Use into_iter & remove cloned

* Pass index to MissingSignature

* Use typed errors instead of strings for BareCryptoStore

* Implement Debug for trait error

* Use hashsets for better performance for supported_keys

* Make sure keys are inserted into the keystore

* Make sign_with_all return type consistent with `sign_with`

* Rename Error to BareCryptoStoreError

* Rename CRYPT_TYPE_ID -> CRYPTO_ID

* Remove unnecessary CRYPTO_ID declaration in Public trait

* Convert pub key to CryptoTypePublicPair

* Fix use

* Fix code style

* Implement From on CryptoTypePublicPair in app_crypto macros

* Change CryptoTypePublicPair to a struct

* Implement Display on CryptoTypePublicPair

* Pass CryptoTypePublicPair to MissingSignature error

* Adjust docs according to function signature

* Unify keys implementation

* Fix RPC author tests

* Fix stackoverflow

* Tabify spaces

* Pass KeyTypeId to error for easier debugging

* Fix asserts

* Use ToHex to format public key

* Use constants from sp_core

* Rename testing KeyTypeId constants

* Please compiler

* Restore KeyTypeId names

apparently, they're not only used in tests

* Use BareCryptoStoreError instead of String

* Document return value

* Fix borrow check

* Convert to hashset internally

* WIP - iter_keys

* Return raw_public_keys

* Address PR feedback

* Address PR Feedback

* Fix hexdisplay import error

* Update primitives/core/src/traits.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-30 13:18:59 +02:00
Bastian Köcher fb3386a5f3 Clarify when changes_root returns None (#5428) 2020-03-27 12:02:37 +01:00
Nikolay Volf 27fe4206ed Update schnorrkel with versioning (#5358)
* use versioning for deprecated api, remove deprecated api from regular verification

* Update primitives/core/src/sr25519.rs

* add test to transaction pool

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-03-24 18:05:57 +01:00
Bastian Köcher 1cfcf5cbfe Don't include :code by default in storage proofs (#5179)
* Don't include `:code` by default in storage proofs (#5060)

* Adds test to verify that the runtime currently is always contained in
the proof

* Start passing the runtime wasm code from the outside

* Fix compilation

* More build fixes

* Make the test work as expected now :)

* Last fixes

* Fixes benchmarks

* Review feedback

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Review feedback

* Fix compilation

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>

* Fix compilation and change the way `RuntimeCode` works

* Fix tests

* Switch to `Cow`

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
2020-03-10 11:13:20 +01:00
Benjamin Kampmann 5a33228ea9 Reduce usage of Blake2Hasher (#5132)
This reduces the usage of `Blake2Hasher` in the code base and replaces
it with `BlakeTwo256`. The most important change is the removal of the
custom extern function for `Blake2Hasher`. The runtime `Hash` trait is
now also simplified and directly requires that the implementing type
implements `Hashable`.
2020-03-05 08:51:03 +01:00
Bastian Köcher e5123166d4 Remove deprecated host functions (#5038)
Sadly we need to keep one function `ext_blake2_256`. This function is
manually defined in `sp-core`.
2020-02-27 13:38:55 +01:00
Bastian Köcher 4dd4655f21 Update the documentation for secp256k1_ecdsa_recover* (#5019) 2020-02-21 14:14:02 +01:00
Bastian Köcher b4ebd41c21 Consolidate frame benchmarking into a frame crate (#4977)
This prs cleans up some of the frame benchmarking stuff:
- Move CLI into `frame-benchmarking-cli`. No frame related CLI should
exists in the default Substrate CLI.
- Move all traits and types related to frame benchmarking into the
`frame-benchmarking` trait. Frame types should be isolated in Frame.
2020-02-19 10:22:36 +01:00
Shawn Tabrizi c7a14db792 Fix various spelling errors (#4940)
* meaningfull -> meaningful

* initialise -> initialize

* tokans -> tokens

* incentivise -> incentivize

* lenght -> length

* incentivisation -> incentivization

* doesnt't -> doesn't

* overwriten -> overwritten

* lifecycle -> life cycle

* lifecycle -> life cycle

* usefull -> useful

* noone -> no one

* spaming -> spamming

* defered -> deferred

* hieght -> height

* sumation -> summation

* ingore -> ignore

* registed -> registered

* Auxialary -> Auxiliary

* loggin -> logging

* independance -> independence

* trailling -> trailing

* responsability -> responsibility

* trunkated -> truncated

* Weither -> Whether

* informations -> information

* Runtume -> Runtime

* choosen -> chosen

* delcataion -> declaration

* Unchekced -> Unchecked

* defintion -> definition

* scrach -> scratch

* imput -> input

* transfered -> transferred

* endownment -> endowment

* Determinator -> Determiner

* relevent -> relevant

* emited -> emitted

* acocunt -> account

* proprotional -> proportional

* instantiaion -> instantiation

* commited -> committed

* tombstonedead -> tombstone

* uwnrap -> unwrap

* acount -> account

* specialised -> specialized

* existant -> existent

* requried -> required

* Anull -> Annul

* AUTHORITES -> AUTHORITIES

* underyling -> underlying

* recognisable -> recognizable

* Capitalise -> Capitalize

* reportfor -> report for

* hearbeat -> heartbeat

* onlineness -> being online

* creater -> creator

* Bytearray -> Byte array

* Despoit -> Deposit

* substratced -> subtracted

* Curent -> Current

* imbalanes -> imbalances

* countfown -> countdown

* inexisting -> inexistent

* additionaly -> additionally

* substracted -> subtracted

* auxilary -> auxiliary

* parital -> partial

* in't -> isn't

* compatability -> compatibility

* infomation -> information

* etected -> detected

* extrinsiscs -> extrinsics

* reprensentation -> representation

* coonfiguration -> configuration

* primtives -> primitives

* miscelanious -> miscellaneous

* VERISON -> VERSION

* endcoded -> encoded

* Genrates -> Generates

* miliseconds -> milliseconds

* occured -> occurred

* trully -> truely

* truely -> truly

* conjuction -> conjunction

* encouters -> encounters

* customised -> customized

* deterministicly -> deterministically

* finalisation -> finalization

* pluggable -> plugable

* wakeup -> wake-up

* interemdiate -> intermediate

* intepreting -> interpreting

* finalzied -> finalized

* throgh -> through

* extinsic -> extrinsic

* convient -> convenient

* allocater -> allocator

* propagateable -> propagatable

* succesfuly -> successfully

* finalising -> finalizing

* publically -> publicly

* phrasee -> phrase

* substration -> substractions

* substractions -> subtractions

* neccessarily -> necessarily

* Inlucde -> Include

* unefficient -> inefficient

* thay -> they

* funtion -> function

* datastructures -> data structures

* infromation -> information

* propagatable -> propagable

* ecountered -> encountered

* recognise -> recognize

* intergration -> integration

* lastet -> latest

* datatypes -> data types

* datatype -> data type

* Strongarming -> Strong Arming

* avaible -> available

* Commiting -> Committing

* Retreiving -> Retrieving

* shoud -> should

* canonicaliziation -> canonicalization

* comitted -> committed

* clonable -> cloneable

* Uknown -> Unknown

* reponse -> response

* arbitary -> arbitrary

* Capapbilities -> Capabilities

* responsbile -> responsible

* initialisation -> initialization

* cames -> came

* intemediate -> intermediate

* reqeust -> request

* intance -> instance

* explcitly -> explicitly

* neighor -> neighbor

* reolving -> resolving

* untill -> until

* Validte -> Validate

* deserailize -> deserialize

* literaly -> literally

* preceeding -> preceding

* abpve -> above

* chcecked -> checked

* numbet -> number

* Unknow -> Unknown

* halfs -> halves

* gossup -> gossip

* givent -> given

* immediatelly -> immediately

* slicable -> sliceable

* conensus -> consensus

* Mimicks -> Mimics

* acccept -> accept

* serialise -> serialize

* exstrinsics -> extrinsics

* panicks -> panics

* maintaince -> maintenance

* repeatidely -> repeatedly

* anecstor -> ancestor

* becasue -> because

* processer -> processor

* Prunning -> Pruning

* insterested -> interested

* unuseful -> not useful

* yeided -> yielded

* descendfing -> descending

* corresponts -> corresponds

* survivew -> survive

* keps -> keeps

* ligh -> light

* prerequisities -> prerequisites

* positiion -> position

* depedency -> dependency

* extrinisic -> extrinsic

* atomicaly -> atomically

* staticly -> statically

* resul -> result

* timestamb -> timestamp

* Utilites -> Utilities

* ammount -> amount

* pocess -> process

* exteral -> external

* Update client/finality-grandpa/src/tests.rs

* Update primitives/io/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update primitives/blockchain/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update frame/support/src/weights.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update bin/node/cli/tests/common.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/cli/src/params.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/executor/common/src/sandbox.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/finality-grandpa/src/communication/mod.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/state-db/src/pruning.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update frame/contracts/src/tests.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Update client/api/src/execution_extensions.rs

* bump impl

* timestamb -> timestamp

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
2020-02-17 15:07:24 +01:00
Bastian Köcher 4254fbf56d Don't expose Benchmarking host functions by default (#4875)
* Don't expose `Benchmarking` host functions by default

* Fix tests

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-02-10 13:13:04 +01:00
Shawn Tabrizi e5a7fcc8ea Create Benchmarking Setup for Identity Pallet #4695 (#4818)
* Starting

* closer

* Compiles!

* comments

* Create seperate mock

* Remove changes to test env

* Fix step calculation

* Add host function

* Add runtime api

* compiles

* Update to use offchain timestamp

* Gives a result

* added some CLI wip

* make generic

* Update instance

* Remove CLI stuff

* Remove last cli stuff

* undo more changes

* Update benchmarks

* Update Cargo.lock

* remove test

* Move loop out of runtime

* Benchmarking externalities

* Benchmarking state

* Implemented commit

* Make CLI work, move loop back into runtime

* Wipe resets to genesis

* Speedup benchmarks

* Use enum to select extrinsic within pallet

* CLI controls which module and extrinsic to call

* Select a pallet with cli

* Add steps and repeats to cli

* Output as CSV format

* Introduce benchmark pallet

* Append bench

* Use Results

* fix merge

* Clear Identity benchmark

* Bench request judgment and cancel request

* Add final benchmarks

* Fix CSV output

* Start cleaning up for PR

* Bump numbers in `wasmtime` integration tests.

* More docs

* Add rockdb feature to bench

* Fix formatting issues

* Add test feature to bench

* Add test feature to bench

* Add rocksdb feature flag

* Update bench.rs

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
2020-02-10 10:23:08 +01:00
Sergei Pepyakin 1185f3ea30 Get rid of in-substrate usages of core_intrinsics feature (#4823)
* Remove usage of unneeded Rust feature core_intrinsics

* core::intrinsics::abort -> arch::wasm32::unreachable

* Don't publish `core::intrinsics`.

* Disable panic_handler and alloc_error_handler for no_std non wasm builds
2020-02-05 17:26:48 +01:00
Bastian Köcher 4c36143375 Add get_global for Sandbox (#4756)
* Add `get_global` for `Sandbox`

This pr adds `get_global` to retrieve a `global` variable from an
instantiated sandbox wasm blob.

* Bump `spec_version`

* Update primitives/wasm-interface/src/lib.rs

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* `get_global` -> `get_global_val`

Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
2020-01-29 16:24:40 +01:00
Bastian Köcher afc3318f21 Implement runtime version checks in set_code (#4548)
* Implement runtime version checks in `set_code`

Check that the new runtime code given to `set_code` fullfills some
requirements:

- `spec_name` matches
- `spec_version` does not decreases
- `impl_version` does not decreases
- Either `spec_version` and `impl_version` increase

* Make tests almost work

* Some fixes after master merge

* Fix tests

* Add missed file

* Make depedency check happy?

* Remove leftover `sc-executor`

* AHHHHH

* Reset debug stuff

* Remove some 'static

* More 'static

* Some docs

* Update `Cargo.lock`
2020-01-16 13:58:37 +01:00
Igor Matuszewski 7b17deda24 Remove now unused lang_items feature in IO primitives (#4596)
We don't use any explicit `#[lang = ...]` items anymore (`panic_fmt`
is replaced with now-stable `#[panic_implementation]`, OOM is handled
via `alloc_error_handler` directly etc.), so I thought it'd be good to
remove it for clarity to limit used nightly features.
2020-01-10 18:35:23 +01:00
Bastian Köcher fd6b29dd2c Remove requirement on Hash = H256, make Proposer return StorageChanges and Proof (#3860)
* Extend `Proposer` to optionally generate a proof of the proposal

* Something

* Refactor sr-api to not depend on client anymore

* Fix benches

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Apply suggestions from code review

* Introduce new `into_storage_changes` function

* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code

* Put the `StorageChanges` into the `Proposal`

* Move the runtime api error to its own trait

* Adds `StorageTransactionCache` to the runtime api

This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(

* Remove `type NodeBlock` and switch to a "better" hack

* Start using the transaction cache from the runtime api

* Make it compile

* Move `InMemory` to its own file

* Make all tests work again

* Return block, storage_changes and proof from Blockbuilder::bake()

* Make sure that we use/set `storage_changes` when possible

* Add test

* Fix deadlock

* Remove accidentally added folders

* Introduce `RecordProof` as argument type to be more explicit

* Update client/src/client.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update primitives/state-machine/src/ext.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Integrates review feedback

* Remove `unsafe` usage

* Update client/block-builder/src/lib.rs

Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>

* Update client/src/call_executor.rs

* Bump versions

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
2020-01-10 10:48:32 +01:00