Commit Graph

122 Commits

Author SHA1 Message Date
Tomasz Drwięga e2f5e40876 Initial: Offchain Workers (#1942)
* Refactor state-machine stuff.

* Fix tests.

* WiP

* WiP2

* Service support for offchain workers.

* Service support for offchain workers.

* Testing offchain worker.

* Initial version working.

* Pass side effects in call.

* Pass OffchainExt in context.

* Submit extrinsics to the pool.

* Support inherents.

* Insert to inherents pool.

* Inserting to the pool asynchronously.

* Add test to offchain worker.

* Implement convenience syntax for modules.

* Dispatching offchain worker through executive.

* Fix offchain test.

* Remove offchain worker from timestamp.

* Update Cargo.lock.

* Address review comments.

* Use latest patch version for futures.

* Add CLI parameter for offchain worker.

* Fix compilation.

* Fix test.

* Fix extrinsics format for tests.

* Fix RPC test.

* Bump spec version.

* Fix executive.

* Fix support macro.

* Address grumbles.

* Bump runtime
2019-03-25 23:22:11 +01:00
Arkadiy Paronyan 704d2a49c7 Don't sync on top of pruned blocks (#2079)
* Don't sync to pruned blocks

* Typo

Co-Authored-By: arkpar <arkady.paronyan@gmail.com>
2019-03-22 16:27:50 +01:00
Robert Habermeier e83a39fee3 remove aura inherent-checking compatibility shim (#2044) 2019-03-19 18:13:23 +01:00
Gav Wood d7fcf5dc9d Overhaul crypto (Schnorr/Ristretto, HDKD, BIP39) (#1795)
* Rijig to Ristretto

* Rebuild wasm

* adds compatibility test with the wasm module

* Add Ed25519-BIP39 support

* Bump subkey version

* Update CLI output

* New keys.

* Standard phrase/password/path keys.

* Subkey uses S-URI for secrets

* Move everything to use new HDKD crypto.

* Test fixes

* Ignore old test vector.

* fix the ^^ old test vector.

* Fix tests

* Test fixes

* Cleanups

* Fix broken key conversion logic in grandpa

CC @rphmeier

* Remove legacy Keyring usage

* Traitify `Pair`

* Replace Ed25519AuthorityId with ed25519::Public

* Expunge Ed25519AuthorityId type!

* Replace Sr25519AuthorityId with sr25519::Public

* Remove dodgy crypto type-punning conversions

* Fix some tests

* Avoid trait

* Deduplicate DeriveJunction string decode

* Remove cruft code

* Fix test

* Minor removals

* Build fix

* Subkey supports sign and verify

* Inspect works for public key URIs

* Remove more crypto type-punning

* Fix typo

* Fix tests
2019-03-13 14:08:31 +01:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Robert Habermeier dfb48a2405 Offline fallback for GRANDPA (#1619)
Co-authored-by: André Silva <andre.beat@gmail.com>

* skeleton for finality tracker

* dispatch events when nothing finalized for a long time

* begin integrating finality tracker into grandpa

* add delay field to pending change

* add has_api_with function to sr_version for querying APIs

* partially integrate new force changes into grandpa

* implement forced changes

* get srml-grandpa compiling

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update core/finality-grandpa/src/authorities.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* remove explicit dependence on CoreApi

* increase node runtime version

* integrate grandpa forced changes into node runtime

* add some tests to finality-tracker

* integrate finality tracking into node-runtime

* test forced-change logic

* test forced changes in the authority-set handler

* kill some unneeded bounds in client

* test forced-changes in finality-grandpa and fix logic

* build wasm and finality-tracker is no-std

* restart voter on forced change

* allow returning custom error type from lock_import_and_run

* extract out most DB logic to aux_schema and use atomic client ops

* unify authority set writing

* implement set pausing

* bump runtime version

* note on DB when we pause.

* core: grandpa: integrate forced changes with multiple pending standard changes

* core: grandpa: fix AuthoritySet tests

* runtime: bump impl_version

* core: clear pending justification requests after forced change import

* srml: finality-tracker: use FinalizedInherentData

* core: log requests for clearing justification requests

* core, node: update runtimes

* core: grandpa: fix tests

* core: grandpa: remove todos and add comments

* core: grandpa: use has_api_with from ApiExt

* core: fix tests

* core: grandpa: remove unnecessary mut modifier

* core: replace PostImportActions bitflags with struct

* core: grandpa: restrict genesis on forced authority set change

* core: grandpa: add more docs

* core: grandpa: prevent safety violations in Environment::finalize_block

* core: grandpa: register finality tracker inherent data provider

* core: grandpa: fix tests

* node: update runtime blobs

* core: grandpa: remove outdated todo

* core: aura: fix typo in log message

* core: grandpa: check re-finalization is on canonical chain

* srml: finality-tracker: fix initialization

* node: update runtime wasm

* srml: finality-tracker: don't re-initialize config keys
2019-03-05 15:41:35 +00:00
Hero Bird 128d164f2b Make use of parity-codec "derive" crate feature (#1919)
* Make use of parity-codec "derive" crate feature

* Bump impl_version

* Update wasm files and Cargo.lock files

* Transferred to parity_codec derive feature for the rest of the crates

* Update wasm binaries and lock files

* Update core and node wasm binaries
2019-03-05 09:00:37 +01:00
Michael Müller c485eb8fdc Get rid of wildcard imports (#1916)
* Get rid of wildcard imports

* Update core/service/src/lib.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Update core/client/src/client.rs

Co-Authored-By: cmichi <mich@elmueller.net>

* Import by namespace
2019-03-04 14:00:46 +01:00
Michael Müller 90e5c5ddfb Telemetry improvements (#1886)
* Fix typo

* Support multiple telemetry endpoints and verbosity levels

* Bump substrate-telemetry version

* Telemetrify Aura consensus

* Telemetrify Grandpa

* Fix CI version conflicts

* Implement style remarks

* Fix fixture

* Implement style remarks

* Clone only when necessary

* Get rid of Arc for URL

* Handle connection issues better
2019-02-28 12:22:05 +01:00
Robert Habermeier 64685c0536 Prune leaves on finality (#1871)
* support multiple leaves at same height in leaves set

* prune leaves on finalization

* test leaves behavior in client-db

* fix no-std compilation
2019-02-26 15:50:55 -05:00
Marcio Diaz 43e4544884 Client provide uncles (#1609)
* feat: add children function to backend

* feat: add test for children hashes

* feat: add uncles function to client

* fix: improve uncles function adds few more tests

* fix: remove children when reverting

* fix: typo and spec version
2019-02-25 11:21:36 +01:00
Tomasz Drwięga 63ec8b335b Refactor state machine to avoid repetitions (#1844)
* Refactor state-machine stuff.

* Fix tests.

* Fix core/client tests.
2019-02-22 13:49:36 +01:00
Svyatoslav Nikolsky 6a6c3155a6 Use DB backend in test client (#1779)
* use DB backend in test client

* Update core/client/db/src/lib.rs

Co-Authored-By: svyatonik <svyatonik@gmail.com>
2019-02-15 11:03:22 +01:00
Stanislav Tkach ff5e4ca87e Migrate everything to the 2018 edition (#1758) 2019-02-13 11:45:59 +01:00
Tomasz Drwięga 9e2710246f EnumerableStorageMap (#1763)
* Refactor decl_storage a bit to allow easier impl of linked map.

* A bunch of refactorings for storage generation.

- Rename StorageMap and ChildrenStorageMap to avoid confusion with generator::StorageMap.
- Separate implementation from the procedural macro code to clean it up.
- Make sure that genesis is initialised using the `StorageValue/StorageMap`
  generated implementations instead of going RAW.

* WiP: Writing test.

* Basic implementation.

* Implement enumeration.

* Fix non-std issues.

* fix warning

* Fix test-client.

* Address review grumbles - part 1

* Avoid cloning the key, relax Storage requirements.

* Rebuild runtime.

* Remove dangling todo.
2019-02-13 08:52:52 +01:00
Wei Tang 22048dba60 Add standalone set head operations (#1600)
* Refactor head setting operation to a separate function

* Fix commit issue and implement set_head standalone in db

* Add standalone set head operations

* Address grumbles

* Change number_and_hash_to_lookup key in light mod to take reference

* Fix bug in set head commit

* Add a convenience fn

* Fix a deadlock

* Fix missing argument
2019-02-12 17:21:25 +01:00
Bastian Köcher 1f05a47cdb Set initialised_block if we called initialise_block (#1767) 2019-02-12 14:52:00 +01:00
Marcio Diaz b8bd49961a Adds new execution strategy nativeElseWasm (#1546)
* fix: adds new execution strategy nativeElseWasm and replace nativeWhenPossible with it

* feat: adds cmd line params for execution strategies

* fix: uses of cmd line execution strategies

* chore: remove white spaces

* chore: remove println

* chore: remove whitespace

* fix: generating functions with context

* feat: add function to generate with_context declarations

* fix: add implementation for with_context function calls

* fix: add execution context to call_api_at function

* fix: making use of context to select strategy for block_builder

* chore: cleaning up

* fix: merging issues

* fix tests

* add wasm files

* chore: small doc for context fields

* chore: delete redundant docs

* fix: use full path for ExecutionContext

* fix: add context functions from inside fold_item_impl

* chore: remove clone

* fix: moving generative function to utils, remove unused imports

* fix: add missing full path for ExecutionContext

* fix: merge issues

* update wasm files

* fix: update to keep up with changes in master

* chore: remove unused functions, clean up

* fix test

* fix grumbles

* fix: add more tests

* fix: some refactorings

* feat: add execution strategy to call

* chore: small improvements

* fix: add message to panic

* fix tests
2019-02-11 15:22:44 +01:00
Stanislav Tkach 31188dc013 Migrate rpc-servers, serializer, service and sr-io to the 2018 edition (#1732) 2019-02-08 13:10:05 +01:00
Arkadiy Paronyan 1dc15be5bd Check for block parent before verification. (#1714)
* Treat verification errors more seriously

* Track obsolete requests

* Check block parent before verification

* Style
2019-02-07 11:58:27 +01:00
Bastian Köcher 190393d476 Make API backwards compatible with CC (#1697)
* Rework how a runtime api calls into the runtime

Now we generate a default implementation for each api call that calls
a generated method `method_runtime_api_impl`. This newly generated
method is the one that will be implemented by the `impl_runtime_apis`
macro in the runtime for the client side.

* Support `changed_in` to change runtime api function signatures

* Update documentation

* Fixes tests

* Implement checking the api version with a predicate

* Make the implementation backwards compatible with CC

* Update wasm files after merge

* Check for wasm runtime differences by building master and current branch

* Update spec_version and wasm files

* Fixes

* Revert my changes

* Remove `patch.crates-io` from test-runtime
2019-02-06 11:47:47 +01:00
Stanislav Tkach df685c9e53 Rename and use --max-heap-pages parameter (#1666) 2019-02-05 12:35:12 +01:00
Bastian Köcher ef4dc12a5d Don't panic if parameter can not be converted between node and native runtime (#1659)
* Don't panic if parameter can not be converted between node and native runtime

* FIxes after merge

* Use correct copyright year
2019-02-02 14:13:50 +01:00
Tomasz Drwięga ecdd33a367 Allow both consensus and runtime to limit block building (#1581)
* Limit block size in runtime,

* Add test for basic authorship.

* Store length of extrinsics instead of computing it.

* Don't rely on note_extrinsic

* Use hashed version of storage and write test.

* Recompile runtime.
2019-02-01 13:58:23 +01:00
Svyatoslav Nikolsky 381cf26f55 Make genesis state locally-available on light client (#1622)
* make genesis state available on light client

* RemoteOrLocalCallExecutor

* code_is_executed_locally_or_remotely

* OnDemandOrGenesisState tests

* some comments
2019-02-01 10:27:54 +01:00
Svyatoslav Nikolsky a4756710c5 fixed aux modifications on light client (#1621) 2019-01-30 14:44:29 +01:00
Benjamin Kampmann 15ae7cfef6 Document more TODOs as tickets (#1418)
Went through the TODOs, removed a bunch, which are outdated or nothing more than a regular comment, documented a bunch more as actual tickets and made them FIXMEs and unified their structure (`FIXME #TICKETNO DESC` for local tickets, `FIXME: DESC LINK` for external tickets) for easier in-editor support. Further more remove unnecessary remarks and related old code that I noticed in that instance.
2019-01-30 10:29:48 +01:00
Robert Habermeier 0953521cc4 don't panic in best_containing (#1559)
* don't panic in best_containing

* core: client: fix borrow in best_containing
2019-01-25 12:16:37 +00:00
Stanislav Tkach 82d014cf35 Remove TODO comment about unboxing state_machine error 2019-01-24 19:10:07 +02:00
Arkadiy Paronyan f38aa0d2de Test syncing to all forks (#1544) 2019-01-24 15:52:48 +00:00
Wei Tang 997c8b4020 Atomic operation and locking when importing a block (#1489)
* Add ClientImportOperation and remove an unused enum

* set_aux to insert_aux so that it can be called multiple times

* [WIP] All basic designs of lock_import_and_run

* [WIP] `apply_block` and `apply_aux` implementation

* Update client db with the new interface

* Always make sure we reset importing_block back to None

* Address grumbles

`apply_block` should be pub

* Add comments on insert_aux

* Fix compile
2019-01-24 15:51:53 +00:00
Bastian Köcher 5be237030d Fix warnings in substrate (#1523)
* Fix warnings in substrate

* More warnings removed and wasm updated

* Fixes error on stable and grumbles

* Update wasm files

* Add links to the github issue for replacing `error-chain`
2019-01-24 16:10:07 +01:00
thiolliere c0568ba3c6 update primitive-types and solve break changes (#1535) 2019-01-23 18:26:44 +01:00
Wei Tang dd88dc6cd6 Split BlockImport and JustificationImport (#1521)
* Split BlockImport and JustificationImport

* Remove unused trait impl

* Fix compile

* Fix grandpa tests

* Fix network tests
2019-01-23 12:22:08 -03:00
joe petrowski fe8fab030c Split execute_and_import_block in client into two functions (#1455)
* type error

* borrow problem

* compiles but run error

* tests pass ready for review

* tests pass ready for review

* fix comments by rob

* removed clone

* style fix

* updating

* resolved conflicts

* fixed comments
2019-01-23 15:27:35 +01:00
Bastian Köcher 70b1af7b1e Rewrite Inherent data (#1488)
* Implement new inherent data

* Fixes compilation on wasm

* Fixes after rebase

* Switch back to generate inherent stuff by macro

* Update after rebase

* Apply suggestions from code review

Co-Authored-By: bkchr <bkchr@users.noreply.github.com>

* Fix compilation after rebase

* Address grumbles

* Remove `InherentDataProviders` from `Client`

* Update wasm files after rebase

* Address grumbles

* Fixes compilation after latest merge

* Last fix
2019-01-22 17:52:08 +01:00
Bastian Köcher 010e63116f Make runtime api calls native when possible (#1302)
* Add simple benchmark for the runtime api

* Make the executor support native calls

* Some documentation

* Hide behind `feature = "std"`

* Rework the native calls

* Make all tests compile again

* Make every parameter using the Block serialized/deserialized in the native call

* Forward `UnwindSafe` requirement

* Remove debug stuff

* Add some documentation

* Fixes warnings

* Fixes errors after master rebase

* Fixes compilation after master rebase

* Fixes compilation after rebase
2019-01-21 14:32:53 +01:00
André Silva 399cee310a Sync block justifications (#1410)
* core: sync protocol for justifications

* core: basic test for justification sync

* core: pass block number with justification

* grandpa: request justifications when importing change blocks

* core: pass finality notifications to chain sync

* core: require justifications for pending change blocks on start

* core: avoid requesting justifications from previous failed peers

* core: timeout block justification requests

* core: add some docs

* core: fix unused variables warning

* core: tick pending justifications fetch periodically

* grandpa: add test for syncing justifications

* core: early exit dispatch of pending justifications

* core: style fix

* core: grandpa: change logging level

* core: sync: add missing docs

* core: network: report peer on bad justification

* core: replace mem::replace with Option::take

* core: revert authority set changes on failed block finalization

* core: grandpa: add docs to import_justification

* core: warn on re-finalization of last finalized block

* core: only notify sync with last finality notification

* core: style fix

* core: add docs for PendingJustifications

* core: network: use BlockRequest messages for justification requests

* core: reference issues in todo comments

* core: grandpa: revert authority set changes on db

* core: grandpa: remove inconsistent state warning
2019-01-21 07:04:01 +01:00
Svyatoslav Nikolsky da1fb3f273 Use changes tries in query_storage RPC (#1082)
* use changes tries in query_storage RPC

* let + match + return + call -> match
2019-01-17 10:08:50 +01:00
Marcio Diaz 9151349b2d Make use of NLL in client crate (#1436)
* fix: set edition to 2018 in Cargo.toml.

* fix: refactor function to make use of NLL.

* fix: result of applying 'cargo fix --edition' command.

* fix: removes extern crate

* fix: remove module uses from lib.rs

* fix: tests imports
2019-01-16 11:49:26 +01:00
Svyatoslav Nikolsky 616716cb4b GRANDPA finality proof draft (#1268)
* grandpa finality proof

* prove GrandpaApi::grandpa_authorities using parent block + some docs

* create justification when consensus data is changed

* generate justifications periodically

* test for ConsensusChanges
2019-01-11 19:25:03 +01:00
Xiliang Chen 0e569e6bf2 new state api state_getKeys to expose storage keys (#1380) 2019-01-11 12:51:19 +01:00
Arkadiy Paronyan b104c02eb6 State cache and other performance optimizations (#1345)
* State caching

* Better code caching

* Execution optimizaton

* More optimizations

* Updated wasmi

* Caching test

* Style

* Style

* Reverted some minor changes

* Style and typos

* Style and typos

* Removed panics on missing memory
2019-01-08 13:13:13 +01:00
Wei Tang 71d889b692 Make AuthorityId generic (#1296)
* BlockAuthorityId convenience type

* Rename AuthorityId -> Ed25519AuthorityId to make it more precise

* Generalize AuthorityId up to substrate-client

* Fix in client-db

* rename: BlockAuthorityId -> AuthorityIdFor

* typo: should be digest item

* Fix test-runtime authorityId mismatch

One states that AuthorityId is u64 while the other states that it's Ed25519AuthorityId.

* Fix more u64 - Ed25519AuthorityId mismatch

* Fix compile of most of the srml modules

* Continue to pin aura and grandpa with ed25519 and fix compile

* Add MaybeHash trait

* Fix node-runtime compile

* Fix network tests
2019-01-08 11:14:18 +01:00
Wei Tang ac1be0665e Support custom fork choice rule (#1339)
* Support custom fork choice rule

* Remove unneeded reexport

* Fix network compile
2019-01-04 20:34:58 +01:00
Gregory Terzian 3add75910a move import queue to consensus-common (#1282) 2018-12-31 12:33:21 +01:00
Arkadiy Paronyan 831fd9947f Make sure displaced leaves are removed from the db (#1338) 2018-12-29 14:08:20 +01:00
Gav Wood a6c77e4f82 Compact extrinsics (#1314) 2018-12-21 12:55:44 +01:00
Svyatoslav Nikolsky 81f90ed0a4 AuxStore on light clients (#1251)
* implement AuxStore on light clients

* Update core/client/db/src/light.rs

Co-Authored-By: svyatonik <svyatonik@gmail.com>

* Update core/client/db/src/light.rs

Co-Authored-By: svyatonik <svyatonik@gmail.com>

* Update core/client/db/src/light.rs

Co-Authored-By: svyatonik <svyatonik@gmail.com>
2018-12-13 17:01:57 +01:00
Svyatoslav Nikolsky c1b0fba486 do not compute storage root (+ changes root) when proving execution (#1246) 2018-12-12 10:35:37 +01:00