Commit Graph

28 Commits

Author SHA1 Message Date
Pierre Krieger bf2551a854 Switch the client to new futures (#3103)
* Switch the client to new futures

* No need for compat in the client

* Fix client tests

* Address review
2019-07-11 16:58:30 +02:00
cheme e777038418 Add storage cache for child trie and notification internals (#2639)
* child cache, and test failing notifications

* fix tests and no listen child on top wildcard

* remove useless method

* bump impl version

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* factoring notification methods to remove some redundant code.

* test child sub removal

* HStorage implementation and some type alias.

* Remove HStorage cache: does not fit

* fix removal

* Make cache use byte length (shared) instead of number of kv

* Make use of hashes cache in rpc

* applying ratio on different lru caches

* Fix format

* break a line

* Remove per element overhead of lru cache.

* typo
2019-06-14 11:25:20 +02:00
Svyatoslav Nikolsky 22586113ea Light GRANDPA import handler (#1669)
* GrandpaLightBlockImport

* extract authorities in AuraVerifier

* post-merge fix

* restore authorities cache

* license

* new finality proof draft

* generalized PendingJustifications

* finality proof messages

* fixed compilation

* pass verifier to import_finality_proof

* do not fetch remote proof from light import directly

* FinalityProofProvider

* fixed authorities cache test

* restored finality proof tests

* finality_proof docs

* use DB backend in test client

* justification_is_fetched_by_light_client_when_consensus_data_changes

* restore justification_is_fetched_by_light_client_when_consensus_data_changes

* some more tests

* added authorities-related TODO

* removed unneeded clear_finality_proof_requests field

* truncated some long lines

* more granular light import tests

* only provide finality proof if it is generated by the requested set

* post-merge fix

* finality_proof_is_none_if_first_justification_is_generated_by_unknown_set

* make light+grandpa test rely on finality proofs (instead of simple justifications)

* empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* missing trait method impl

* fixed proof-of-finality docs

* one more doc fix

* fix docs

* initialize authorities cache (post-merge fix)

* fixed cache initialization (post-merge fix)

* post-fix merge: fix light + GRANDPA tests (bad way)

* proper fix of empty_finality_proof_is_returned_to_light_client_when_authority_set_is_different

* fixed easy grumbles

* import finality proofs in BlockImportWorker thread

* allow import of finality proofs for non-requested blocks

* limit number of fragments in finality proof

* GRANDPA post-merge fix

* BABE: pos-merge fix
2019-05-13 11:36:52 +02:00
Benjamin Kampmann 18ca0170c3 Allow for customisation of chain selection systems (#2240)
* move SelectChain trait out of client

* Extend SelectChain, move longest chain implementation into it

* Bring SelectChain into service

* implement LongestChain SelectChain

* implement longest chain for node

* update Cargo.lock's

* in between erroring tests

* deprecate ::backend and ::import_lock

* Remove unneded space

Co-Authored-By: gnunicorn <ben.kampmann@googlemail.com>

* Remove unneded space

Co-Authored-By: gnunicorn <ben.kampmann@googlemail.com>

* Fixes test compilation

* remove todo

* re-enable client test

* add doc

* fixing tests

* Clarify SelectChain Interface, intended implementation and usage

* minor components cleanups

* minor cleanups

* Update lock files

* Implement cleaner interface for SelectChain

* addressing comments

* Updating tests

* bump node runtime impl version

* address grumbles
2019-05-10 14:08:12 +02:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Robert Habermeier 64685c0536 Prune leaves on finality (#1871)
* support multiple leaves at same height in leaves set

* prune leaves on finalization

* test leaves behavior in client-db

* fix no-std compilation
2019-02-26 15:50:55 -05:00
Marcio Diaz 43e4544884 Client provide uncles (#1609)
* feat: add children function to backend

* feat: add test for children hashes

* feat: add uncles function to client

* fix: improve uncles function adds few more tests

* fix: remove children when reverting

* fix: typo and spec version
2019-02-25 11:21:36 +01:00
Stanislav Tkach ff5e4ca87e Migrate everything to the 2018 edition (#1758) 2019-02-13 11:45:59 +01:00
Marcio Diaz b8bd49961a Adds new execution strategy nativeElseWasm (#1546)
* fix: adds new execution strategy nativeElseWasm and replace nativeWhenPossible with it

* feat: adds cmd line params for execution strategies

* fix: uses of cmd line execution strategies

* chore: remove white spaces

* chore: remove println

* chore: remove whitespace

* fix: generating functions with context

* feat: add function to generate with_context declarations

* fix: add implementation for with_context function calls

* fix: add execution context to call_api_at function

* fix: making use of context to select strategy for block_builder

* chore: cleaning up

* fix: merging issues

* fix tests

* add wasm files

* chore: small doc for context fields

* chore: delete redundant docs

* fix: use full path for ExecutionContext

* fix: add context functions from inside fold_item_impl

* chore: remove clone

* fix: moving generative function to utils, remove unused imports

* fix: add missing full path for ExecutionContext

* fix: merge issues

* update wasm files

* fix: update to keep up with changes in master

* chore: remove unused functions, clean up

* fix test

* fix grumbles

* fix: add more tests

* fix: some refactorings

* feat: add execution strategy to call

* chore: small improvements

* fix: add message to panic

* fix tests
2019-02-11 15:22:44 +01:00
Bastian Köcher 190393d476 Make API backwards compatible with CC (#1697)
* Rework how a runtime api calls into the runtime

Now we generate a default implementation for each api call that calls
a generated method `method_runtime_api_impl`. This newly generated
method is the one that will be implemented by the `impl_runtime_apis`
macro in the runtime for the client side.

* Support `changed_in` to change runtime api function signatures

* Update documentation

* Fixes tests

* Implement checking the api version with a predicate

* Make the implementation backwards compatible with CC

* Update wasm files after merge

* Check for wasm runtime differences by building master and current branch

* Update spec_version and wasm files

* Fixes

* Revert my changes

* Remove `patch.crates-io` from test-runtime
2019-02-06 11:47:47 +01:00
Marcio Diaz 9151349b2d Make use of NLL in client crate (#1436)
* fix: set edition to 2018 in Cargo.toml.

* fix: refactor function to make use of NLL.

* fix: result of applying 'cargo fix --edition' command.

* fix: removes extern crate

* fix: remove module uses from lib.rs

* fix: tests imports
2019-01-16 11:49:26 +01:00
Arkadiy Paronyan b104c02eb6 State cache and other performance optimizations (#1345)
* State caching

* Better code caching

* Execution optimizaton

* More optimizations

* Updated wasmi

* Caching test

* Style

* Style

* Reverted some minor changes

* Style and typos

* Style and typos

* Removed panics on missing memory
2019-01-08 13:13:13 +01:00
Bastian Köcher ed421c56ee Rewrite impl_runtime_apis! and decl_runtime_apis! as proc-macro (#1174)
* Rewrites `impl_runtime_apis!` macro as `proc-macro`

* Adds some documentation

* Require the `impl_runtime_apis` to use a path for accessing the trait

* Make the runtime implement `GetNodeBlockType`

* Moves first chunk of runtime api code into the `impl_runtime_apis` macro

This also renames `ClientWithApi` into `RuntimeApi`.

* Make `impl_runtime_apis` use `runtime` api version automatically

* `decl_runtime_apis` automatically adds `Block: BlockT` as generic parameter

* Remove function generic arguments in block builder api

* Remove some unnused stuff from the `decl_runtime_apis` macro

* Make `InherentData` working again

* Make `impl_runtime_apis!` implement the `RuntimeApi` side as well

* Make it compile again after rebasing with master

* Split `sr-api-macros` into multiple files

* Reimplement `decl_runtime_apis!` as proc_macro

* Use `decl_runtime_apis!` for `Core` as well and improve error reporting

* Adds documentation for `decl_runtime_apis!` and `impl_runtime_apis!`

* Move some code

* Adds compile fail tests

* Adds a test and fixes some bugs

* Make `impl_runtime_apis!` support `_` as parameter name

* Fixes build errors with wasm

* Wasm rebuild after master rebase

* Apply suggestions from code review

Co-Authored-By: bkchr <bkchr@users.noreply.github.com>

* Addresses some grumbles

* Adds test to ensure that method signatures need to match

* New wasm files
2018-11-30 11:42:46 +01:00
Robert Habermeier 59af4de4fd GRANDPA: add commit messages (#1151)
* get compiling with latest version of grandpa

* generalize UntilImported to prepare for waiting for commit message targets

* extract until_imported out to own module

* logic for blocking commits until enough blocks imported

* add tests for commit message blocking logic

* pass through commit mesage round number as well

* extract communication streams to own module

* add Error implementation for ExitOrError

* introduce stream adapter for checking commit messages

* output sink for commits

* implement the unimplemented

* remove extra line

* update to latest version of grandpa api

* update finality-grandpa to 0.4.0

* Use filter_map earlier when checking incoming commits messages

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* address some grumbles
2018-11-26 16:24:26 +01:00
Bastian Köcher 9063d1acae Implement runtime api client side directly in the runtime (#1094)
* Move `initialise_block` into `Core` trait as it is crucial calling the API functions

* Switch to first version of new runtime API implementation

* Fixes bug in tests

* Reenable asserts

* Directly use the `TestAPI` in the tests

* Start improving the api traits

:100644 100644 898aadc7 49217199 M	Cargo.lock
:100644 100644 61570436 465ed664 M	core/client/src/backend.rs
:100644 100644 5d0c886b 64d710fd M	core/client/src/block_builder.rs
:100644 100644 c447855e 5ecbe474 M	core/client/src/client.rs
:100644 100644 139cef13 f90dbf3d M	core/client/src/error.rs
:100644 100644 2800c503 3298e66a M	core/client/src/runtime_api.rs
:100644 100644 affa1c5c 809b08bc M	core/primitives/src/lib.rs
:100644 100644 2877dfa9 d5547413 M	core/sr-api/Cargo.toml
:100644 100644 9a49784d 6a625a03 M	core/sr-api/src/lib.rs
:100644 100644 7c28e1c7 a1a444a9 M	core/sr-primitives/src/traits.rs
:100644 100644 2e113ab6 dcc01a6d M	srml/metadata/Cargo.toml
:100644 100644 ea722a70 0809531a M	srml/metadata/src/lib.rs

* Refactoring

* Move `sr-api` into client and more refactoring

* Fixes tests

* Some documentation and cleanup

* Fixes compilation after rebase

* More refactoring and more documentation

* Makes `substrate-client` compilable on `wasm`

On `wasm` it basically just exports the runtime api stuff.

* Fixes grumbles

* Updates wasm files after rebasing the master

* Remove TODO comment

* Remove whitespaces

* Fixes after rebasing master

* Another rebase, another fix commit
2018-11-13 13:33:28 +01:00
Luke Schoen 037f9dde10 docs: Extends #999. Remove disused package readmes. Rename to sr-metadata. Fix Readme Rustdoc names (#1016)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme

* docs: Remove link to old docs

* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names

* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory

* Change example to use package name that opens all packages in RustDocs

* Fix list of packages names that are available to open in RustDocs

* fix typo

* fix ordering of CLI options and add missing backslash
2018-11-12 21:18:06 +01:00
Benjamin Kampmann 50adea6220 Consensus Engines Implementation: Aura (#911)
* Generalize BlockImport

 - move ImportBlock, BlockOrigin, ImportResult into shared sr-primitives
 - let Consensus provide  and  traits again
 - update consensus traits to latest development
 - implement traits on client::Client, test_client::TestClient
 - update RHD to use the new import_block API

* Move ImportBlock into consensus-common
* Send import notification in aura tests
* Integrating aura into service
* Make Signatures more generic
* Aura Block Production with the given key
* run aura on the thread pool
* start at exact step start in aura
* Add needed wasm blob, in leiu of better solutions.
* Make API ids consistent with traits and bring upstream for sharing.
* Add decrease_free_balance to Balances module
* Encode `Metadata` once instead of two times
* Bitops include xor
* Upgrade key module.
* Default pages to somewhat bigger.
* Introduce upgrade key into node
* Add `Created` event
2018-10-27 15:59:18 +02:00
Luke Schoen f923e38cc7 docs: Relates to Issue #995. Fix links in existing documentation (#999)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core
2018-10-26 11:09:48 +02:00
Benjamin Kampmann ac4bcf879f Generalize the Consensus Infrastructure (#883)
* Split out Consensus
* Supply ImportQueue through network-service
  - simplify ImportQueue.import_blocks
  - remove Deadlock on import_block
  - Adding Verifier-Trait
  - Implement import_queue provisioning in service; allow cli to import
* Allow to actually customize import queue
* Consensus Gossip: Cache Message hash per Topic
2018-10-16 13:40:33 +02:00
Bastian Köcher 12f12bff28 Make all copyrights use the correct year (#894)
* Adds a script for automatically update all copyrights

* Ran the `update-copyright` script
2018-10-10 10:28:41 +02:00
Bastian Köcher 2c65ad6c7b Refactor the runtime API to use traits. (#878)
* Add missing `As` imports.

* Adds new API traits that will be used by the client and runtime

* Switch consensus to new API's

* Switches transaction-pool to new API's

* Move runtime api stuff into its own crate

* Adds `impl_apis!` macro for implementing the new API traits

* Make `metadata` return directly a blob

* Runtime replace `impl_stubs!` with `impl_apis!`

* Switches to none feature based approach for declaring the different API traits

* Fixes compilation error

* Fixes errors

* Make the `decl_apis!` trait usable from the outside

* Make the `test-client` use the new API traits

* Remove last `impl_stubs!` bits and move some of them into wasm executor for tests

* A little bit more documentation
2018-10-09 10:58:29 +02:00
Bastian Köcher 2332d6dd04 More dependency cleanup (#825)
* Removes `rlp` dependency

* Enable warnings about unused crates in the CI

* Remove all the unused crates

* Make tests working again
2018-09-27 17:52:51 +01:00
snd 58cc0992df best_containing operations (issue 603) (#740)
* add stub for Client.best_chain_containing_block_hash

* add fn blockchain::Backend::leaf_hashes

* fix typo

* sketch out Client.best_chain_containing_block_hash

* fix indent

* Blockchain.leaf_hashes -> Blockchain.leaves

* add unimplemented! stub impls for Blockchain.leaves

* start impl of Blockchain.leaves for in-memory client db

* Client.best_chain_containing...: check canonical first and make compile

* first rough attempt at maintaining leaf list in in-memory db

* fix tab indent

* add test best_chain_containing_single_block

* add failing test best_chain_containing_with_fork

* pub use client::blockchain; in test-client to prevent circular dep in client tests

* best_chain_containing_with_single_block: improve and test leaves

* far improve in-memory Backend::leaves impl

* test blockchain::Backend::leaves more thoroughly

* handle more edge cases in blockchain::Backend::leaves impl for in memory

* fix test best_chain_containing_with_fork (two distinct test blocks had same hash)

* make best_chain_containing_block_hash pass existing tests

* improve docstring for Blockchain::leaves

* Client.best_chain_containing: some cleanup. support max_block_number

* best_chain_containing: remove broken outcommented fast check for best = canonical

* remove blank line

* best_block_containing: return None if target_hash not found

* best_chain_containing: add unreachable! at end of function

* improve tests for best_chain_containing

* renames

* more elaborate test scenario for best_containing

* best_containing: fix restriction of search through maybe_max_number

* best_containing: tests for restriction of search

* get rid of unnecessary clones

* replace Client::new_in_mem by new_with_backend which is useful for testing backends

* add test_client::new_with_backend for testing different backend impls

* add test for in_mem::Backend::leaves

* remove unused imports

* in_mem test_leaves: simplify

* flesh out tests for in_mem leaves impl

* remove tests for leaves from client which are now covered in implementing module

* improve comment

* add Client.new_in_mem again

* unwrap in test_client::new_with_backend

* make test_client::BlockBuilderExt work not just with in-mem backend

* make test client ext not just work with in mem backend

* add failing Backend.leaves test for client-db

* update Cargo.lock

* replace KeccakHasher with Blake2Hasher

* refactor

address grumble https://github.com/paritytech/substrate/pull/740#discussion_r217822862

* refactor using NumberFor

address grumble https://github.com/paritytech/substrate/pull/740#discussion_r217823341

* add test that exposes possible problem

* update docstring for Client.best_containing

* extract test for Backend.leaves for reuse

* improve test blockchain_header_and_hash_return_blocks_from_canonical_chain_given_block_numbers

* extract test_blockchain_query_by_number_gets_canonical to easily test multiple impls

* remove whitespace

* remove todo

* Client.best_containing: pre-empt search loop when target in canonical

* best_containing: prevent race condition by holding import lock

* add todo

* extract leaf list update code into function

* add comment

* client-db: use in-memory-kvdb for tests

* use BTreeSet to store leaves for in-mem which is faster and simpler

* add docstring

* add comments and fix formatting

* add initial raw version of LeafSet

* remove Client::update_leaves which has been superceded by LeafSet

* use LeafSet in in-mem backend

* keccak -> blake2

* don't reexport codec traits in primitives

addresses https://github.com/paritytech/substrate/pull/740#discussion_r219538185

* fix rebase mistake

* improve LeafSet and use it in state-db

* correct Transfer nonces to fix ApplyExtinsicFailed(Stale)

* use given backend in canoncal test

* kill dead tree-route code in util

* fix warnings

* tests for leafset

* reorganizations in in_mem backend

* fix reorganization canon block logic

* DB commit and safe reversion on write error

* fix style nits
2018-09-26 12:34:05 +01:00
Gav Wood 82d6ca3484 Switch to shiny new fast, RLP-less trie (#795)
* Bump codec

* Fix tests

* Patricia trie builds

* Introduce trie

* Some yak shaving.

* Some fixes

* Remove RLP ref

* Fixes

* It builds!

* Some tests fixed

* Another test fix

* Rejig more hashes

* substrate-trie::iterator_works test

* Update lock

* Polish

* Docs

* Undo incorrect "fix" for tests

* Fix nits
2018-09-25 16:32:22 +02:00
Robert Habermeier b02c274374 Finality notification streams (#791)
* finalization for in_mem

* fetch last finalized block

* pruning: use canonical term instead of final

* finalize blocks in full node

* begin to port light client DB

* add tree-route

* keep number index consistent in full nodes

* fix tests

* disable cache and finish porting light client

* add AsMut to system module

* final leaf is always best

* fix all tests

* Fix comment and trace

* removed unused Into call

* add comment on behavior of `finalize_block`

* move `tree_route` to client common

* tree_route tests

* return slices in TreeRoute

* apply finality up to parent

* add `finalize_block` call

* adjust formatting

* finality notifications and add last finalized block to chain info

* exhaustive match and comments

* fix sync tests by using non-instant finality
2018-09-24 18:45:37 +02:00
Bastian Köcher 5d64be26c3 Replace JSON metadata with custom metadata structures (#766)
* Move module metadata from json string to custom metadata

* Revisit the metadata structures

1. Move the structures into the metadata crate.
2. Switch to using Cow/MaybeOwnedArray to support Encode/Decode

* Adapt to new metadata structures

* Convert event json metadata to new metadata structures

* Convert storage json metadata to new metadata structures

* Convert runtime metadata from json to new metadata structs

* Implements new metadata structures in client and runtime

* Fixes errors on `no_std`

* Fixes errors after rebasing master

* Do not use `Cow` anymore in metadata

Also replace `String` with our own type definition `StringBuf`.
This fixes compilation on `no_std`.

* Wrap `RuntimeMetadata` in `RuntimeMetadataVersioned` to support versioning

* Move metadata into `srml` and make core unaware of the implementation
2018-09-20 07:35:32 +02:00
Svyatoslav Nikolsky 7fa337afbc Light friendly storage tracking: changes trie + extending over ranges (#628)
* changes_trie

* changs_trie: continue

* changes_trie: adding tests

* fixed TODO

* removed obsolete ExtrinsicChanges

* encodable ChangesTrieConfiguration

* removed polkadot fle

* fixed grumbles

* ext_storage_changes_root returns u32

* moved changes trie root to digest

* removed commented code

* read storage values from native code

* fixed grumbles

* fixed grumbles

* missing comma
2018-09-18 09:14:41 +02:00
Gav Wood 1e01162505 Phase 1 of repo reorg (#719)
* Remove unneeded script

* Rename Substrate Demo -> Substrate

* Rename demo -> node

* Build wasm from last rename.

* Merge ed25519 into substrate-primitives

* Minor tweak

* Rename substrate -> core

* Move substrate-runtime-support to core/runtime/support

* Rename/move substrate-runtime-version

* Move codec up a level

* Rename substrate-codec -> parity-codec

* Move environmental up a level

* Move pwasm-* up to top, ready for removal

* Remove requirement of s-r-support from s-r-primitives

* Move core/runtime/primitives into core/runtime-primitives

* Remove s-r-support dep from s-r-version

* Remove dep of s-r-support from bft

* Remove dep of s-r-support from node/consensus

* Sever all other core deps from s-r-support

* Forgot the no_std directive

* Rename non-SRML modules to sr-* to avoid match clashes

* Move runtime/* to srml/*

* Rename substrate-runtime-* -> srml-*

* Move srml to top-level
2018-09-12 11:13:31 +02:00