Commit Graph

24 Commits

Author SHA1 Message Date
Bastian Köcher 8a39be474e Move Externalities into its own crate (#3775)
* Move `Externalities` into `substrate-externalities`

- `Externalities` now support generic extensions
- Split of `primtives-storage` for storage primitive types

* Move the externalities scoping into `substrate-externalities`

* Fix compilation

* Review feedback

* Adds macro for declaring extensions

* Fix benchmarks

* Introduce `ExtensionStore` trait

* Last review comments

* Implement it for `ExtensionStore`
2019-10-09 15:50:30 +02:00
Bastian Köcher 9607afd629 Cleanup of the state-machine crate (#3524)
* Start refactoring state-machine crate

* More improvement to state-machine

* Fix tests compilation on master and remove warnings

* Fix compilation

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/state-machine/src/basic.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Line width

* Update core/primitives/src/storage.rs

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Update core/state-machine/src/error.rs

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Review feedback
2019-09-10 17:00:00 +02:00
cheme e3d0c60a31 Implement change trie for child trie. (#3122)
* Initial implementation, some redundancy is awkward and there is
some useless computation (but there is a pending pr for that).
Next are tests.

* Minimal tests and fix extend child.

* implement iterator for change child trie.

* prune child trie.

* Fix pruning test.

* bump spec version.

* Avoid empty child trie (could also be checked before)

* tabs.

* Fix child digest overriding each others.

* Restore doc deleted on merge.

* Check correct child value on extrinsics build.

* Revert runtime version update.
2019-09-02 12:10:42 +03:00
cheme b0e1212d48 Child storage tests and genesis fix. (#3185)
* Using child storage, (srml-support only), test failing .

* fix simple tests.

* Enumerable by requiring owned struct (previous form only allow
&'static).
Broken tests are from genesis init.

* implement for_child_keys_with_prefix

* indent

* clear_child_prefix fix.

* clear_child_prefix fix 2.

* fix for storage_impl, if/when allowing child and not child this could be
reverted.

* Fix lot of urlinked child genesis, still need to look upon actual
genesis srml module code.
Probably still a lot of broken code needing debugging.

* switch well_known_key to their associated module child trie.
Fix a genesis init (balance).
Complete some testing.
Comment some tests before using.

* fixing test runtime child keys

* latest commit fix broken genesis init

* fix system balances child name.

* Important fix: storage_root from test externalities need children (it is
already the case for ext).

* executive root with child calculation

* Avoid empty trie on test ext.

* Symetric removal of key for system.

* commenting changes related tests.

* Remove child module specifics.

* fix issues.

* fix some formatting

* fix bench and bump runtime

* Remove extend_storage_overlays, assimilate_storage do the same as is
proper considering srml macro.

* Fix warning for assimilate.

* Removing kill as they do not impact any test cases.

* Use tuple of storage map instead of two parameters. This changes the
behavior of decl_storage genesis build closure (breaking api).

* Do not use build storage before assimilate.

* fix error

* Update core/state-machine/src/backend.rs
2019-08-08 15:05:25 +02:00
Gavin Wood 1a524b8207 Refactor key management (#3296)
* Add Call type to extensible transactions.

Cleanup some naming

* Merge Resource and BlockExhausted into just Exhausted

* Fix

* Another fix

* Call

* Some fixes

* Fix srml tests.

* Fix all tests.

* Refactor crypto so each application of it has its own type.

* Introduce new AuthorityProvider API into Aura

This will eventually allow for dynamic determination of authority
keys and avoid having to set them directly on CLI.

* Introduce authority determinator for Babe.

Experiment with modular consensus API.

* Work in progress to introduce KeyTypeId and avoid polluting API
with validator IDs

* Finish up drafting imonline

* Rework offchain workers API.

* Rework API implementation.

* Make it compile for wasm, simplify app_crypto.

* Fix compilation of im-online.

* Fix compilation of im-online.

* Fix more compilation errors.

* Make it compile.

* Fixing tests.

* Rewrite `keystore`

* Fix session tests

* Bring back `TryFrom`'s'

* Fix `srml-grandpa`

* Fix `srml-aura`

* Fix consensus babe

* More fixes

* Make service generate keys from dev_seed

* Build fixes

* Remove offchain tests

* More fixes and cleanups

* Fixes finality grandpa

* Fix `consensus-aura`

* Fix cli

* Fix `node-cli`

* Fix chain_spec builder

* Fix doc tests

* Add authority getter for grandpa.

* Test fix

* Fixes

* Make keystore accessible from the runtime

* Move app crypto to its own crate

* Update `Cargo.lock`

* Make the crypto stuff usable from the runtime

* Adds some runtime crypto tests

* Use last finalized block for grandpa authority

* Fix warning

* Adds `SessionKeys` runtime api

* Remove `FinalityPair` and `ConsensusPair`

* Minor governance tweaks to get it inline with docs.

* Make the governance be up to date with the docs.

* Build fixes.

* Generate the inital session keys

* Failing keystore is a hard error

* Make babe work again

* Fix grandpa

* Fix tests

* Disable `keystore` in consensus critical stuff

* Build fix.

* ImOnline supports multiple authorities at once.

* Update core/application-crypto/src/ed25519.rs

* Merge branch 'master' into gav-in-progress

* Remove unneeded code for now.

* Some `session` testing

* Support querying the public keys

* Cleanup offchain

* Remove warnings

* More cleanup

* Apply suggestions from code review

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* More cleanups

* JSONRPC API for setting keys.

Also, rename traits::KeyStore* -> traits::BareCryptoStore*

* Bad merge

* Fix integration tests

* Fix test build

* Test fix

* Fixes

* Warnings

* Another warning

* Bump version.
2019-08-07 20:47:48 +02:00
thiolliere 4ed67e03a4 Update to parity-scale-codec (#3232)
* WIP: update codec

* WIP

* compiling

* WIP

* rename parity-scale-codec to codec

* WIP

* fix

* remove old comments

* use published crates

* fix expected error msg

* bump version

* fmt and fix

* remove old comment

* fix wrong decoding impl

* implement encode like for structures

* undo removal of old pending changes

* trailingzeroinput

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* update codec

* fmt

* version is 1.0.0

* show more error

* fmt
2019-08-06 19:36:23 +02:00
cheme da8b91ae7b Trie simplification. (#2815)
* switch to simple codec, trie broken for now

* Actualy use trie_root_noext

* align some hash, failing test on EMCH comment

* Fix trie code over layout instead of hash, revert legacy code for legacy
mainnet ??

* stub behind LayOut

* fix no_std

* temp solution for legacy trie behind feature legacy-key in various crate

* use remote project

* rc client db need prefix

* update trie deps

* bum spec runtime version

* Removing legacy as default.

* Switch mode to non legacy.

* bump runtime version

* Remove legacy trie compatibility features.

* fix warning

* bump version

* change hash on new test.

* Move dependency (#11 trie PR) patched to a parity repo.
Bench reverted to correct hasher.
Some renaming and doc improvments.

* ChildBitmap renaming to BitMap.

* Renaming of LayOut to Layout.

* formatting.

* Removing abreviation such as _ix nb_ or bm.

* Update deps and apply renaming 'Buff' -> 'Buffer'.

* Align to latest trie crates naming changes.

* Update trie dependency.

* Update trie dependency.

* change block_import test hash

* update trie deps (trie use new scale codec but it does not seems to be
an issue).

* update to use latest trie version (no mgmt of multiple radix).

* tabify

* Restoring test to 10 000.

* Use published crate, trie bench is currently down until publishing
(require another pr to update version).

* Update trie-bench.
2019-08-02 19:51:59 +02:00
Svyatoslav Nikolsky 95061beb79 Some changes tries optimizations (#2840)
* changes tries initial optimizations

* line width
2019-07-16 10:25:08 +09:00
cheme e777038418 Add storage cache for child trie and notification internals (#2639)
* child cache, and test failing notifications

* fix tests and no listen child on top wildcard

* remove useless method

* bump impl version

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* factoring notification methods to remove some redundant code.

* test child sub removal

* HStorage implementation and some type alias.

* Remove HStorage cache: does not fit

* fix removal

* Make cache use byte length (shared) instead of number of kv

* Make use of hashes cache in rpc

* applying ratio on different lru caches

* Fix format

* break a line

* Remove per element overhead of lru cache.

* typo
2019-06-14 11:25:20 +02:00
Svyatoslav Nikolsky 549d9e1da1 Remove BlockNumber <-> u64 conversions from light-client related code (#2666)
* Remove As usage from CHT

* Remove As usage from CHT (continue)

* Restrict BN <-> int conversions in CT

* more BN <-> u64 conversions removed

* upd spec_version

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <github@gavwood.com>

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <github@gavwood.com>

* more grumbles

* fix last grumbles + compilation

* too long lines

* too long lines
2019-05-28 15:07:16 +02:00
cheme 0d8379d5d2 Child trie storage proof (#2433)
* proof on child trie

* higher level api for child storage proof

* boilerplate for proof from light fetch

* actually check proof on light fetch

* Do not break former encoding

* tabify

* tabify2

* Add child trie root tx to full_storage_root transaction.

* Shorten long lines.

* Temp rename for audit

* Make full_storage a trait method

* Name back and replace some code with full_storage where it looks fine.

* fix indentations, remove unused import

* flush child root to top when calculated

* impl +1
2019-05-10 14:31:41 +02:00
Stanislav Tkach 0f02bed702 Update hex-literal version, simplify imports and remove unused dependencies (#2371) 2019-04-25 11:17:40 +02:00
Tomasz Drwięga e2f5e40876 Initial: Offchain Workers (#1942)
* Refactor state-machine stuff.

* Fix tests.

* WiP

* WiP2

* Service support for offchain workers.

* Service support for offchain workers.

* Testing offchain worker.

* Initial version working.

* Pass side effects in call.

* Pass OffchainExt in context.

* Submit extrinsics to the pool.

* Support inherents.

* Insert to inherents pool.

* Inserting to the pool asynchronously.

* Add test to offchain worker.

* Implement convenience syntax for modules.

* Dispatching offchain worker through executive.

* Fix offchain test.

* Remove offchain worker from timestamp.

* Update Cargo.lock.

* Address review comments.

* Use latest patch version for futures.

* Add CLI parameter for offchain worker.

* Fix compilation.

* Fix test.

* Fix extrinsics format for tests.

* Fix RPC test.

* Bump spec version.

* Fix executive.

* Fix support macro.

* Address grumbles.

* Bump runtime
2019-03-25 23:22:11 +01:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Stanislav Tkach f98f9ac58a Migrate state-db, state-machine, telemetry, test-client and test-runtime to the 2018 edition (#1623) 2019-01-30 15:51:59 +01:00
Arkadiy Paronyan b104c02eb6 State cache and other performance optimizations (#1345)
* State caching

* Better code caching

* Execution optimizaton

* More optimizations

* Updated wasmi

* Caching test

* Style

* Style

* Reverted some minor changes

* Style and typos

* Style and typos

* Removed panics on missing memory
2019-01-08 13:13:13 +01:00
Svyatoslav Nikolsky 742cb33d90 Make CallApiAt::call_api_at work at light client (#1213)
* light client contextual method call

* fixed off-by-one in test

* newlines + commas
2018-12-10 16:02:00 +03:00
Hero Bird 660c747c51 Update fixed-hash & uint utilities (#1078)
* [core/primitives] Update crunchy 0.1 => 0.2

* [core/primitives] Update fixed-hash 0.2 => 0.3.0-beta

* [core/primitives] Update `uint` 0.4 => 0.5.0-beta

* [core/state-machine] Fix errors emerged by fixed-hash update

* [core/state-db] Fix errors that emerged from fixed-hash update

* [core/sr-io] Fix errors that emerged from the fixed-hash update

* [core/trie] Fix errors emerged from fixed-hash updates

* [core/trie] Make use of new Hash::as_fixed_bytes_mut method in tests

* [core/sr-primitives] Fix errors emerged from updating fixed-hash

* [core/executor] Fix errors that emerged from fixed-hash update

* [core/test-runtime] Fix errors that emerged from updating fixed-hash

* [core/test-runtime] Fix an error that emerged from fixed-hash update

* [core/transaction-pool] Fix errors that emerged from updating fixed-hash

* Add From<u64> impl for hash types defined in core/primitives

* [core/client/db] Fix errors that emerged from fixed-hash update

* [core/{client/network/rpc}]: Fix errors emerged by fixed-hash update

* [node/{cli/executor/runtime}]: Fix errors emerged by updating fixed-hash

* [core/network]: Fix bug in parsing constant str

* Update Cargo.lock

- Add crunchy 0.2.1
- fixed-hash 0.2.2 => 0.3.0-beta.3
- Add static-assertions 0.2.5
- uint 0.4.1 => 0.5.0-beta.1

* [core/primitives]: Add fixed-hash/rustc-hex feature for FromStr impl

* [core/primitives] No longer provide From<u64> impl for hash types if byteorder support is not enabled

* [core/primitives] Revert to using From impl again in primitives tests

* [core/..] Fix some bugs that emerged by recent fixed-hash updates

* Update a bunch of Cargo.lock files

* [core/state-db] Re-add whitespace between attr and extern crate/mod

* [core/primitives] Fix bug in From<u64> impl for hash types using the wrong feature guard
2018-11-06 15:39:55 +01:00
Wei Tang 2604474880 Multiple storage root support (#902)
* Implement a non generic version of child delta trie

* Use delta_trie_root in state_machine

* Expand InMemory backend to support multi-storage

* Create Consolidate trait

* Fix all crate compile and remove unused OverlayedChanges::drain

* Implement child storage root support and overlay changes

* Add child storage reader

* Add child storage writer

* Implement child storage cleaning

* Fix light backend compile

* Add all required ext functions for wasm executor

* Add ext def to io

* Add all io functions

* Fix nostd compile

* Add simple test

* Remove unnecessary vec copy in child_storage_root_transaction

* Use values_mut/for_each to make it shorter

* Use extend to shorter a for loop

* Move record_all_keys to trie so it's easier to generic them later

* space -> tab

* Remove to_owned in debug format

* Clean out all to_owned

* Break debug_trace to multiple lines

* Remove 0..

* UserError copy/paste typo

* Replace Vec::from_raw_parts by slice::from_raw_parts

* Use iter::empty()

* Wrap some long lines

* Wrap a missing line

* Remove unnecessary map

https://github.com/paritytech/substrate/pull/856#discussion_r226222663

* Call ext_free after from_raw_parts

* Fix tests in other crates
2018-10-18 18:54:02 +02:00
Bastian Köcher 12f12bff28 Make all copyrights use the correct year (#894)
* Adds a script for automatically update all copyrights

* Ran the `update-copyright` script
2018-10-10 10:28:41 +02:00
Svyatoslav Nikolsky c54350661d Fetching changes proof from remote nodes (#769)
* changes_trie

* changs_trie: continue

* changes_trie: adding tests

* fixed TODO

* removed obsolete ExtrinsicChanges

* encodable ChangesTrieConfiguration

* removed polkadot fle

* fixed grumbles

* ext_storage_changes_root returns u32

* moved changes trie root to digest

* removed commented code

* read storage values from native code

* fixed grumbles

* fixed grumbles

* missing comma

* key changes proof generation + query

* fix grumbles

* check that changes trie config is not changed by block.finalize()

* fixed changes trie config check
2018-09-29 09:47:29 +01:00
Gav Wood 82d6ca3484 Switch to shiny new fast, RLP-less trie (#795)
* Bump codec

* Fix tests

* Patricia trie builds

* Introduce trie

* Some yak shaving.

* Some fixes

* Remove RLP ref

* Fixes

* It builds!

* Some tests fixed

* Another test fix

* Rejig more hashes

* substrate-trie::iterator_works test

* Update lock

* Polish

* Docs

* Undo incorrect "fix" for tests

* Fix nits
2018-09-25 16:32:22 +02:00
Sergey Pepyakin e7d1933d25 Extract well known keys into a single place (#764)
* Extract well known keys into a single place

* Fixes.
2018-09-18 16:20:36 +02:00
Svyatoslav Nikolsky 7fa337afbc Light friendly storage tracking: changes trie + extending over ranges (#628)
* changes_trie

* changs_trie: continue

* changes_trie: adding tests

* fixed TODO

* removed obsolete ExtrinsicChanges

* encodable ChangesTrieConfiguration

* removed polkadot fle

* fixed grumbles

* ext_storage_changes_root returns u32

* moved changes trie root to digest

* removed commented code

* read storage values from native code

* fixed grumbles

* fixed grumbles

* missing comma
2018-09-18 09:14:41 +02:00