Commit Graph

24 Commits

Author SHA1 Message Date
Arkadiy Paronyan a07e567fe1 Storage tracing (#3614)
* Storage tracing

* Whitepsaces

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock
2019-09-17 20:42:15 +02:00
Bastian Köcher 9607afd629 Cleanup of the state-machine crate (#3524)
* Start refactoring state-machine crate

* More improvement to state-machine

* Fix tests compilation on master and remove warnings

* Fix compilation

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/state-machine/src/basic.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Line width

* Update core/primitives/src/storage.rs

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Update core/state-machine/src/error.rs

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Review feedback
2019-09-10 17:00:00 +02:00
cheme e3d0c60a31 Implement change trie for child trie. (#3122)
* Initial implementation, some redundancy is awkward and there is
some useless computation (but there is a pending pr for that).
Next are tests.

* Minimal tests and fix extend child.

* implement iterator for change child trie.

* prune child trie.

* Fix pruning test.

* bump spec version.

* Avoid empty child trie (could also be checked before)

* tabs.

* Fix child digest overriding each others.

* Restore doc deleted on merge.

* Check correct child value on extrinsics build.

* Revert runtime version update.
2019-09-02 12:10:42 +03:00
cheme b0e1212d48 Child storage tests and genesis fix. (#3185)
* Using child storage, (srml-support only), test failing .

* fix simple tests.

* Enumerable by requiring owned struct (previous form only allow
&'static).
Broken tests are from genesis init.

* implement for_child_keys_with_prefix

* indent

* clear_child_prefix fix.

* clear_child_prefix fix 2.

* fix for storage_impl, if/when allowing child and not child this could be
reverted.

* Fix lot of urlinked child genesis, still need to look upon actual
genesis srml module code.
Probably still a lot of broken code needing debugging.

* switch well_known_key to their associated module child trie.
Fix a genesis init (balance).
Complete some testing.
Comment some tests before using.

* fixing test runtime child keys

* latest commit fix broken genesis init

* fix system balances child name.

* Important fix: storage_root from test externalities need children (it is
already the case for ext).

* executive root with child calculation

* Avoid empty trie on test ext.

* Symetric removal of key for system.

* commenting changes related tests.

* Remove child module specifics.

* fix issues.

* fix some formatting

* fix bench and bump runtime

* Remove extend_storage_overlays, assimilate_storage do the same as is
proper considering srml macro.

* Fix warning for assimilate.

* Removing kill as they do not impact any test cases.

* Use tuple of storage map instead of two parameters. This changes the
behavior of decl_storage genesis build closure (breaking api).

* Do not use build storage before assimilate.

* fix error

* Update core/state-machine/src/backend.rs
2019-08-08 15:05:25 +02:00
cheme da8b91ae7b Trie simplification. (#2815)
* switch to simple codec, trie broken for now

* Actualy use trie_root_noext

* align some hash, failing test on EMCH comment

* Fix trie code over layout instead of hash, revert legacy code for legacy
mainnet ??

* stub behind LayOut

* fix no_std

* temp solution for legacy trie behind feature legacy-key in various crate

* use remote project

* rc client db need prefix

* update trie deps

* bum spec runtime version

* Removing legacy as default.

* Switch mode to non legacy.

* bump runtime version

* Remove legacy trie compatibility features.

* fix warning

* bump version

* change hash on new test.

* Move dependency (#11 trie PR) patched to a parity repo.
Bench reverted to correct hasher.
Some renaming and doc improvments.

* ChildBitmap renaming to BitMap.

* Renaming of LayOut to Layout.

* formatting.

* Removing abreviation such as _ix nb_ or bm.

* Update deps and apply renaming 'Buff' -> 'Buffer'.

* Align to latest trie crates naming changes.

* Update trie dependency.

* Update trie dependency.

* change block_import test hash

* update trie deps (trie use new scale codec but it does not seems to be
an issue).

* update to use latest trie version (no mgmt of multiple radix).

* tabify

* Restoring test to 10 000.

* Use published crate, trie bench is currently down until publishing
(require another pr to update version).

* Update trie-bench.
2019-08-02 19:51:59 +02:00
cheme e777038418 Add storage cache for child trie and notification internals (#2639)
* child cache, and test failing notifications

* fix tests and no listen child on top wildcard

* remove useless method

* bump impl version

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update core/client/src/notifications.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* factoring notification methods to remove some redundant code.

* test child sub removal

* HStorage implementation and some type alias.

* Remove HStorage cache: does not fit

* fix removal

* Make cache use byte length (shared) instead of number of kv

* Make use of hashes cache in rpc

* applying ratio on different lru caches

* Fix format

* break a line

* Remove per element overhead of lru cache.

* typo
2019-06-14 11:25:20 +02:00
Arkadiy Paronyan 3b26453047 Pin states in memory so that they are not pruned away while still referenced (#2761)
* State pinning in client

* Canonicalization queue

* Fixed prioritization queue

* possible fix of "hash mismatch"

* Check for pinned discarded states

* Release state before finalization

* Style

* Style
2019-06-04 10:01:12 +02:00
Tomasz Drwięga 98de97e1d9 Access child storage over RPC. (#2586)
* Access child storage over RPC.

* Address review grumbles.

* Test happy case in child_storage rpc.

* Remove stray printlns

* Fix line widths.

* Bump runtime again.

* Fix genesis storage root calculation for light clients.

* Don't pass values to full_storage_root child_delta.
2019-05-19 13:02:09 -04:00
cheme 0d8379d5d2 Child trie storage proof (#2433)
* proof on child trie

* higher level api for child storage proof

* boilerplate for proof from light fetch

* actually check proof on light fetch

* Do not break former encoding

* tabify

* tabify2

* Add child trie root tx to full_storage_root transaction.

* Shorten long lines.

* Temp rename for audit

* Make full_storage a trait method

* Name back and replace some code with full_storage where it looks fine.

* fix indentations, remove unused import

* flush child root to top when calculated

* impl +1
2019-05-10 14:31:41 +02:00
Pierre Krieger 5a820db7f5 Remove the dependency on heapsize (#2459)
* Remove heapsize dependency

* Finish removing heapsize

* Revert accidental change

* Fix Cargo.lock
2019-05-04 13:59:59 +02:00
Arkadiy Paronyan 7046e13de2 Use prefixed keys for trie node. (#2130)
* Account for pending insertions when pruning

* Prefixed trie storage

* Comments

* Prefixed trie storage

* Fixed tests

* Fixed tests

* Bumped runtime version

* Bumped runtime version again
2019-03-28 18:46:21 +01:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Stanislav Tkach f98f9ac58a Migrate state-db, state-machine, telemetry, test-client and test-runtime to the 2018 edition (#1623) 2019-01-30 15:51:59 +01:00
Benjamin Kampmann 15ae7cfef6 Document more TODOs as tickets (#1418)
Went through the TODOs, removed a bunch, which are outdated or nothing more than a regular comment, documented a bunch more as actual tickets and made them FIXMEs and unified their structure (`FIXME #TICKETNO DESC` for local tickets, `FIXME: DESC LINK` for external tickets) for easier in-editor support. Further more remove unnecessary remarks and related old code that I noticed in that instance.
2019-01-30 10:29:48 +01:00
Xiliang Chen 0e569e6bf2 new state api state_getKeys to expose storage keys (#1380) 2019-01-11 12:51:19 +01:00
Arkadiy Paronyan b104c02eb6 State cache and other performance optimizations (#1345)
* State caching

* Better code caching

* Execution optimizaton

* More optimizations

* Updated wasmi

* Caching test

* Style

* Style

* Reverted some minor changes

* Style and typos

* Style and typos

* Removed panics on missing memory
2019-01-08 13:13:13 +01:00
Wei Tang b21de8a0b5 Support multi trie in genesis generation (#958)
* Support multi trie in genesis generation

* Fix merge issues
2018-11-01 09:30:03 +01:00
Wei Tang 2604474880 Multiple storage root support (#902)
* Implement a non generic version of child delta trie

* Use delta_trie_root in state_machine

* Expand InMemory backend to support multi-storage

* Create Consolidate trait

* Fix all crate compile and remove unused OverlayedChanges::drain

* Implement child storage root support and overlay changes

* Add child storage reader

* Add child storage writer

* Implement child storage cleaning

* Fix light backend compile

* Add all required ext functions for wasm executor

* Add ext def to io

* Add all io functions

* Fix nostd compile

* Add simple test

* Remove unnecessary vec copy in child_storage_root_transaction

* Use values_mut/for_each to make it shorter

* Use extend to shorter a for loop

* Move record_all_keys to trie so it's easier to generic them later

* space -> tab

* Remove to_owned in debug format

* Clean out all to_owned

* Break debug_trace to multiple lines

* Remove 0..

* UserError copy/paste typo

* Replace Vec::from_raw_parts by slice::from_raw_parts

* Use iter::empty()

* Wrap some long lines

* Wrap a missing line

* Remove unnecessary map

https://github.com/paritytech/substrate/pull/856#discussion_r226222663

* Call ext_free after from_raw_parts

* Fix tests in other crates
2018-10-18 18:54:02 +02:00
Svyatoslav Nikolsky 9886d12c26 Pruning changes tries (#856)
* changes trie pruning

* add comment

* do not prune changes tries on archive nodes
2018-10-17 10:08:45 +02:00
Bastian Köcher 12f12bff28 Make all copyrights use the correct year (#894)
* Adds a script for automatically update all copyrights

* Ran the `update-copyright` script
2018-10-10 10:28:41 +02:00
Gav Wood 82d6ca3484 Switch to shiny new fast, RLP-less trie (#795)
* Bump codec

* Fix tests

* Patricia trie builds

* Introduce trie

* Some yak shaving.

* Some fixes

* Remove RLP ref

* Fixes

* It builds!

* Some tests fixed

* Another test fix

* Rejig more hashes

* substrate-trie::iterator_works test

* Update lock

* Polish

* Docs

* Undo incorrect "fix" for tests

* Fix nits
2018-09-25 16:32:22 +02:00
Guanqun Lu 43068f8fc3 remove duplicate try_into_trie_backend (#779) 2018-09-20 06:36:24 -04:00
Svyatoslav Nikolsky 7fa337afbc Light friendly storage tracking: changes trie + extending over ranges (#628)
* changes_trie

* changs_trie: continue

* changes_trie: adding tests

* fixed TODO

* removed obsolete ExtrinsicChanges

* encodable ChangesTrieConfiguration

* removed polkadot fle

* fixed grumbles

* ext_storage_changes_root returns u32

* moved changes trie root to digest

* removed commented code

* read storage values from native code

* fixed grumbles

* fixed grumbles

* missing comma
2018-09-18 09:14:41 +02:00
Gav Wood 1e01162505 Phase 1 of repo reorg (#719)
* Remove unneeded script

* Rename Substrate Demo -> Substrate

* Rename demo -> node

* Build wasm from last rename.

* Merge ed25519 into substrate-primitives

* Minor tweak

* Rename substrate -> core

* Move substrate-runtime-support to core/runtime/support

* Rename/move substrate-runtime-version

* Move codec up a level

* Rename substrate-codec -> parity-codec

* Move environmental up a level

* Move pwasm-* up to top, ready for removal

* Remove requirement of s-r-support from s-r-primitives

* Move core/runtime/primitives into core/runtime-primitives

* Remove s-r-support dep from s-r-version

* Remove dep of s-r-support from bft

* Remove dep of s-r-support from node/consensus

* Sever all other core deps from s-r-support

* Forgot the no_std directive

* Rename non-SRML modules to sr-* to avoid match clashes

* Move runtime/* to srml/*

* Rename substrate-runtime-* -> srml-*

* Move srml to top-level
2018-09-12 11:13:31 +02:00