Commit Graph

28 Commits

Author SHA1 Message Date
cheme d9cffa0bb5 Code redundancy between ext implementation and testing. (#3830)
* fix child_storage_hash

* extract common implementation for ext and testing

* cleaning impl.

* replace ExtBasisMut by actual Ext

* remove extbasis.

* Update tests to use Ext from test externalities.

* use Ext constructor for getting ext from TestExternalities.

* Add missing extensions from ext.

* fix wasmi test

* Fix merge error.
2019-10-18 09:52:25 +02:00
Bastian Köcher 81c400e958 Make wasmi_execution public to use it from tests (#3829)
* Make `wasmi_execution` public to use it from tests

* Make `WasmRuntime` accessible as well

* Add `call_in_wasm` instead of making stuff public

* Use `WasmRuntime`

* Move test

* More feedback
2019-10-16 21:00:31 +02:00
Bastian Köcher 8a39be474e Move Externalities into its own crate (#3775)
* Move `Externalities` into `substrate-externalities`

- `Externalities` now support generic extensions
- Split of `primtives-storage` for storage primitive types

* Move the externalities scoping into `substrate-externalities`

* Fix compilation

* Review feedback

* Adds macro for declaring extensions

* Fix benchmarks

* Introduce `ExtensionStore` trait

* Last review comments

* Implement it for `ExtensionStore`
2019-10-09 15:50:30 +02:00
Jim Posen 6cebbbf8b2 Refactor NativeExecutor to support multiple Wasm execution methods (#3677)
* executor: Move definitions of externals out of wasm_executor module.

* executor: Create WasmRuntime trait.

This will be used to decouple the runtime cache from wasmi execution.

* executor: Remove WasmExecutor and move methods to wasmi_execution.

These will now be crate-internal functions and there is no need
for the struct.

* executor: Set default default_heap_pages in NativeExecutor.

* cli: CLI configuration for Wasm execution method.

* executor: Remove wasmi-specific code from wasm_runtime.

* Respond to review comments.
2019-10-08 12:57:12 +02:00
Bastian Köcher 9607afd629 Cleanup of the state-machine crate (#3524)
* Start refactoring state-machine crate

* More improvement to state-machine

* Fix tests compilation on master and remove warnings

* Fix compilation

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Update core/state-machine/src/basic.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Line width

* Update core/primitives/src/storage.rs

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Update core/state-machine/src/error.rs

Co-Authored-By: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Review feedback
2019-09-10 17:00:00 +02:00
Bastian Köcher 1450719acc First step for generating host externals out of the function definition in sr-io (#3567)
* Adds new wrapper traits for wasm executor

* Add new crate `substrate-wasm-interface`

Thew new crate holds types and traits for the communicating between the
wasm runtime and the host.

* Rewrite externals with new macro etc

* Fix vec initialization

* Make executor tests working

* Remove unused code + warnings

* Introduce `Pointer` and `WordSize` for working with wasm

* Fix tests and compilation

* Fix compilation

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <sergei@parity.io>

* Review feedback

* Remove unused conversions

* Make each host function its own struct

`HostFunctions` now just returns these function structs. Each function
can be executed by using one of the function structs. The inherent host
functions are now moved to the "normal" host functions.

* Remove byteorder

* Add floating point types

* Make pointer interface more safe

* Add type alias for wasm-interface Result

* More review comments
2019-09-10 16:07:25 +02:00
Sergei Pepyakin 4f7e36e072 Update docs for the wasm executor crate (#3569)
* Grammar.

* Update docs for wasm executor.

* Update core/executor/src/lib.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>
2019-09-06 23:08:08 +02:00
thiolliere 4ed67e03a4 Update to parity-scale-codec (#3232)
* WIP: update codec

* WIP

* compiling

* WIP

* rename parity-scale-codec to codec

* WIP

* fix

* remove old comments

* use published crates

* fix expected error msg

* bump version

* fmt and fix

* remove old comment

* fix wrong decoding impl

* implement encode like for structures

* undo removal of old pending changes

* trailingzeroinput

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* update codec

* fmt

* version is 1.0.0

* show more error

* fmt
2019-08-06 19:36:23 +02:00
Sergei Pepyakin af914e9f40 Fair reusing of wasm runtime instances (#3011)
* Add test from original bug report

Original is from @pepyakin in 3d7b27f3421818e8d6de568e02fbc2947a06246b.
I adapted it to work with the latest master.

* No longer cleanup module instance

* Replace runtime cache with synchronous clone

* Fix test

* Preserve initial runtime memory and restore it on fetch

* Remove leftover comment

* Fix style

* Improve variable naming

* Replace get_into() with get()

* Handle missing memory export better

* Return earlier when creating runtime first time

* Improve comments

* fmt

* Fix #2967.

* Eradicate `code` from `Error::InvalidCode`

* tidy

* A state snapshot doc.

* Store multiple runtimes by hash.

* Get rid of deref.

* Docs

* Use Self for instantiate_module

* REVERT ME

* Should be ok

* Commit

* Remove dbg

* Use fast-memory's erase

* Clean and undo hacks.

* Introduce a dedicated error for heap_base

* Ban the start function.

* Clean, docs and refactor

* Add rustflags.

* Update Cargo.lock

* Apply Basti's suggestions

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Rename allocates_huge_stack_array

* Extend TestClientBuilder with set_heap_pages

* Update the test.

* Update core/executor/src/wasm_executor.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/executor/src/wasm_runtimes_cache.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/executor/src/error.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update core/executor/src/error.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix tests.

* Update cargo-lock

* Use wasmi master

* Use master wasmi

* Move tests.

* Use wasmi crates.io

* Update Cargo.lock

* Fix build.rs

* Bump runtime version

* Revert initial_heap_pages renaming

* Bump wasmi up to 0.5.0

* Bump runtime version

* Don't restore an instance every now and then

* Update core/executor/src/wasm_runtimes_cache.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Propagate error in CacheError

* Clarify the get_heap_base call in instantiation

* Supply --export=__heap_base

See https://reviews.llvm.org/D62744

Co-authored-by: Jim Posen <jim.posen@gmail.com>

* Bump version.

* Use combinators for segments.

* Fix build.rs

* Fix build.rs for runtime-test
2019-07-25 16:01:08 +03:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Stanislav Tkach e60be1ad12 Migrate network, primitives and rpc to the 2018 edition (#1710) 2019-02-06 19:07:48 +01:00
Michael Müller bf49b00c90 Replace allocator with freeing-bump allocator (#1656)
* Link substrate issue tracker in panic

* Replace allocator with freeing-bump allocator

* Revert me: Panic on double allocate/free

* Revert me: Add shallow benchmark for a first impression

* Revert "Revert me: Add shallow benchmark for a first impression"

This reverts commit 5f0d4df39395eb55c9ec2d6fe1ed992533475fec.

* Revert "Revert me: Panic on double allocate/free"

This reverts commit a114df7d6cfc21d60af396ccca02e5c8205db6ce.

* Rename heap to FreeingBumpHeapAllocator

* Rename heap.rs to allocator.rs

* Use sandbox heap

* Move functions

* Move variables into constructor

* Revert "Move variables into constructor"

This reverts commit f46fa0d0cdf4ea97760ccce58003b0d33f433743.

* Remove unnecessary casts

* Add comment for new parameter

* Improve typing

* Move variables into constructor

* Avoid dynamic allocation

* Remove unused variables

* Revert "Link substrate issue tracker in panic"

This reverts commit 32dfa1d02bcf881d1d514a930fcc0fdf3c5f8e08.

In the meantime this was fixed in https://github.com/paritytech/substrate/pull/1667.

* Improve naming

* Only assert in debug mode

* Remove dynamic allocation
2019-02-04 16:23:12 +01:00
Bastian Köcher 4ea3ac1c4a Use absolute path in native_executor for Blake2Hasher (#1671) 2019-02-04 11:26:51 +01:00
Michael Müller aa5e0658f8 Fix consensus error between wasm and native (#1595)
* Decrease bucket size

A bucket size of 8192 bytes is quite large and it turned
out that this can exhaust the available heap space too
too quickly.

This is because even for allocating 1 byte a bucket of
8192 bytes is allocated/wasted.

* Return 0 if requested size too large

* Improve test

The test didn't use an offset when setting up the heap.
Hence the first successfully allocated pointer was
always `0`.

This is unfortunate since `0` is also the return value
when there is an error.

This lead to us not noticing that the test was failing,
because it did not distinguish between success and error.

* Revert to linear allocator
2019-01-30 14:37:32 +01:00
Stanislav Tkach e6839d2d41 Migrate executor, xinherens, keyring and keystore to 2018 edition (#1585) 2019-01-29 14:30:07 +01:00
Gav Wood c132e84572 Additional externs for ecdsa recovery and keccak (#1583)
* Additional externs for ecdsa recovery and keccak

* Update core/executor/src/wasm_executor.rs

Co-Authored-By: gavofyork <github@gavwood.com>

* Update core/executor/src/wasm_executor.rs

Co-Authored-By: gavofyork <github@gavwood.com>

* Fix grumble

* Grumble.
2019-01-25 19:07:02 +00:00
Michael Müller c2f7993d0f Fix wasm allocator (#1506)
* Better allocator for wasm (#1460)

* Add Arch Linux installation instructions

* Enable tracing heap size

* Extract heap

* Replace linear allocator with buddy allocator

* Fix test

The purose of this big block is for the tests
to test a failure when the block is too big
though. The improved buddy allocation strategy
results in this block fitting on the heap now.
Hence the increase.

* Get rid of memcpy in to_vec()

* fixup: Style and comments

* fixup: Split Linux instructions by distribution

To prevent misunderstandings of people executing both.

* fixup: Remove unnecessary types and code

* fixup: Make Pointers start from 1, remove some panics, code improvements

* fixup: Return 0 on errors

* fixup: Move loop to separate function

* fixup: Use FnvHashMap instead of HashMap

* fixup: Fix error handling

* fixup: Use current_size() instead of used_size()

* fixup: Fix and document allocation offset

* fixup: Remove unnecessary multiplication

* fixup: Fix comments

* fixup: Remove Arch installation instructions

* Revert "Fix test"

This reverts commit 292c177df8efaa4658293748a13b1ab1c0b76ea8.

* fixup: Remove unused code, improve import

* fixup: Proper alignment

* fixup: Do not use internal constant in public description

* fixup: Add comment regarding invariants

* fixup: Move assertion to compile-time check

* Fix bug in necessary tree level calculation

The tree levels necessary to house a number
of nodes was calculated incorrectly.

* Improve naming
2019-01-21 16:20:18 +01:00
Bastian Köcher aad569a1e6 Revert new wasm allocator (#1502)
* Revert "Better allocator for wasm (#1460)"

This reverts commit 1ccb590d18.

* Update wasm files
2019-01-21 12:52:44 +01:00
Michael Müller 1ccb590d18 Better allocator for wasm (#1460)
* Add Arch Linux installation instructions

* Enable tracing heap size

* Extract heap

* Replace linear allocator with buddy allocator

* Fix test

The purose of this big block is for the tests
to test a failure when the block is too big
though. The improved buddy allocation strategy
results in this block fitting on the heap now.
Hence the increase.

* Get rid of memcpy in to_vec()

* fixup: Style and comments

* fixup: Split Linux instructions by distribution

To prevent misunderstandings of people executing both.

* fixup: Remove unnecessary types and code

* fixup: Make Pointers start from 1, remove some panics, code improvements

* fixup: Return 0 on errors

* fixup: Move loop to separate function

* fixup: Use FnvHashMap instead of HashMap

* fixup: Fix error handling

* fixup: Use current_size() instead of used_size()

* fixup: Fix and document allocation offset

* fixup: Remove unnecessary multiplication

* fixup: Fix comments

* fixup: Remove Arch installation instructions

* Revert "Fix test"

This reverts commit 292c177df8efaa4658293748a13b1ab1c0b76ea8.

* fixup: Remove unused code, improve import

* fixup: Proper alignment

* fixup: Do not use internal constant in public description

* fixup: Add comment regarding invariants

* fixup: Move assertion to compile-time check
2019-01-18 17:29:47 +01:00
Arkadiy Paronyan b104c02eb6 State cache and other performance optimizations (#1345)
* State caching

* Better code caching

* Execution optimizaton

* More optimizations

* Updated wasmi

* Caching test

* Style

* Style

* Reverted some minor changes

* Style and typos

* Style and typos

* Removed panics on missing memory
2019-01-08 13:13:13 +01:00
Luke Schoen 037f9dde10 docs: Extends #999. Remove disused package readmes. Rename to sr-metadata. Fix Readme Rustdoc names (#1016)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme

* docs: Remove link to old docs

* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names

* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory

* Change example to use package name that opens all packages in RustDocs

* Fix list of packages names that are available to open in RustDocs

* fix typo

* fix ordering of CLI options and add missing backslash
2018-11-12 21:18:06 +01:00
Luke Schoen f923e38cc7 docs: Relates to Issue #995. Fix links in existing documentation (#999)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core
2018-10-26 11:09:48 +02:00
Bastian Köcher 12f12bff28 Make all copyrights use the correct year (#894)
* Adds a script for automatically update all copyrights

* Ran the `update-copyright` script
2018-10-10 10:28:41 +02:00
guanqun 35f3fe7e77 use the map! macro instead of defining it all over the place (#867)
* remove one map implementation in test

* remove one map macro

* remove one map macro

* remove one map macro
2018-10-03 13:22:58 +01:00
Bastian Köcher 2332d6dd04 More dependency cleanup (#825)
* Removes `rlp` dependency

* Enable warnings about unused crates in the CI

* Remove all the unused crates

* Make tests working again
2018-09-27 17:52:51 +01:00
Arkadiy Paronyan 29c9719568 Support authoring for multiple runtime versions (#816) 2018-09-27 13:56:40 +01:00
Gav Wood 82d6ca3484 Switch to shiny new fast, RLP-less trie (#795)
* Bump codec

* Fix tests

* Patricia trie builds

* Introduce trie

* Some yak shaving.

* Some fixes

* Remove RLP ref

* Fixes

* It builds!

* Some tests fixed

* Another test fix

* Rejig more hashes

* substrate-trie::iterator_works test

* Update lock

* Polish

* Docs

* Undo incorrect "fix" for tests

* Fix nits
2018-09-25 16:32:22 +02:00
Gav Wood 1e01162505 Phase 1 of repo reorg (#719)
* Remove unneeded script

* Rename Substrate Demo -> Substrate

* Rename demo -> node

* Build wasm from last rename.

* Merge ed25519 into substrate-primitives

* Minor tweak

* Rename substrate -> core

* Move substrate-runtime-support to core/runtime/support

* Rename/move substrate-runtime-version

* Move codec up a level

* Rename substrate-codec -> parity-codec

* Move environmental up a level

* Move pwasm-* up to top, ready for removal

* Remove requirement of s-r-support from s-r-primitives

* Move core/runtime/primitives into core/runtime-primitives

* Remove s-r-support dep from s-r-version

* Remove dep of s-r-support from bft

* Remove dep of s-r-support from node/consensus

* Sever all other core deps from s-r-support

* Forgot the no_std directive

* Rename non-SRML modules to sr-* to avoid match clashes

* Move runtime/* to srml/*

* Rename substrate-runtime-* -> srml-*

* Move srml to top-level
2018-09-12 11:13:31 +02:00