Commit Graph

65 Commits

Author SHA1 Message Date
Tomasz Drwięga e729dbabbe [offchain] Support for sign & verify for crypto keys (#3023)
* Implement sign & verify.

* Use phrases and password.

* Sign & verify with authority keys.

* Fix tests.

* WiP

* WiP

* Allow the caller to decide on 'CryptoKind'.

* Remove TODO.

* Make seed private back.

* Fix non-std build and bump version.

* Use Into<u32> instead of asses.

* Add missing typedef.
2019-07-09 17:09:14 +02:00
Bastian Köcher fe08221479 Build WASM binaries as part of cargo build (#2868)
* Introduce `wasm-builder` and `wasm-builder-runner` to retire `build.sh`

Make use of `wasm-builder` in `test-runtime`.

* Add build script and remove the wasm project

* Port `node-runtime` to new wasm-builder

* Make `substrate-executor` tests work with `wasm-builder`

* Move `node-template` to `wasm-builder`

* Remove `build.sh` :)

* Remove the last include_bytes

* Adds the missing build.rs files

* Remove `build.sh` from CI

* Debug CI

* Make it work in CI

* CI attempt 3

* Make `substrate-runtime-test` compile on stable

* Ahhh, some missed `include_bytes!`

* AHH

* Add suggestions

* Improve search for `Cargo.lock` and don't panic if it is not found

* Searching from manifest path was no good idea

* Make the `wasm-builder` source better configurable

* Expose the bloaty wasm binary as well

* Make sure to rerun WASM recompilation on changes in dependencies

* Introduce new `WASM_BUILD_TYPE` env and make sure to call `build.rs` on
changes to env variables

* Remove `build.sh` from READMEs

* Rename the projects

* Fixes CI

* Update lock file

* Fixes merge-conflict

* Apply suggestions from code review

Co-Authored-By: TriplEight <denis.pisarev@parity.io>

* Try to make windows happy

* Replace all back slashes in paths with slashes

* Apply suggestions from code review

Co-Authored-By: Pierre Krieger <pierre.krieger1708@gmail.com>

* Use cargo from `CARGO` env variable

* Fix compilation

* Use `rustup` for running the nightly build

* Make individual projects skipable

* Fix compilation

* Fixes compilation

* Build all WASM projects in one workspace

* Replace more back slashes!

* Remove `inlcude_bytes!`

* Adds some documentation

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* More review comments

* Update `Cargo.lock`

* Set license

* Apply suggestions from code review

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* More review comments + adds `TRIGGER_WASM_BUILD` env

* Fix doc tests

* Increase version + update README

* Switch crates.io version of `wasm-builder`

* Update README

* Switch to released version of `wasm-builder-runner`
2019-07-04 11:34:06 +02:00
Tomasz Drwięga 2217c1e9a1 Persistent Local Storage for offchain workers. (#2894)
* WiP.

* Implement offchain storage APIs.

* Change compare_and_set to return bool.

* Add offchain http test.

* Fix tests.

* Bump spec version.

* Fix warnings and test.

* Fix compilation.

* Remove unused code.

* Introduce Local (fork-aware) and Persistent storage.

* Fix borked merge.

* Prevent warning on depreacated client.backend

* Fix long lines.

* Clean up dependencies.

* Update core/primitives/src/offchain.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Update core/primitives/src/offchain.rs

Co-Authored-By: André Silva <andre.beat@gmail.com>
2019-07-02 20:11:06 +02:00
kaichao 3e2c77aa21 typo fix in core (#2987) 2019-07-01 19:33:38 +02:00
Bastian Köcher 37acb90847 Remove UserError and switch to error::Error (#2899)
* Remove `UserError` and switch to `error::Error`

* More cleanup

* Update core/executor/src/error.rs

Co-Authored-By: Sergei Pepyakin <s.pepyakin@gmail.com>

* Fix typo

* Update core/executor/src/allocator.rs

Co-Authored-By: Stanislav Tkach <stanislav.tkach@gmail.com>
2019-06-19 15:08:59 +02:00
Gavin Wood bda8641892 New sessions, kill consensus module (#2802)
* Draft of new sessions

* Reintroduce tuple impls

* Move staking module to new session API

* More work on staking and grandpa.

* Use iterator to avoid cloning and tuple macro

* Make runtime build again

* Polish the OpaqueKeys devex

* Move consensus logic into system & aura.

* Fix up system module

* Get build mostly going. Stuck at service.rs

* Building again

* Update srml/staking/src/lib.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Refactoring out Consensus module, AuthorityIdOf, &c.

* Refactored out DigestItem::AuthoritiesChanged. Building.

* Remove tentative code

* Remove invalid comment

* Make Seal opaque and introduce nice methods for handling opaque items.

* Start to use proper digest for Aura authorities tracking.

* Fix up grandpa, remove system::Raw/Log

* Refactor Grandpa to use new logging infrastructure.

Also make authorityid/sessionkey static. Switch over to storing
authorities in a straight Vec.

* Building again

* Tidy up some AuthorityIds

* Expunge most of the rest of the AuthorityKey confusion.

Also, de-generify Babe and re-generify Aura.

* Remove cruft

* Untangle last of the `AuthorityId`s.

* Sort out finality_tracker

* Refactor median getting

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Session tests works

* Update core/sr-primitives/src/generic/digest.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Session tests works

* Fix for staking from @dvc94ch

* log an error

* fix test runtime build

* Some test fixes

* Staking mock update to new session api.

* Fix build.

* Move OpaqueKeys to primitives.

* Use on_initialize instead of check_rotate_session.

* Update tests to new staking api.

* fixup mock

* Fix bond_extra_and_withdraw_unbonded_works.

* Fix bond_with_little_staked_value_bounded_by_slot_stake.

* Fix bond_with_no_staked_value.

* Fix change_controller_works.

* Fix less_than_needed_candidates_works.

* Fix multi_era_reward_should_work.

* Fix nominating_and_rewards_should_work.

* Fix nominators_also_get_slashed.

* Fix phragmen_large_scale_test.

* Fix phragmen_poc_works.

* Fix phragmen_score_should_be_accurate_on_large_stakes.

* Fix phragmen_should_not_overflow.

* Fix reward_destination_works.

* Fix rewards_should_work.

* Fix sessions_and_eras_should_work.

* Fix slot_stake_is_least_staked_validator.

* Fix too_many_unbond_calls_should_not_work.

* Fix wrong_vote_is_null.

* Fix runtime.

* Fix wasm runtime build.

* Update Cargo.lock

* Fix warnings.

* Fix grandpa tests.

* Fix test-runtime build.

* Fix template node build.

* Fix stuff.

* Update Cargo.lock to fix CI

* Re-add missing AuRa logs

Runtimes are required to know about every digest they receive ― they
panic otherwise.  This re-adds support for AuRa pre-runtime digests.

* Update core/consensus/babe/src/digest.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Kill log trait and all that jazz.

* Refactor staking tests.

* Fix ci runtime wasm check.

* Line length 120.

* Make tests build again

* Remove trailing commas in function declarations

The `extern_functions!` macro doesn’t like them, perhaps due to a bug in
rustc.

* Fix type error

* Fix compilation errors

* Fix a test

* Another couple of fixes

* Fix another test

* More test fixes

* Another test fix

* Bump runtime.

* Wrap long line

* Fix build, remove redundant code.

* Issue to track TODO

* Leave the benchmark code alone.

* Fix missing `std::time::{Instant, Duration}`

* Indentation

* Aura ConsensusLog as enum
2019-06-14 16:34:34 +02:00
Gavin Wood f4afdd2f0b Subkey supports 24-word phrases (#2827)
* Revamp crypto API and make seeds work better in subkey

* Final tweaks

* Update tests

* line spacing

* Avoid escapes in hex constants

* Fix build

* Another fix

* More fixes

* Minor nits
2019-06-13 11:06:30 +02:00
Bastian Köcher 4f888f34d3 Build LongestChain with TestClientBuilder (#2792)
* Switch to `TestClientBuilder` and support generating LongestChain

* Make sure test-client works without the wasm blob

* Use `TestClientBuilder` in more places
2019-06-05 13:45:18 +02:00
Bastian Köcher 6142f95611 Fix tons of warnings in newest nightly (#2784)
* Fix tons of warnings in newest nightly

* Fix sr-api-macro doc tests
2019-06-04 20:09:49 +02:00
Tomasz Drwięga 308ab4f269 Offchain-worker APIs stubs (#2615)
* WiP: HTTP Apis.

* Working on the API.

* Add docs, clean up the API.

* Expose ext_ stuff as well.

* Implement HTTP helpers for offchain sr-io.

* Remove HTTP stuff.

* Revert "Remove HTTP stuff."

This reverts commit 7cca029d6ae93c5849b50edfcc6d2c313ba3e5bf.

* HTTP apis.

* Additional offchain methods.

* Make it compile.

* Implement wasm-ext boundary of offchain methods.

* Add stubs for offchain stuff to prevent panics.

* Fix tests.

* Addres some more issues.

* Introduce typedef, use unsafe from_utf8

* Bump runtime version.

* Introduce error to distinguish deadline and io errors.

* Add local_storage_cas

* Some tests for offchain stuff.

* Address more grumbles.

* Fix tests compilation.

* Fix borked merge.

* Improve docs for expected return values from ext functions.

* Adding new sign/enrypt/decrypt APIs.
2019-05-31 09:33:44 +02:00
DemiMarie-parity c7d1204ce5 Introduce inherent digests (#2466)
* Introduce inherent digests

* Implement inherent digests

* fix silly error

* Implementation of inherent digests in BABE

All tests pass. There are still limitations:

1. The runtime strips out inherent digests, so BABE must re-add them.
2. The test runtime checks that it can re-compute all digests.  It
   can’t, so I had to comment out that test.

* Fix compilation and seal import

Seals were not imported correctly: the pre-digest was imported twice,
instead of both it and the seal being imported.  Also, other parts of
the code did not compile due to incomplete refactoring.

* Remove bogus assertion

* Fix testsuite compilation

* Remove unused import

* Fix compiler diagnostics

* Add inherent digest parameters to block constructors

This enforces that inherent digests are added first.

* Fixup Cargo.lock

* Fix build errors

* Re-add an incorrectly removed import

* Bump primitive-types version

* Update Cargo.lock

* Refactoring

* Use inherent digests for AuRa

They do reach the runtime, but get stripped.  I have not figured out
where.

* Fix compilation errors

* Fix compilation errors due to incorrect types

* Fix whitespace

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Add preamble

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Fix silly compile error

* Refactor pre-digest finding code into a separate function

* Remove unwanted assertion

It is too likely to bring down the entire blockchain.

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Use `find_pre_digest` after runtime, too

Also, use `Member` trait rather than rolling our own requirements.

Suggested-by: Tomasz Drwiega <tomasz@parity.io>

* Fix various warnings

mostly due to upgrading the dependency on `error_chain`.

* Pre-digests nearly complete

This nearly completes the implementation of pre-runtime digests.

* `Seal2` → `Seal` and fix test suite

* Try to fix the storage error

* Try to fix storage (again)

* Fix tests

* Hopefully finish pre-runtime digests

The key is to pass *only* the pre-runtime digests to the runtime.  The
others must be stripped out by `initialize_block`.

* Fix silly typo

* Fix another silly mistake

* Remove unnecessary filtering of BABE pre-digests

We no longer get duplicate BABE pre-digests, so if they appear, the
header should be rejected outright.

* Update Cargo.lock files

* Reformatting

* Fix silly typo in inherent digest code

Also, revert `error.rs` files that contained calls to the `error_chain!`
macro.

* Try to keep the runtime from stripping pre-digests

Currently runs into the “Storage root must match that calculated”
assertion.

* Don’t compute storage root until storage changes are done.

Also, fix a compilation error.

* Fix compile-time error

* Fix compilation errors

* Fix more compile errors

* Hopefully it compiles this time…

* Fix compilation and add docs

* Prevent BABE from adding duplicate pre-runtime digests

Found by comparing with the AuRa code.  I also did some refactoring.

* Respond to review and fix some warnings

* Delete some dead code introduced earlier

* More dead code goes away

* `ref mut` → `&mut`

* Respond to review and fix some warnings

* Fix compilation error

* Remove unneeded `HashT` type parameter

Suggested-by: Robert Habermeier <robert@parity.io>

* Remove spurious #[allow(deprecated)]

* Document inherent digest parameter to `build_block`

* Delete `Simple` trait

It wasn’t needed

* delete wrongly added files

* Fix trait bounds

* Digest serialization tests

I also did some reformatting and cleanup.

* Apply suggestions from code review

Reformatting

Co-Authored-By: André Silva <andre.beat@gmail.com>

* Swap two arguments to `propose` and `propose_with`

Also, remove some needless unsafe code.

* Remove bogus `#![allow(deprecated)]` annotations

With the removal of the deprecated `Seal` variant, these are not needed.

* Add a missing `#[allow(deprecated)]` in the AuRa tests

* Fix silly compile error

* Fix silly compiler error

RLS did not tell me that I hadn’t fixed `babe/lib.rs`, so I missed it.

* Fixes made automatically by Cargo
2019-05-29 14:13:38 +02:00
Stanislav Tkach 1df8ac9751 Use $crate inside of native_executor_instance and parameter_types macros (#2702)
* Use  inside of native_executor_instance macro

* Same for the parameter_types macro
2019-05-28 17:14:48 +02:00
Svyatoslav Nikolsky 549d9e1da1 Remove BlockNumber <-> u64 conversions from light-client related code (#2666)
* Remove As usage from CHT

* Remove As usage from CHT (continue)

* Restrict BN <-> int conversions in CT

* more BN <-> u64 conversions removed

* upd spec_version

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <github@gavwood.com>

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <github@gavwood.com>

* more grumbles

* fix last grumbles + compilation

* too long lines

* too long lines
2019-05-28 15:07:16 +02:00
Tomasz Drwięga c162fc5ff1 Expunge error-chain (feat. tomaka) (#2662)
* Remove error_chain

* Expunge error-chain from rpc and service.

* Expunge from transaction pool.

* Expunge from node/cli

* Expunge from keystore.

* Remove some boilerplate.

* Fix remaining stuff.

* Improve on deprecation message.

* Fix issues.

* Fix trnsaction pool tests.

* Fix the rest.

* Fix borked merge.

* Update lock
2019-05-24 10:35:31 +01:00
Bastian Köcher bad3ce4e17 Cumulus changes version 2 (#2313)
* ensure imbalances are properly accounted for (#2183)

* ensure imbalances are properly accounted for

* bump runtime version

* Update node/runtime/src/lib.rs

* implement contract events (#2161)

* implement contract events

* update runtime

* renaming

* update test code hash

* improve complexity details

* add deposit event base cost

* add test

* Revert "add deposit event base cost"

This reverts commit 58ec010c0f4f4f0e16935ad41da32aedd17a8c57.

* update test

* Revert "update test"

This reverts commit 6fe61a593ccf0d41f09a0b97472b28ed8751a999.

* Revert "Revert "add deposit event base cost""

This reverts commit 145e8a9bac15313a4c380aa66b94fd4d36fa3f6d.

* Fix format a bit

*  Replace Vec<u8> with [u8; 32] for contract storage key (#2184)

* Replace Vec<u8> with [u8; 32] for contract storage key

* Read storage keys from sandbox memory into fixed size buffer

* Increment `impl_version`

* Remove redundant Ok(()) and explicitly specify StorageKey buffer type (#2188)

* Switch to `derive(Encode, Decode)` for `Call` (#2178)

* Add some tests

* More tests

* Switch to `derive(Encode, Decode)` for `Call`

* Update lock files

* Simplify the macro cases

* Cache changes trie config in db storage (#2170)

* cache changes trie config in db storage

* Update core/client/db/src/lib.rs

Co-Authored-By: svyatonik <svyatonik@gmail.com>

* Update core/client/db/src/lib.rs

Co-Authored-By: svyatonik <svyatonik@gmail.com>

* Fix version check for renamed runtime api methods (#2190)

* Add feature to disable including the test-runtime wasm blob

* Enable `std` feature for `consensus_authorities`

* Implement `skip_initialize_block` and `initialize_block` for runtime api

* Add test and fixes bug

* Begin to implement support for passing the `ProofRecorder`

* Make sure proof generation works as intended

* Fixes tests

* Make `BlockBuilder` generate proofs on request.

* Adds `TestClientBuilder` to simplify creating a test client

* Add `include-wasm-blob` to `test-client` as well

* Make `test-client` compile without including the wasm file

* Disable more stuff in test-client without wasm

* Reorganize the re-exports

* Use correct bounds

* Update docs

* Update core/client/src/block_builder/block_builder.rs

Co-Authored-By: bkchr <bkchr@users.noreply.github.com>

* Extend test to actually generated proof

* Switch to enum for `skip_initialize_block`

* Some wasm files updates
2019-04-29 16:55:20 +02:00
Stanislav Tkach 0f02bed702 Update hex-literal version, simplify imports and remove unused dependencies (#2371) 2019-04-25 11:17:40 +02:00
Bastian Köcher f0862606b7 Forward port blake2 storage support (#2360)
* move storage maps to blake2_128 (#2268)

* remove default hash, introduce twox_128 and blake2

* use blake2_128 & create ext_blake2_128

* refactor code

* add benchmark

* factorize generator

* fix

* parameterizable hasher

* some fix

* fix

* fix

* fix

* metadata

* fix

* remove debug print

* map -> blake2_256

* fix test

* fix test

* Apply suggestions from code review

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* impl twox 128 concat (#2353)

* impl twox_128_concat

* comment addressed

* fix

* impl twox_128->64_concat

* fix test

* Fix compilation and cleanup some docs

* Apply suggestions from code review

Co-Authored-By: bkchr <bkchr@users.noreply.github.com>
2019-04-24 11:05:22 +02:00
Sergei Pepyakin 517746bd62 Check storage_key for validity (#2316)
* Intro `ChildStorageKey` for checked child keys

* Get rid of Into in Externalities trait

* Use Cow in ChildStorageKey

* Fix tests for state-machine.

* Clean

* child_storage_root always return a value

* Don't return Option from Ext::child_storage_root

* Return 42 in child_storage_root

* Return CHILD_STORAGE_KEY_PREFIX from trie id gen

* Bump spec and impl version.

* Require `:default:` in `is_child_trie_key_valid`

* Add `default:` prefix.

* Introduce `into_owned` for `ChildStorageKey`.

* Add documentation.

* Fix state-machine tests

* Remove outdated TODO

I check out with Emeric and he is ok with that

* child_storage_root is infailable

* Nit

* Move assert after check.

* Apply suggestions from @DemiMarie-parity

Co-Authored-By: pepyakin <s.pepyakin@gmail.com>

* Formatting nit in core/executor/src/wasm_executor.rs

Co-Authored-By: DemiMarie-parity <48690212+DemiMarie-parity@users.noreply.github.com>

* Formatting nits from @thiolliere
2019-04-23 18:14:45 +02:00
Tomasz Drwięga 18df051947 Add missing docs to prevent warnings. (#2254) 2019-04-11 14:54:58 +02:00
Bastian Köcher 6920b169cd Introduce original_storage and original_storage_hash (#2211)
Both functions will ignore any overlayed changes and access the backend
directly.
2019-04-04 16:56:16 +02:00
Tomasz Drwięga e2f5e40876 Initial: Offchain Workers (#1942)
* Refactor state-machine stuff.

* Fix tests.

* WiP

* WiP2

* Service support for offchain workers.

* Service support for offchain workers.

* Testing offchain worker.

* Initial version working.

* Pass side effects in call.

* Pass OffchainExt in context.

* Submit extrinsics to the pool.

* Support inherents.

* Insert to inherents pool.

* Inserting to the pool asynchronously.

* Add test to offchain worker.

* Implement convenience syntax for modules.

* Dispatching offchain worker through executive.

* Fix offchain test.

* Remove offchain worker from timestamp.

* Update Cargo.lock.

* Address review comments.

* Use latest patch version for futures.

* Add CLI parameter for offchain worker.

* Fix compilation.

* Fix test.

* Fix extrinsics format for tests.

* Fix RPC test.

* Bump spec version.

* Fix executive.

* Fix support macro.

* Address grumbles.

* Bump runtime
2019-03-25 23:22:11 +01:00
Bastian Köcher 990d368f0d Prerequisites for validate_block in Cumulus (#1926)
* Adds benchmark for direct/indirect wasm function calls

* Store the benchmark function pointer in a `Cell`

* Add some documentation

* Make function implementations exchangeable

* Add parachain stub

* Add macro for registering the `validate_block` function

* Make all functions replace-able by unimplemented

* Some more refactoring

* Adds tests for executing empty parachain block

* Work on a new test with empty witness data

* Don't exchange `ext_print_*` stuff

* Some cleanup and one more function for `validate_block`

* More tests and more functions

* Fixes after merging master

* Use `parity-codec` `derive` feature

* CHange implementation of `wasm-nice-panic-message`

* Move `parachain` stuff to cumulus

* Updated wasm files

* Integrate feedback

* Switch to `ExchangeableFunction` struct

* More fixes

* Switch to Cell and panic on multiple replaces

* Increase `impl_version`

* Fix shifting

* Make the API more verbose of `ExchangeableFunction`

* Increase `impl_version`
2019-03-14 21:29:12 +01:00
Gav Wood d7fcf5dc9d Overhaul crypto (Schnorr/Ristretto, HDKD, BIP39) (#1795)
* Rijig to Ristretto

* Rebuild wasm

* adds compatibility test with the wasm module

* Add Ed25519-BIP39 support

* Bump subkey version

* Update CLI output

* New keys.

* Standard phrase/password/path keys.

* Subkey uses S-URI for secrets

* Move everything to use new HDKD crypto.

* Test fixes

* Ignore old test vector.

* fix the ^^ old test vector.

* Fix tests

* Test fixes

* Cleanups

* Fix broken key conversion logic in grandpa

CC @rphmeier

* Remove legacy Keyring usage

* Traitify `Pair`

* Replace Ed25519AuthorityId with ed25519::Public

* Expunge Ed25519AuthorityId type!

* Replace Sr25519AuthorityId with sr25519::Public

* Remove dodgy crypto type-punning conversions

* Fix some tests

* Avoid trait

* Deduplicate DeriveJunction string decode

* Remove cruft code

* Fix test

* Minor removals

* Build fix

* Subkey supports sign and verify

* Inspect works for public key URIs

* Remove more crypto type-punning

* Fix typo

* Fix tests
2019-03-13 14:08:31 +01:00
Bastian Köcher 7927054f86 Update copyrights to 2019 (#1932) 2019-03-06 15:04:28 +01:00
Gav Wood a61c218cc3 Introduce Ristretto signing (#1730)
* first draft of ristretto crypto module #1685

* adds better comments and code-style

* remove the last evil unwrap

* remove a mistakenly committed lockfile

* add a fresh new lockfile --will probably need a manual merge later

* fix an invalid old test vector

* Wire in ristretto

* Update comment

* Fix use.

* new Signature type api alias to be compatible with substrate

* Add new keyring, fix node executor tests

* Bump version.

* Remove all hashes.

* Update core/primitives/src/sr25519.rs

Co-Authored-By: gavofyork <github@gavwood.com>

* Revert back to Ed25519 (until JS UI is ready)

* Fix test
2019-02-13 10:10:17 +01:00
Michael Müller de4bb87bea Replace unwrap with expect (#1684)
* Replace unwrap with expect

* Move expect to call sites

* Bubble errors up and trap

* Update wasm

* Update invalid pointer

* Remove test which makes problems in CI

* Check for underflow
2019-02-10 12:29:45 +01:00
Stanislav Tkach e60be1ad12 Migrate network, primitives and rpc to the 2018 edition (#1710) 2019-02-06 19:07:48 +01:00
Stanislav Tkach df685c9e53 Rename and use --max-heap-pages parameter (#1666) 2019-02-05 12:35:12 +01:00
Michael Müller bf49b00c90 Replace allocator with freeing-bump allocator (#1656)
* Link substrate issue tracker in panic

* Replace allocator with freeing-bump allocator

* Revert me: Panic on double allocate/free

* Revert me: Add shallow benchmark for a first impression

* Revert "Revert me: Add shallow benchmark for a first impression"

This reverts commit 5f0d4df39395eb55c9ec2d6fe1ed992533475fec.

* Revert "Revert me: Panic on double allocate/free"

This reverts commit a114df7d6cfc21d60af396ccca02e5c8205db6ce.

* Rename heap to FreeingBumpHeapAllocator

* Rename heap.rs to allocator.rs

* Use sandbox heap

* Move functions

* Move variables into constructor

* Revert "Move variables into constructor"

This reverts commit f46fa0d0cdf4ea97760ccce58003b0d33f433743.

* Remove unnecessary casts

* Add comment for new parameter

* Improve typing

* Move variables into constructor

* Avoid dynamic allocation

* Remove unused variables

* Revert "Link substrate issue tracker in panic"

This reverts commit 32dfa1d02bcf881d1d514a930fcc0fdf3c5f8e08.

In the meantime this was fixed in https://github.com/paritytech/substrate/pull/1667.

* Improve naming

* Only assert in debug mode

* Remove dynamic allocation
2019-02-04 16:23:12 +01:00
Bastian Köcher 4ea3ac1c4a Use absolute path in native_executor for Blake2Hasher (#1671) 2019-02-04 11:26:51 +01:00
Arkadiy Paronyan 6c341f4e9d Fixed panic handling (#1667) 2019-02-04 09:11:29 +01:00
Bastian Köcher ef4dc12a5d Don't panic if parameter can not be converted between node and native runtime (#1659)
* Don't panic if parameter can not be converted between node and native runtime

* FIxes after merge

* Use correct copyright year
2019-02-02 14:13:50 +01:00
Michael Müller aa5e0658f8 Fix consensus error between wasm and native (#1595)
* Decrease bucket size

A bucket size of 8192 bytes is quite large and it turned
out that this can exhaust the available heap space too
too quickly.

This is because even for allocating 1 byte a bucket of
8192 bytes is allocated/wasted.

* Return 0 if requested size too large

* Improve test

The test didn't use an offset when setting up the heap.
Hence the first successfully allocated pointer was
always `0`.

This is unfortunate since `0` is also the return value
when there is an error.

This lead to us not noticing that the test was failing,
because it did not distinguish between success and error.

* Revert to linear allocator
2019-01-30 14:37:32 +01:00
Benjamin Kampmann 15ae7cfef6 Document more TODOs as tickets (#1418)
Went through the TODOs, removed a bunch, which are outdated or nothing more than a regular comment, documented a bunch more as actual tickets and made them FIXMEs and unified their structure (`FIXME #TICKETNO DESC` for local tickets, `FIXME: DESC LINK` for external tickets) for easier in-editor support. Further more remove unnecessary remarks and related old code that I noticed in that instance.
2019-01-30 10:29:48 +01:00
Stanislav Tkach e6839d2d41 Migrate executor, xinherens, keyring and keystore to 2018 edition (#1585) 2019-01-29 14:30:07 +01:00
Gav Wood c132e84572 Additional externs for ecdsa recovery and keccak (#1583)
* Additional externs for ecdsa recovery and keccak

* Update core/executor/src/wasm_executor.rs

Co-Authored-By: gavofyork <github@gavwood.com>

* Update core/executor/src/wasm_executor.rs

Co-Authored-By: gavofyork <github@gavwood.com>

* Fix grumble

* Grumble.
2019-01-25 19:07:02 +00:00
Bastian Köcher 5be237030d Fix warnings in substrate (#1523)
* Fix warnings in substrate

* More warnings removed and wasm updated

* Fixes error on stable and grumbles

* Update wasm files

* Add links to the github issue for replacing `error-chain`
2019-01-24 16:10:07 +01:00
Michael Müller c2f7993d0f Fix wasm allocator (#1506)
* Better allocator for wasm (#1460)

* Add Arch Linux installation instructions

* Enable tracing heap size

* Extract heap

* Replace linear allocator with buddy allocator

* Fix test

The purose of this big block is for the tests
to test a failure when the block is too big
though. The improved buddy allocation strategy
results in this block fitting on the heap now.
Hence the increase.

* Get rid of memcpy in to_vec()

* fixup: Style and comments

* fixup: Split Linux instructions by distribution

To prevent misunderstandings of people executing both.

* fixup: Remove unnecessary types and code

* fixup: Make Pointers start from 1, remove some panics, code improvements

* fixup: Return 0 on errors

* fixup: Move loop to separate function

* fixup: Use FnvHashMap instead of HashMap

* fixup: Fix error handling

* fixup: Use current_size() instead of used_size()

* fixup: Fix and document allocation offset

* fixup: Remove unnecessary multiplication

* fixup: Fix comments

* fixup: Remove Arch installation instructions

* Revert "Fix test"

This reverts commit 292c177df8efaa4658293748a13b1ab1c0b76ea8.

* fixup: Remove unused code, improve import

* fixup: Proper alignment

* fixup: Do not use internal constant in public description

* fixup: Add comment regarding invariants

* fixup: Move assertion to compile-time check

* Fix bug in necessary tree level calculation

The tree levels necessary to house a number
of nodes was calculated incorrectly.

* Improve naming
2019-01-21 16:20:18 +01:00
Bastian Köcher 010e63116f Make runtime api calls native when possible (#1302)
* Add simple benchmark for the runtime api

* Make the executor support native calls

* Some documentation

* Hide behind `feature = "std"`

* Rework the native calls

* Make all tests compile again

* Make every parameter using the Block serialized/deserialized in the native call

* Forward `UnwindSafe` requirement

* Remove debug stuff

* Add some documentation

* Fixes warnings

* Fixes errors after master rebase

* Fixes compilation after master rebase

* Fixes compilation after rebase
2019-01-21 14:32:53 +01:00
Bastian Köcher aad569a1e6 Revert new wasm allocator (#1502)
* Revert "Better allocator for wasm (#1460)"

This reverts commit 1ccb590d18.

* Update wasm files
2019-01-21 12:52:44 +01:00
Michael Müller 1ccb590d18 Better allocator for wasm (#1460)
* Add Arch Linux installation instructions

* Enable tracing heap size

* Extract heap

* Replace linear allocator with buddy allocator

* Fix test

The purose of this big block is for the tests
to test a failure when the block is too big
though. The improved buddy allocation strategy
results in this block fitting on the heap now.
Hence the increase.

* Get rid of memcpy in to_vec()

* fixup: Style and comments

* fixup: Split Linux instructions by distribution

To prevent misunderstandings of people executing both.

* fixup: Remove unnecessary types and code

* fixup: Make Pointers start from 1, remove some panics, code improvements

* fixup: Return 0 on errors

* fixup: Move loop to separate function

* fixup: Use FnvHashMap instead of HashMap

* fixup: Fix error handling

* fixup: Use current_size() instead of used_size()

* fixup: Fix and document allocation offset

* fixup: Remove unnecessary multiplication

* fixup: Fix comments

* fixup: Remove Arch installation instructions

* Revert "Fix test"

This reverts commit 292c177df8efaa4658293748a13b1ab1c0b76ea8.

* fixup: Remove unused code, improve import

* fixup: Proper alignment

* fixup: Do not use internal constant in public description

* fixup: Add comment regarding invariants

* fixup: Move assertion to compile-time check
2019-01-18 17:29:47 +01:00
Gav Wood c9f047fe84 Split Indices module from Balances (#1404)
* Indices module

* Remove indices stuff from balances

* Rejob node, move Lookup into system.

* Fix up some modules.

* Fix democracy tests

* Fix staking tests

* Fix more tests

* Final test fixes

* Bump runtime versions

* Assets uses compact dispatchers

* Contracts module uses indexed addressing

* Democracy has more compact encoding

* Example now demonstrates compact eencoding

* Sudo uses indexed address

* Upgrade key also uses indexed lookups

* Assets more compact types.

* Fix test

* Rebuild runtime, whitespace

* Remove TOODs

* Remove TODOs

* Add a couple of tests back to balances.

* Update lib.rs

* Update lib.rs
2019-01-16 15:57:19 +01:00
Arkadiy Paronyan b104c02eb6 State cache and other performance optimizations (#1345)
* State caching

* Better code caching

* Execution optimizaton

* More optimizations

* Updated wasmi

* Caching test

* Style

* Style

* Reverted some minor changes

* Style and typos

* Style and typos

* Removed panics on missing memory
2019-01-08 13:13:13 +01:00
kyegupov 11f77464fb Improved test for conditional panic (#1320) 2018-12-23 20:22:45 +01:00
Bastian Köcher 1f6719346f Make runtime api generate version and identifier information (#1226)
* Make `decl_runtime_apis!` implement `RuntimeApiInfo` for all runtime
apis

* Make the runtime side generate the info constants as well

* Make `RuntimeApiInfo` implementation use the correct generics

* Adds a test for the runtime api info stuff

* Remove duplicated code by using block from `test-client`

* Adds `compile_fail` tests for `api_version`

* Adds documentation for `api_version`

* Make `impl_runtime_apis!` generate `RUNTIME_API_VERSIONS`

* Update documentation and tests for `RUNTIME_API_VERSIONS`

* Implement `has_api` by using the `RuntimeApiInfo`

* Make `impl_runtime_apis` check that trait identifiers are unique

* Prefix all runtime api function with the corresponding trait

So `execute_block` will be called `Core_execute_block`.
This makes it possible to have traits implement a function with the
same name.

* Rebase master

* Update after master rebase
2018-12-10 11:48:07 +01:00
Svyatoslav Nikolsky 6e3c56c135 Changes tries: support forks (#950)
* forks support in changes trie storage

* moved convert_hash to primitives

* optimize ChangesTrieRootsStorage::root when anchor is canonicalized
2018-11-13 12:17:14 +01:00
Luke Schoen 037f9dde10 docs: Extends #999. Remove disused package readmes. Rename to sr-metadata. Fix Readme Rustdoc names (#1016)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme

* docs: Remove link to old docs

* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names

* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory

* Change example to use package name that opens all packages in RustDocs

* Fix list of packages names that are available to open in RustDocs

* fix typo

* fix ordering of CLI options and add missing backslash
2018-11-12 21:18:06 +01:00
Hero Bird 660c747c51 Update fixed-hash & uint utilities (#1078)
* [core/primitives] Update crunchy 0.1 => 0.2

* [core/primitives] Update fixed-hash 0.2 => 0.3.0-beta

* [core/primitives] Update `uint` 0.4 => 0.5.0-beta

* [core/state-machine] Fix errors emerged by fixed-hash update

* [core/state-db] Fix errors that emerged from fixed-hash update

* [core/sr-io] Fix errors that emerged from the fixed-hash update

* [core/trie] Fix errors emerged from fixed-hash updates

* [core/trie] Make use of new Hash::as_fixed_bytes_mut method in tests

* [core/sr-primitives] Fix errors emerged from updating fixed-hash

* [core/executor] Fix errors that emerged from fixed-hash update

* [core/test-runtime] Fix errors that emerged from updating fixed-hash

* [core/test-runtime] Fix an error that emerged from fixed-hash update

* [core/transaction-pool] Fix errors that emerged from updating fixed-hash

* Add From<u64> impl for hash types defined in core/primitives

* [core/client/db] Fix errors that emerged from fixed-hash update

* [core/{client/network/rpc}]: Fix errors emerged by fixed-hash update

* [node/{cli/executor/runtime}]: Fix errors emerged by updating fixed-hash

* [core/network]: Fix bug in parsing constant str

* Update Cargo.lock

- Add crunchy 0.2.1
- fixed-hash 0.2.2 => 0.3.0-beta.3
- Add static-assertions 0.2.5
- uint 0.4.1 => 0.5.0-beta.1

* [core/primitives]: Add fixed-hash/rustc-hex feature for FromStr impl

* [core/primitives] No longer provide From<u64> impl for hash types if byteorder support is not enabled

* [core/primitives] Revert to using From impl again in primitives tests

* [core/..] Fix some bugs that emerged by recent fixed-hash updates

* Update a bunch of Cargo.lock files

* [core/state-db] Re-add whitespace between attr and extern crate/mod

* [core/primitives] Fix bug in From<u64> impl for hash types using the wrong feature guard
2018-11-06 15:39:55 +01:00
guanqun c0f7021427 Remove pub in macros (#1027)
* use :vis in macro_rules!

* use :vis in meta
2018-10-27 12:15:15 +02:00
Luke Schoen f923e38cc7 docs: Relates to Issue #995. Fix links in existing documentation (#999)
* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core
2018-10-26 11:09:48 +02:00